There is a lot riding on the outcome of a high-stakes English language proficiency test.
Recruiters from Higher Education Institutions (HEIs) and immigration officials alike need to feel confident that tests are conducted in a secure environment. And test takers want to know that, if they have a query or need assistance during the test, there’s someone there to help them.
This is why language tests have traditionally been taken at test centers overseen by trained invigilators, also known as proctors.
Online proctoring is not new in assessment – but it is new in language testing. The rise of remote working, compounded by recent lockdowns, has led to increased acceptance of online proctoring by Higher Education Institutions that had previously been suspicious of remote methods of invigilation.
We speak to Lilac Peterson, Product Manager for Skills for English. She tells us more about the challenges she faced prior to launching Skills for English via online live proctoring and how she and her team overcame them.
We also answer the following questions:
⦁ What is online proctoring?
⦁ What is the purpose of online proctoring?
⦁ What are the different approaches to online proctoring?
⦁ How does online proctoring work?
⦁ What is the future of online proctoring?
What is online proctoring?
Online proctoring is the use of secure software and online identity verification to enable candidates and students to take tests remotely. Online proctoring means invigilating tests remotely using specialized technology, training and processes to establish a secure environment in the home.
It has evolved over the past few years in language testing to be recognized as a credible form of online examination.
What is the purpose of online proctoring?
Some educational institutions are still concerned about the integrity of online testing and the rise of malpractice.
The integrity and reputation of online proctoring rests on the security of the software and the testing procedure that accompanies the test;
As a consequence, all forms of online proctoring need to address the following challenges:
⦁ Identity verification
⦁ Environmental security
⦁ Fraud prevention
⦁ How to deal with violations during the test
Most language assessment companies now offer online proctoring but they approach it in different ways. It depends on how they balance the need to ensure the test is secure with a concern for the quality of the ‘test taker experience.’
Sometimes these needs conflict with each other, hence the variety of methodologies each testing company employs.
What are the different approaches to online proctoring?
Some companies have chosen to prioritize security, meaning test takers are watched by a visible online proctor throughout their test, or the test taker can hear the proctor. This can make the experience feel overbearing and unsettling for the test taker.
In an effort to avoid being intrusive, some companies avoid live proctoring altogether. Instead, test takers are videoed throughout their test and AI tools are employed to check for potential violations.
For PSI Bridge, Lilac and the team have taken a more balanced approach. PSI Bridge seeks to adhere to the highest levels of security whilst providing “a reassuring, enjoyable experience for the test taker at the same time.”
A language test with PSI Bridge comprises three different roles:
⦁ The Test Taker
⦁ The Check-in Specialist
⦁ The Proctor
The Test Taker is obviously the person who is taking an online language test, either to gain entry into a Higher Education Institution or for other language demonstration purposes.
The Check-in Specialist is trained to ensure that the test environment is secure and that the Test Taker is who they say they are. PSI Bridge ensures there is in most cases a one-to-one relationship between the Test Taker and the Check-in Specialist.
The Proctor is the person who invigilates each test. They are trained to look for signs of security violation, content piracy or fraud during a test and to respond accordingly. PSI Bridge can manage a ratio of up to 8 to 1 between Proctor and Test Taker, but the normal ratio is 4 to 1.
How does online proctoring work?
Most of us are familiar with taking tests under exam conditions, either at school, university or at a dedicated test center.
To recreate the same conditions in a remote context, the online proctoring sector has developed innovative new technology and processes. A language test that is proctored online by PSI Bridge consists of the following steps:
1. Identity verification
2. Room scan
3. Technical setup
4. Taking the test
5. Data forensics
This section describes how each of these steps addresses the competing challenges of security and test taker experience at the same time.
Step 0: Before the test
Online tests are booked a few days days to weeks in advance. Test takers can register and pay for a test online via PSI’s test taker portal and will receive emails confirming the time and date and outlining what they can expect from the online proctoring process.
On test day, the test taker can download the PSI Bridge secure browser, which test takers are proctored through, and launch that online test environment up to thirty minutes before the test start time.
Step 1: Technical Setup
To take a test with PSI Bridge, the test taker must launch the software by downloading and running an application on their computer. The application does not take over the test taker’s entire screen, but instead auto-detects any other applications that are running on their computer and will typically auto-terminate all of them. The test is taken through the secure browser application, and that view will cover the test taker’s entire screen.
Once the software is downloaded, live video monitoring will begin. The video footage will also be available to review internally by PSI staff.
If the test taker attempts to re-open any prohibited programmes, or if they try to view anything other than the PSI Bridge screen during the test, it will be detected and prevented. If they do this persistently, the proctor may terminate the test.
Step 2: Identity Verification
Identity verification of the test taker is overseen by the check-in specialist. The test taker cannot begin the exam without completing the following tasks:
1. Take a selfie via their webcam, which will then be uploaded for the check-in specialist to review to ensure the face matches the face on the ID check in step 2
2. The test taker must also present government-issued photo ID and hold it up to the camera for photo capture
3. The check-in specialist must be able to clearly read the information on the Photo ID to ensure it is valid, unexpired, and that the test taker’s legal name matches their registration details
Check-in specialists are trained to know the types of ID that are valid in the test taker’s region as well as the common types of ID fraud.
If the name on the photo ID does not match the registered name for the test, or the check-in specialist cannot see the photo ID clearly enough to be confident the names match, the test taker will not be able to proceed with the test on that occasion.
Once a test taker’s ID has been secured, the check-in specialist can proceed to securing the environment.
Step 3: Live Room Scan
One of the biggest challenges for any organization that offers online proctoring is ensuring that a test taker’s home environment provides the same integrity as a test center.
The proctor needs to be certain that the test taker is not cheating during a test but also that they are not trying to record or pirate the test itself.
PSI Bridge ensures environmental security via a combination of process and technology. The room scan process, also overseen by the check-in specialist, ensures that the space around the test taker has been checked carefully.
The room scan is conducted live and is recorded. Using their computer’s camera, the test taker must show the check-in specialist the ceiling, the space around, behind and in front of them, as well as their desk and the space underneath it.
There are strict rules about what a test taker can (and cannot) have on their desk. They are only allowed one monitor. They cannot use headphones or wear anything on their wrists. They also must place their phone out of reach and then the check-in specialist will ask the test taker to show via webcam where it has been placed to ensure it is not within reach. They are allowed a beverage and chewing gum, but cannot have food.
Once the check-in specialist is comfortable that the test taker’s environment is secure, the test taker will see a message that the test is being set up, and will then transfer to the proctor.
PSI Bridge uses live online proctors during their tests. This means that the proctor, using the PSI Bridge proctor portal, can see the test taker at all times via the test taker’s web camera, which will be turned on throughout.
This doesn’t mean, however, that the test taker can see the proctor or that the proctor communicates with the test taker with their own voice.
Lilac says, “Our team made an explicit decision NOT to allow the test taker to see or hear the proctor. Having someone on screen watching you can feel intrusive and distracting to the test taker, who needs to be able to concentrate on the exam and be in a relaxed frame of mind.”
PSI Bridge supports communication between the test taker and the proctor via a chat tray that can be opened on the right-hand side of the screen at all times.
The chat tray is the main channel through which the proctor communicates with the test taker. It is designed to be unobtrusive and friendly and set the test taker at ease. If a test taker requires technical assistance, the proctor can request technical support to join the proctor chat directly.
If the test taker prefers live phone assistance, however, under the Live Chat button, Skills for English has two toll-free lines for test takers to call for customer or technical assistance if needed throughout their entire experience.
Proctors are trained to empathize with the test-taker, who may well be nervous during the test. Proctors take pains to only communicate with test takers when absolutely necessary.
“A lot of time and care has been invested in making the test taker’s experience as lightweight and smooth as possible,” says Lilac.
Her team is sensitive to the fact that test takers are non-native English speakers and that messages relayed by the proctor must be clear and free of ambiguous language.
Step 4: Taking the Test
Once the test taker has begun the test, the online proctor adheres to a rigorous set of standards throughout.
The test taker must be visible in front of the screen and on camera at all times. If they get up and walk out of view without informing the proctor first, this would constitute a “major violation” and the proctor will likely terminate the test.
The appearance of anyone other than the test taker on camera is also regarded as suspicious, especially if there is someone in clear view behind the test taker, which could be constituted as an attempt at content piracy or unfair assistance to the test taker. Then the same “major violation” will be issued and will usually result in test termination.
Test takers are not allowed to take breaks and leave their computer. If circumstances are exceptional and legitimate, the proctor can allow breaks, but the screen will be covered by a privacy banner, and upon return, the test taker will need to complete the environmental security check (i.e. live room scan) a second time.
In an online context, technical disruptions are unavoidable and PSI Bridge always tries to give the test taker the benefit of the doubt if, for instance, the internet connection happens to dip for a few seconds. If test takers need technical assistance at any time during the test, they can ask for it via the chat or by calling one of the dedicated support numbers. If a technical issue prevents the test from being completed that day, test takers will be assisted by PSI Candidate Care via email or phone for a timely reschedule.
Step 5: Data forensics
PSI Bridge does not use AI technology during a test – all proctoring is human. After tests, however, PSI makes extensive use of data forensics as an additional security measure after a test has been completed.
They use forensic algorithms to search for patterns in the data of each test that may indicate fraudulent behavior. Data Points that are analyzed include:
- Time taken on each question
- Similarity of response to other tests
The insights from data forensics are constantly fed back to the team and inform their security decision-making processes.
The future of online proctoring
Online proctoring has come a long way in a short period of time. Lilac and her team have invested a lot of time and effort to improve the test taker experience. Today, test takers will have an experience that is more lightweight and empathetic to their needs.
Concerns about online privacy have also been mitigated by rigorous processes of identity verification and environmental security reassuring officials from HEIs and other institutions that online language testing is a credible means of examination.
If you’re interested in learning more about Lilac’s journey building the online proctoring experience for Skills for English product, check out this webinar.
