The design of the onSET

The onSET is a standardised language placement test in C-test format. It measures general language proficiency in German and English – quickly, accurately, and commensurably. Participants are required to fill in gaps in texts on various topics linguistically correctly, each text containing 20 gaps created by the removal of parts of words. The difficulty increases from text to text whilst maintaining the same overall level; each test-taker receives a personalised set of texts. Participants receive one point for each gap filled correctly. The total score is assigned to a CEFR level (A2 to C1) once the test has been completed.

Quality is assured through a rigorous editing process: Experienced authors research suitable texts. Subsequently, g.a.s.t. experts review the texts through the perspectives of test theory, linguistics and native language proficiency. Only tasks that successfully pass all review stages are approved for pilot testing. Regular professional dialogue between all those involved ensures consistent quality.

Every newly created C-Test text is carefully tested before deployment in onSET. At least 200 language learners from onSET’s target group take part in the piloting; the evaluation is carried out centrally by g.a.s.t. No text is deployed without rigorous vetting.

Testing takes place on an ongoing basis in collaboration with licensed g.a.s.t. test centres. Further information on trials can be found here.

New items undergo comprehensive statistical testing using psychometric methods, including Rasch models (mathematical measurement models that map item difficulty and test-taker ability onto a common scale.) This process assesses the severity of a task, the accuracy of measurement, and the discrimatory power.  Only items that meet the quality criteria are included in the item bank. 

By the same token, g.a.s.t. uses the Differential Item Functioning (DIF) method to analyse whether a task is equally fair to all participants, irrespective e.g. of language of origin or gender. These analyses also form the basis for determining the cut-off scores (thresholds that determine which CEFR level a test result is assigned to). The cut scores are defined using scientifically established methods, including the prototype group method and ROC analysis.

The onSET uses the LOFT method (Linear-on-the-Fly Testing — task-based real-time compilation): each test-taker receives an individually compiled set of tasks, selected from the item bank according to the difficulty and subject matter of the texts. The overall level of difficulty is the same for everyone. Combined with automatic marking, this method ensures commensurability of the results and an objective and fair assessment for all participants.