Angoff method

What is the Angoff Method?

The Angoff method is a statistical technique used to set cut scores, the minimum scores required to pass a test or evaluation. It is named after William H. Angoff, who developed the method in the 1970s.

The basic idea behind the Angoff method is to ask a group of subject matter experts to consider a hypothetical group of test takers and estimate what percentage of those test takers would be able to pass the test if they were to take it. Those experts are asked to consider the test takers’ level of knowledge and skills and the difficulty of the test questions. Based on these estimates, a cut score is determined that will ensure a certain percentage of test takers pass the test.

The Angoff method is often used to set cut scores for professional licensing exams, such as the bar exam for lawyers or the medical licensing exam. It is also used in education to set passing scores for high stakes tests, such as state-mandated assessments.

One advantage of the Angoff method is that it involves the input of subject matter experts who deeply understand the content and skills being tested. This can help ensure that the cut score is fair and reflects the knowledge and skills necessary for success in a particular field. However, the Angoff method also has some limitations, such as the subjectivity of the experts’ estimates and the potential for bias in their evaluations.

How does the Angoff Method work?

The Angoff method involves several steps. First, the test developer selects a panel of subject matter experts who are representative of the examinee population for the test. The panel members should thoroughly understand the content and skills the test intends to assess.

Next, the panel members are given a set of test items and asked to evaluate each item’s difficulty. They are asked to consider the level of difficulty and the content of the items and to assign a probability based on their expert judgment of the likelihood that a minimally qualified examinee would correctly answer the item.

Once the panel members have assigned probabilities to all test items, each item’s mean probability score is calculated. This mean probability score represents the panel’s consensus judgment of an item’s difficulty.

The final step in the Angoff method is to use the mean probability scores to set the cut score for the test. This is done by calculating the total number of points needed to pass the test, based on the mean probability scores of the items. The cut score is then set at the point where a minimally qualified examinee is expected to correctly answer a sufficient number of items to pass the test.

Angoff in Cirrus

With this feature, our users can streamline the cut-score setting process and increase its efficiency, accuracy, and transparency. A user-friendly interface will allow experts to enter their estimates, and provide a system for aggregating and analysing data, and setting final cut-scores based on the expert estimates.

Imagine you have 3 items in your test form, and all 3 items have a max score of 1 point. De Angoff values set for these items are: 0.2, 0.5 and 0.8.
The passmark points (i.e. what score the candidates must achieve to pass in order to successfully answer the question) are:
1 x 0.2 = 0.2
1 x 0.5 = 0.5
1 x 0.8 = 0.8
The cut score, or passmark as it's called in Cirrus are automatically calculated, and is 0.2 + 0.5 + 0.8 = 1.5. In other words, the candidates must achieve at least 1.5 points in order to pass the exam.

Activating Angoff

In order to activate Angoff and be able to set the Angoff values in Cirrus go to 'Admin > Global settings' Under the general tab > Item difficulty scale you can enable Angoff.

Set Angoff values

Once activated a new column 'Angoff 'appears under the 'Library > Collection > Statistics' which allow you to set an Angoff value with a 3 decimal entry.

Setting cut-score Angoff

In order to set a cutscore you'll need to make sure all your questions in the assessment form containing a angoff value. Once this is done it will automaticly set the cut-score and it's passmark, see image below.

In this example it means that the candidate needs 9.238 or more to pass, the passmark percentage is 54.34% (54%) (9.238 / 17 * 100%)

In other words, a difficult exam would give a lower cut-score than an easy exam. In the above example, if all items had an Angoff value of 1 (very easy), the cut score would be 6. You would thereby need a 100% score to pass the exam.

A great use case for Angoff is if you use Random forms. In case some candidates get a more easy or difficult form, the cut-score is automatically adjusted for this, as to not allow advantages of getting easier questions.