Angoff method
  • 07 Apr 2023
  • 3 Minutes to read
  • Contributors
  • Dark
  • PDF

Angoff method

  • Dark
  • PDF

Article Summary

What is the Angoff Method?

The Angoff method is a statistical technique used to set cut scores, the minimum scores required to pass a test or evaluation. It is named after William H. Angoff, who developed the method in the 1970s.

The basic idea behind the Angoff method is to ask a group of subject matter experts to consider a hypothetical group of test takers and estimate what percentage of those test takers would be able to pass the test if they were to take it. Those experts are asked to consider the test takers’ level of knowledge and skills and the difficulty of the test questions. Based on these estimates, a cut score is determined that will ensure a certain percentage of test takers pass the test.

The Angoff method is often used to set cut scores for professional licensing exams, such as the bar exam for lawyers or the medical licensing exam. It is also used in education to set passing scores for high stakes tests, such as state-mandated assessments.

One advantage of the Angoff method is that it involves the input of subject matter experts who deeply understand the content and skills being tested. This can help ensure that the cut score is fair and reflects the knowledge and skills necessary for success in a particular field. However, the Angoff method also has some limitations, such as the subjectivity of the experts’ estimates and the potential for bias in their evaluations.

How does the Angoff Method work?

The Angoff method involves several steps. First, the test developer selects a panel of subject matter experts who are representative of the examinee population for the test. The panel members should thoroughly understand the content and skills the test intends to assess.

Next, the panel members are given a set of test items and asked to evaluate each item’s difficulty. They are asked to consider the level of difficulty and the content of the items and to assign a probability based on their expert judgment of the likelihood that a minimally qualified examinee would correctly answer the item.

Once the panel members have assigned probabilities to all test items, each item’s mean probability score is calculated. This mean probability score represents the panel’s consensus judgment of an item’s difficulty.

The final step in the Angoff method is to use the mean probability scores to set the cut score for the test. This is done by calculating the total number of points needed to pass the test, based on the mean probability scores of the items. The cut score is then set at the point where a minimally qualified examinee is expected to correctly answer a sufficient number of items to pass the test.

Angoff in Cirrus

With this feature, our users can streamline the cut-score setting process and increase its efficiency, accuracy, and transparency. A user-friendly interface will allow experts to enter their estimates, and provide a system for aggregating and analysing data, and setting final cut-scores based on the expert estimates.

Imagine you have 3 items in your test form, and all 3 items have a max score of 1 point. De Angoff values set for these items are: 0.2, 0.5 and 0.8.
The passmark points (i.e. what score the candidates must achieve to pass in order to successfully answer the question) are:
1 x 0.2 = 0.2
1 x 0.5 = 0.5
1 x 0.8 = 0.8
The cut score, or passmark as it's called in Cirrus are automatically calculated, and is 0.2 + 0.5 + 0.8 = 1.5. In other words, the candidates must achieve at least 1.5 points in order to pass the exam.

Activating Angoff

In order to activate Angoff and be able to set the Angoff values in Cirrus go to 'Admin > Global settings' Under the general tab > Item difficulty scale you can enable Angoff.
Screenshot 2023-04-07 at 09.20.41.png

Set Angoff values

Once activated a new column 'Angoff 'appears under the 'Library > Collection > Statistics' which allow you to set an Angoff value.

Screenshot 2023-04-07 at 09.25.10.png

Roadmap Angoff method

This is the first step in our introduction of Angoff. We will continue with allowing you to use Angoff values to (automatically) set cut scores on assessment forms level.

Was this article helpful?