• Knowledgebase
• Dark
Light
• PDF

# Using polytomous scoring rules

Article Summary

This article explains the difference between Dichotomous and Polytomous scoring in Cirrus.

When questions in Cirrus allow for multiple answers, you can apply Dichotomous and Polytomous scoring rules:

• Dichotomous:the candidate must answer everything correct to get the max score. When answered partially correct, the candidate will score 0.
• Polytomous:gives a partial score when the candidate answers the question partially correct.
Auto-scored items only
These rules only applies to auto-scored items with more then one answer. For example Multiple Response questions Fill in the Blank, Select from List and many more.

### Examples with and without polytomous scoring

Let's take a look at some questions that support multiple correct answers.

These examples clarify the difference between polytomous and dichotomous scoring rules.

#### Order

Picture a simple Order question with 4 items to put in the correct order. The example has a maximum score of 4.

Question Type: Order (4 alternatives)DichotomousPolytomous
no value correct00
1 value is correct 01
2 values are correct02
4 values are correct44

For more complex questions like Multiple Response (MR) or Extended Match however, the amount of options a candidate can select affect the scoring model used.

#### Multiple Response (MR)

When using polytomous scoring on MR, the author is offered two scoring models. These are based on the number of answer options the candidate can select:

1. The number of answers to select is equal to the correct options
2. The number of answers to select has no limit.

You will find these options using the Options tab, when authoring a MR question.

Polytomous with option 1

In this scenario the candidate 'knows' how many options will be correct. Candidate will receive partial scoring for each option selected correctly.

Consider the following example question:

Select the 2 cities with the most tourist visiting in 2018

• Barcelona
• Paris
• Amsterdam
• New York
• Moscow
Question Type: Multiple ResponseMR (polytomous - limited)
Number of alternatives5
Number of correct alternatives2
Number of incorrect alternatives3
Max score2
Score per correct alternative1

Determining the candidate's score:

• 2 points are divided across 2 correct alternatives
• So points per correctly selected option 2/2 = 1

Polytomous with option 2

When using polytomous scoring together with 'no limit' the candidate does not know the number of correct options to select.

He is awarded points for:

• Avoiding incorrect alternatives. I.e. you are also awarded points for not selecting incorrect alternatives.

Consider the following example question:

Which cities had the most tourists in 2018?

Question Type: Multiple ResponseMR (polytomous - no limit)
Number of alternatives5
Number of correct alternatives2
Number of incorrect alternatives to avoid3
Max score2

Determining the candidate's score:

• In this case 2 points are divided across 5 alternatives. Meaning each alternative is valued at 0.4 point.
• So points per correctly selected answer + avoided alternative: 2/5 = 0.4.

Say your candidate selected 2 alternatives but only 1 is correct. This means the candidate selected 1 alternative correctly and avoided 2 incorrect alternatives.

Result: this candidate scores 3*0.4=1.2

#### Extended Match (ExtMatch)

When using polytomous scoring for ExtMatch, the author is offered two scoring models. Both based on the number of answer options the candidate can select:

1. The number of answers to select is equal to the correct options
2. The number of answers to select has no limit.

You will find these options using the Options tab, when authoring an ExtMatch question.

Option 2 is selected by default
Because option 2 is selected by default, candidates will also get points for avoiding incorrect answers (see below). Switch to option 1 if this is not desirable.

Polytomous with option 1

In this scenario the candidate 'knows' how many options will be correct. He will receive partial scoring for each option selected correctly.

Consider the following example question:

Connect the options on the left with the correct numbers on the right

 Even 2 Uneven 3 Prime 4 6 7

Question Type: Extended Match (polytomous - limited)Total
Number of option 'left' 3
Number of options 'right' 5
Number of correct connections3 (even) + 2 (uneven) + 3 (prime)8
Max score 4
Score per correct alternative=4/80.5

Determining the candidate's score:

• 4 points are divided across 8 correct alternatives
• So points per correctly selected connection: 4/8 = 0.5.

Polytomous with option 2

For ExtMatch questions in Cirrus using polytomous with 'no limit' the candidate does not know the number of correct options to select. He is awarded points for:

• Selecting the correct alternatives
• Avoiding incorrect alternatives, i.e. you are also awarded points for not selecting incorrect alternatives.

Consider the same example question:

Connect the options on the left with the correct numbers on the right

 Even 2 Uneven 3 Prime 4 6 7

Question Type: Extended Match(polytomous - no limit)Total
Number of option 'left' 3
Number of options 'right' 5
Number of correct connections3 (even) + 2 (uneven) + 3 (prime)8
Number of incorrect connections= 2 + 3 + 2 7
Total number of connections= 5 + 5 + 5 15
Max score 4
Score per (in)correct alternative= 4/150.267

Determining the candidate's score:

This means that for determining the candidate's score 5 points are divided across 15 alternatives. This means each alternative is valued at 4/15.
In other words: points per correctly selected / avoided alternative: 4/15 =0.26667.

#### Hotspot

Consider a simple Hotspot question with 4 items to click. Do realise this gives a number of answer options:

• 4 hotspots clicked
• 3 hotspots clicked, one not clicked
• 2 hotspots clicked, two not clicked
• 1 hotspot clicked, three not clicked
• No hotspots clicked

The candidate will be scored on correctly checked options and avoided incorrect options. With a maximum score of 5 points this leads to the following scoring table:

 Dichotomous Polytomous nothing selected 0 0 4/4 incorrect - no value correct 0 0 1/4 value is correct (3 are incorrect) 0 1 1/4 value is correct (0 are incorrect) 0 2 2/4 values are correct / 2 incorrect 0 2 2/4 values are correct / 0 incorrect 0 3 3/4 values correct / 1 incorrect 0 3 3/4 values correct / 0 incorrect 0 4 4 values are correct 5 5

#### Taking 'result by chance' into account

For many question types candidates could opt to just simply guess the correct answer. This is why many organisations take the chance of guessing (result by chance) into account. This can be done during scoring and subsequently in determining a 'pass' or 'fail' or grade.

Example

Picture an assessment with 80 MC4 questions (i.e. only Multiple Choice questions with four answer options). Your scoring / assessment scale should take into account the chance to score of your questions.

80 MC4 questions means that a candidate has a 1 in 4 chance of simply guessing the answers correctly. Meaning on average 0,25*80=20 points can be scored by simply guessing!

What does this mean for the pass mark?

Let's assume the uncorrected pass mark is set at 50%. This means you need to score 50% of 80 points = 40 points to pass.

But when taking result by chance into account, you can adjust this. For example:

• Score 20 points + 50% of the remaining points > 20 + (0.5*60) points) = 50 points out of 80 to pass.

Now your percentage to pass mark is adjusted to 62,5% to avoid chance score.

#### Some useful notes:

• You set the pass mark when creating an assessment Form, using the 'Options-tab'.