Views:

Introduction

We often get questions about how the partial scoring works with Multiple Select questions in Brillium Assessment Builder. It can be confusing because the results are often unexpected, but make sense once you understand how the system is calculating partial credit and why it is done that way. Below is a description of how the calculation is done with examples to help illustrate.

Example Question: Let’s take the following question that is worth a total of 10 points. The correct response is shown.

Multiple Select Sample Question - Select Fruits

How Partial Credit Scoring Works

To score a partial credit multiple select question, Brillium looks at each checkbox and determines whether that checkbox should be checked or unchecked. If the respondent, checks a box that should be checked, they get credit. If the respondent leaves a checkbox unchecked and it should be unchecked, they get credit. Otherwise, if a checkbox is checked that shouldn’t be or vice-versa, the respondent will not get credit.
Based on this method, Brillium determines what percentage of the question the respondent has received credit for. That percentage is multiplied by the number of available points for the question, and the respondent will receive that number of points for their assessment final score calculation.
So, the calculation for scoring looks like this:
(Num Checkboxes Correct / Total Num Checkboxes) x Question Point Value

Example 1

Assume “Sally” answers the question as follows:

Multiple Select Question Example 1

Analysis:

Sally’s answer is 80% correct (4 out of 5 answers are checked or unchecked correctly – the answer “orange” is incorrect, as it should be checked). Since the question is worth 10 points, Sally will receive 8 out of the 10 available points towards her final score for the assessment.

Example 2

Assume “John” answers the question as follows:

Multiple Select Question Example 2

Analysis:

John’s answer is 20% correct (1 out of 5 answers are checked or unchecked correctly – the answer “apple” is the only correctly indicated checkbox, all others are incorrect). Since the question is worth 10 points, John will receive 2 out of the 10 available points towards his final score for the assessment.

Rounding of Question Points

It is also important to note that Brillium does not award fractions of a point for questions. Points are assigned in whole numbers. Traditional rounding rules are applied (< 0.5 is rounded down, >= 0.5 is rounded up).

Let’s now assume our question is worth 2 points (instead of 10). Sally would receive 2 out of the 2 points (as 80% of 2 is 1.6, which would be rounded up to 2 points). John would receive 0 points (as 20% of 2 is 0.4, which would be rounded down to 0 points). 

Why Does Partial Credit Work Like This?

A valid question might be: "Why don’t we just determine if the answers that should be checked are checked and leave it at that (without considering answers that should be left unchecked)?" 

The problem with that methodology (and others) is people can easily cheat the system if they figure out how the scoring works. If we are just checking for checked boxes, then a savvy respondent could simply check all the checkboxes and be correct 100% of the time.

Keywords: partial scoring, multiple select questions, partial credit, partial credit on multiple select