Updating Question or Answer Content on Active Assessments

Overview

After an assessment goes live and respondents start completing it, you may find questions in the assessment are causing confusion and require corrections or edits due to their wording. You may also find that you've accidentally marked the wrong answer as correct. In these situations, Brillium allows you to make the necessary edits, but you may want to consider how to deal with respondents having already completed the assessment.

When making that consideration, here are some options:

  1. Do nothing. Simply make edits to the content and leave the existing results alone. This is what happens by default (i.e. no further action required). The existing respondents' scores are unchanged. Future respondents will be scored taking into account the newly made edits.
  2. Manually give credit. With this option, you have total control over which respondents' scores will be affected by your edit. It is recommended you run a report (for example, the Respondent Row/Column Analysis Report) to determine which respondents are affected by the edits you have made. Then you can determine how you would like to proceed and which respondents you would like to make adjustments to. From there, you may add/remove whole percentage points from those respondents individually using the Respondent Edit page.
  3. Request an automated re-score. You may send an email or support ticket to Brillium Support to request a "re-score" be run on your assessment. A re-score is a process we run behind-the-scenes on your specified assessment. It loads each respondent individually and applies the newly updated assessment content settings (i.e. any edits you have made) to determine the respondent's new score based on those edits. This can move your respondents' scores up or down and can also change their pass/fail status in either direction.

Removing a Question Altogether

In some situations, you may determine the best course of action is to remove a question from an assessment altogether. You may accomplish this by editing the question and changing its Group to "UNASSIGNED QUESTIONS". This is a special group that exists on all assessments. It is a place holder group for questions that should not be presented to future respondents. By placing a question in this group, existing respondents will not be affected, but future respondents (i.e. those starting the assessment after the change was made) will not see this question on their assessment.

If you intend on requesting a re-score after moving one or more questions to the unassigned questions group, you may wish to also set the point value of those moved questions to 0 (zero). This will effectively remove those questions from the score calculation for those respondents. In this situation, respondents scoring correctly on those questions may see their score go down, and respondents scoring incorrectly on those questions may see their score go up.

Re-Scoring an Assessment

If you should decide to request a re-score of your assessment, there are some things you will need to know. First, a re-score is performed by support personnel and may take up to 1-2 business days. We try to get to them quickly, but it is possible they could take that amount of time.

When a re-score is run, it generates a report log to be emailed back to you by the support person handling your request. This report log will contain entries for each respondent having a score or pass/fail status change. You will see the respondent's primary key (email address by default), full name (if being captured by the assessment), prior score, new score, prior pass/fail status, and new pass/fail status. If no updates are required to a respondent, they will not appear in the log file.

Which Edits Affect a Respondent's Score and Which Do Not?

These types of content edits WILL likely affect a re-score:

  • Changes to the assessment's passing score.
  • Changes to a question group's passing score or passing requirement.
  • Changes to a question's point value.
  • Placing a question in a different question group.
  • Changes to the correct answer of a question.
  • Changes to the "Partial Credit" setting for questions supporting it.

These types of content edits WILL NOT affect a re-score:

  • Wording edits in the question group, question, or answer text fields.
  • Changes to options like "Shuffle Answers", "Allow Comments", etc.
  • Changes to reference ID, resources, explanation text, or author's notes.
  • Moving a question to "Unassigned Questions" without also changing its point value.

Options When Requesting a Re-Score

When you request a re-score, you may elect the following options:

  • Preview Only. This option runs the re-score process and generates the report log, but doesn't actually change the scores of any respondents. It is intended to give you an idea of what will change once the re-score is actually run. This can help you decide if you want to run the re-score process or if you would like to use any of the further options below.
  • Adjust Up Only. This option runs the re-score but prevents any respondent's score from going down. Only changes where the score is going up will be made. It also prevents any pass/fail status from going from pass to fail.
  • Most Recent Attempt Only. This option ensures only the most recent completed result is re-scored for any given respondent. For example, if a respondent completes the assessment three times, only the third attempt will be re-scored.

Another option is to give existing respondents credit for one or more specific questions regardless of how they answered them. This is a seldom used option, but notify us if it is required in your particular situation.

Effects of Running a Re-Score

Once a re-score has been run, you will most likely see the following happen to the results of your respondents:

  • Affected respondents will have their overall score and pass/fail status adjusted either up or down based on their performance being scored against the current assessment's settings.
  • Respondents will have their individual question group scores and pass/fail status adjusted up or down, where necessary.
  • When viewing a report where question-level data is shown, you will see the new correct/incorrect indicators and points earned when a question was moved from an incorrect status to a correct status and vise-versa.
  • Aggregate reports (like the Question Basic Analysis Report) will be calculated with the newly updated respondents' scores rather than their old scores prior to the re-score.

How to Request a Re-Score

To request a re-score, open a support ticket or send an email to Brillium Support. Please remember to include which organization you are from, the Assessment ID you would like the re-score to be run on, and any options you wish to elect (i.e. "Preview Only", "Adjust Up Only", or "Most Recent Attempt Only").

 

Have more questions? Submit a request

Comments

0 comments

Please sign in to leave a comment.