AI Essay Grading Enhancements - Workflow Discussion

Through our development and release of AI Essay grading, it became very apparent that humans and machines approach essay reading and review quite differently. We are enhancing the workflows to better accommodate how humans work alongside AI Essay grading, and we are seeking input from our user community to help shape the workflow.

Basically, the workflow methods are:

• Manual Grading - Humans grade assessments as a whole, question items are reviewed in succession (typically), and the workflow is sequential with rubrics being referenced by the graders they work with.
• Validate Grade (AI + Manual Review) - AI grades the assessment questions individually, then awaits acknowledgement and review by a human grader who reviews questions individually OR sequentially. Rubric items that are flagged are often the only ones referenced. The process and workflow are largely left up to the customer and user case. This results in a wide range of workflows, especially when only one rubric is used, as was the previous limitation with essay question functionality.
• Automatic Grade (AI Only) - With AI grading essays, each question is graded individually. Sequence is not important, and AI flags any issue or anomaly, which is a very different workflow compared to human grading.

Each use case has a specific level of nuance, so we would like to hear from our community what general methods they use or plan to use and why. Also, what that workflow looks like in your environment. We hope this will lead to further discussion that everyone can benefit from.

Note: We are not asking anyone to displace anything proprietary, just general information that may help us better accommodate your workflows.