AI-Assisted Grading: A Partner, Not A Replacement

At Catalyst, we believe that AI should amplify human expertise, not replace it. Our AI-assisted grading system is designed with a "Human-in-the-Loop" philosophy. You remain the architect of the assessment; the system simply acts as a tireless teaching assistant that evaluates your criteria consistently across every student submission and provides suggestions to your grader(s).

How It Works: A Two-Stage Process

To ensure the system grades exactly how you want it to, we utilize a transparent two-stage workflow that relies on your input and review.

Stage 1: Enhancement (The Setup)

When you click "Enhance," our system analyzes your specific grading criteria, the question prompt, and the context of the assignment (all of the content in that topic block). It uses this information to build a detailed, transparent ‘enhanced’ grading rubric.

  • Your Input Matters: The system takes your original grading criteria (grading directions and rubric) as its "jumping off point". The clearer your instructions are, the better the system can align with your expectations.

  • Context Aware: The system can even look at specific variables from earlier in the assignment—such as data measurements students entered earlier—to ensure their current answers are consistent with their own previous work.

Stage 2: Review & Refine (The Control)

Before a single student is graded, you have full visibility. You can view the newly generated enhanced rubrics to ensure they meet your assessment goals.

  • Adjusting the Criteria: If the enhanced rubric isn't quite right, you can simply click "Revise Question." A wizard will walk you through adjusting the prompt, rubric, and/or grading directions to refine the system's instructions. You can even do this after students have started, if you realize, based on their answers, that the rubric wasn't quite right.

  • Final Say: Even after grading begins, you (or your TAs) review the system's suggestions. You can accept them, override them, or add your own feedback at any time.


What Can AI Grade?

We are constantly testing and vetting new question types. Below is the current list question types that have been rigourously validated by our team of teaching experts, and are ready for use.

1. Essay Questions (Text-Based)

The system is highly capable of grading text-based responses, ranging from simple definitions to complex arguments.
Key Feature: "Looking Back" at Prior Work One of our most powerful features is Contextual Grading. The system can evaluate an essay based on work the student completed earlier in the assignment. Examples:

  • Data Consistency: If a student enters measurement data in Part A, the system can evaluate their Error Analysis essay in Part B based on their specific numbers, not just a generic answer key.

  • Hypothesis Tracking: If a student writes a hypothesis early on, the system can evaluate their conclusion to see if it logically addresses that specific hypothesis.

  • Detailed Answers: One area in which the system excels is processing detailed student answers - where graders can get weary. For example, the system will consistently evaluate grading student definitions of psychology terms, regardless of how many students.

  • Note: The system cannot currently "look back" at file uploads (e.g., it cannot grade an essay by referencing an image uploaded earlier in the assignment). This applies to grading file uploads as well as to grading essays.

2. File Uploads (Images & Handwritten Work)

We have thoroughly tested and approved three specific types of file uploads for AI-assisted grading:

  • Handwritten Calculations: "Show your work" questions where students upload images of their math. And just as with essays, the system can look back at data entered earlier in the report and evaluate whether the handwritten math is consistent with their values entered earlier.

  • Chemical Structures: Drawings of single chemical structures, including Lewis structures or VSEPR models

  • Simple Images: Drawings, photos, or images of simple objects.

    • Advisory: While the system is excellent at reading text within an image, we are still testing its ability to interpret scientifically complex diagrams. For now, stick to simpler image tasks.

Tips for Best Results

  • Clarity over Brevity: When writing your grading directions, be explicit. Instead of writing "Grade for correctness," try "Grade based on whether the student identifies the unknown compound consistent with their observations in Part A".

  • Define Your Strategy: Are you grading for Accuracy (did they get the right answer?) or Consistency (did their logic follow their data, even if the data was flawed?)? Specifying this in your directions helps the system create the perfect rubric.

  • Trust but Verify: The system provides a justification for every score it suggests. Use these justifications to ensure the system is performing as you expect and that your graders are maintaining the standards set by the rubric.