Tracking course health with Data Insights

Running lab courses takes a lot of work. Lab coordinators and instructors are responsible for administering a course, often to hundreds or thousands of students.

In addition to managing students, you also are often managing a team of graduate teaching assistants (TAs), professional instructors, and even undergraduate teaching assistants to instruct, grade, and evaluate students.

How do you ensure that you students are getting regular, graded feedback? How do you make sure that one group of students is not getting either implausibly high or low marks?

In short: How do you know that your course is in good health?

What is Data Insights?

At Catalyst Education our answer to this question is Data Insights.

Data Insights is a suite of always up-to-date, interactive data visualizations that display important analytics about your course.

Insights are provided on demand about your graders and sections, including:

Grading Status: Course and report level grading progress
Score Distribution: Report and quiz score distributions
Score History: Report and quiz scores across time
Item Analysis: Report and quiz level assessment properties

Grading Status

Tracking grading and making sure that students are getting regular feedback on their work is a top concern of running any lab course. Lags in grading are an early indicator that things are not running as intended.

When you need a bird’s-eye-view of grading, Grading Status shows you how much grading needs to be done on reports and the progress of each of your graders.

Score Distribution

When activities have been graded, the next thing we want to know is how students’ grades are distributed. Outliers or anomalies in scores can tell you whether something requires your attention.

Score Distribution shows you both a distribution of all students’ scores (histogram; left) and also breaks that distribution down for each TA/section (interquartile ranges; right).

Extreme high and low median scores as well as overly wide score ranges for subgroups of students will stand out clearly to you and help you determine whether something needs to be done.

Score History

When you need to see patterns in report and quiz scores at the course level, Score History gives you insight into grading trends over time.

A graph of median report and quiz scores over the entire term lets you spot grading bias as it happens over multiple activities. Catching these issues earlier not only ensures a better experience for your students, but should ease the amount of work required at the end of the semester when finalizing grades.

Item Analysis

Measuring what students know and what they’ve learned requires having well-calibrated assessments. Reports and quizzes that ask questions that are too easy or too hard make it difficult to know where students need instructional reinforcement.

Item Analysis is designed to apply principles from Classical Test Theory (CTT) to give you feedback on the functioning of your assessments. We analyze each students’ first graded attempt and report the accuracy and discrimination of each item on your assessments.

These statistics let you evaluate whether your assessments are properly aligned to your students and measuring what they know.

Questions that are too hard/easy or questions that have poor/bad discrimination are extra work for students and provide you with little instructional insight. Item Analysis lets you see those more clearly.

Ask us about Labflow & Data Insights!

Catching a problem before it becomes problematic is the most effective way to save yourself time. Our mission is to make the work of running a laboratory course easier.

Data Insights brings you timely information about the health of your course so you can catch issues before they bubble over into meetings and emails that take up your time.

If you would like to schedule a demo, please book an appointment today. If you have questions or would like to see new features added to Data Insights, please contact dane@catalystedu.com.