Essay-Checking Software
I have been looking at a software system from ETS called Criterion, which is an online essay-evaluation program. In other words, students write essays in the system in response to a large bank of prompts. The artificial intelligence in the software compares the submitted work and generates a detailed feedback report. More importantly, it generates and immediate feedback report, not next week after the teacher has spent a frustratingly short weekend grading 120 papers. It also offers the possibility of students writing more essays and getting more feedback than any overworked teacher could ever provide.
The biggest hurdle is the deep, gut feeling many people have that a computer could never do a good job of evaluating something like an essay. A new research study discussed today in an article in the New York Times (free registration required) looks at the use of the software in Indiana, where it is being used as part of the 11th grade testing statewide. A two-year pilot study had student work graded by both the program and by real teachers. The results showed virtually no difference between the software and live raters. Still, virtually all of the teachers interviewed for the article are very skeptical of the idea.
I’m not sure I’m too excited about having the student work actually graded by a computer, but I think using it as an add-on to increase the amount of writing and feedback students can do is a really viable option. And while a good teacher can almost certainly grade a few essays with more detail and precision than the computer, once that pile of essays gets to be in the range of 100 or so, I suspect the quality of the teacher assessments begins to degrade quite a bit.