A recent story in the New York Times covers a software program by nonprofit EdX that will soon be available for free to any institution that wants to use it. Using sophisticated machine learning algorithms to train its artificial intelligence, the software will grade essays and short response questions and provide nearly instant feedback. Naturally there are strong supporters for the new software touting it for “freeing professors for other tasks” (like what?). And just as naturally there are strong critics who have formed a group called Professors Against Machine Scoring Essays in High-Stakes Assessment. From the group’s petition:

Let’s face the realities of automatic essay scoring. Computers cannot “read.” They cannot measure the essentials of effective written communication: accuracy, reasoning, adequacy of evidence, good sense, ethical stance, convincing argument, meaningful organization, clarity, and veracity, among others.

While criticism is certainly warranted, I find the quote to be somewhat bullish. Can these people really claim that they understand how they are able to read and measure the essentials of effective written communication well enough that they can look at a computer and say with confidence, “that can not do what I am doing, and here’s why”? It very well may be that current AI programs do not have the ability to comprehend written communication to a degree necessary to assign grades, but to make the argument that the software shouldn’t be used because “computers cannot ‘read'”, as if that were a self-evident fact is just poor communication.

Now to be fair, I disagree with the supporters of the software as well.

“There is a huge value in learning with instant feedback,” Dr. Agarwal said. “Students are telling us they learn much better with instant feedback.”

Ok, well, not that part, I agree with that part in principle. But what kind of feedback? Supposedly the software can generate a grade and also comments whether or not the essay was “on topic”. So a student could get instant feedback, which is great, and then edit and modify, which is great, and resubmit, which is also great… and then what? What would they be learning?

I promise to be highly skeptical of any answer to that question that isn’t “how to write an essay that receives high marks from an automatic grading AI”.

All this talk about feedback. What about feedback for the professor? I find reading through 60 essays just as tedious and time consuming as the next out-of-place grad student in a department that doesn’t value teaching, but I also recognize that reading those essays is a valuable way for me to gauge how I’m doing. Are the concepts that I think are important showing up? Are there any major communication issues? What about individuals, are some struggling, what can I do to help? How will I learn my students’ personalities and how that might affect their personal engagement with the material? How will I learn to be a better educator?

Granted, even though 60 feels overwhelming, it’s nowhere near 200 or more. I can’t even imagine trying to read through that many assignments myself. I’m confident that if I were force to I would not emerge with my sanity intact. This problem does not go unaddressed.

With increasingly large classes, it is impossible for most teachers to give students meaningful feedback on writing assignments, he said. Plus, he noted, critics of the technology have tended to come from the nation’s best universities, where the level of pedagogy is much better than at most schools.

“Often they come from very prestigious institutions where, in fact, they do a much better job of providing feedback than a machine ever could,” Dr. Shermis said. “There seems to be a lack of appreciation of what is actually going on in the real world.”

An “A” for recognizing the problem. But the proposed solution is nothing more than a patch. In fact, it’s worse, because it is a tool that will enable the continual ballooning of class size. And to what expense? Why don’t you rethink your solution and have it on my desk in the morning. I can’t promise instant feedback, but maybe, just maybe, the feedback provided will be the start to moving in a direction that actually addresses the underlying problems, rather than just using technology to hide them.