[PRL] Measuring the Effectiveness of Error Messages Designed for Novice Programmers

[David Van Horn]

For those of you who may have missed Guillaume's Scheme Workshop talk, 
this is really interesting stuff and fortunately he'll be talking about 
it again today at MIT (2pm).

David

Measuring the Effectiveness of Error Messages Designed for Novice 
Programmers
Speaker: Guillaume Marceau  PhD student
Speaker Affiliation: Worcester Polytechnic Institute (WPI)
Host: Daniel Jackson
Host Affiliation: MIT-CSAIL

Date: 2-10-2011
Time: 2:00 PM - 3:00 PM
Location: 32-G7 Floor Lounge

Good error messages are critical for novice programmers. Recognizing
this, the DrRacket programming environment provides a series of
pedagogically-inspired language subsets with error messages customized
to each subset. We apply human-factors research methods to explore the
effectiveness of these messages. Unlike existing work in this area, we
study messages at a finegrained level by analyzing the edits students
make in response to various classes of errors. We present a rubric
(which is not language specific) to evaluate student responses, apply
it to a courseworth of student lab work, and describe what we have
learned about using the rubric effectively. We also discuss some
concrete observations on the effectiveness of these messages.

Joint work with Kathi Fisler and Shriram Krishnamurthi.

Relevant URL(S):
For more information please contact: Maria Rebelo, 617-253-5895, mr at 
csail.mit.edu