[Colloq] =?Windows-1252?Q?Craig_Zilles_Education_Talk:_=93Effective_Secure, _and_Ef?= ficient Summative Assessment using a Computer-based Testing Facility” sponsored by the Department of Electrical and Computer Engineering, Tues. Oct 8th @ 10 am in 136 ISEC

[Khoury Academic Affairs]

Craig Zilles Education Talk, sponsored by the Department of Electrical and Computer Engineering

Speaker: Assistant Professor, Craig Zilles, University of Illinois at Urbana- Champaign

Date: Tuesday, October 8, 2019

Time: 10:00 am – 11:00 am

Location: 136 ISEC

Title: “Effective Secure, and Efficient Summative Assessment using a Computer-based Testing Facility”

Abstract:

Exams are a widely used method for summative assessment in college education, especially in introductory courses. However, at many universities, introductory courses are large (e.g., 200+ students). Running traditional pencil and-paper exams at this scale presents management challenges that include requesting space, printing exams, proctoring, timely grading, and handling conflict exams. These practical concerns often have more influence on how assessment is performed than pedagogical concerns. In this talk, we'll discuss an effective, secure, and efficient alternative to traditional pencil-and-paper exams. At the College of Engineering at the University of Illinois, we've been running a Computer-Based Testing Facility (CBTF) for more than four years now and have been running at scale (30+ courses, 50,000+ exams/semester) for the past couple years. The CBTF is a proctored, "locked-down" computer lab that is operated as a service to courses. The CBTF has changed how we teach, leading to improved student learning and enabling the introduction of more project and group work in large classes, because graduate TAs are freed from routine proctoring and grading. The goal of the CBTF is to make assessment with exams better for everyone involved--students, faculty, and course staff. Four concepts are key to achieving this goal. First, by running the exams on computers, we can write complex, authentic (e.g., numeric, programming, graphical, design) questions that are auto-gradable, allowing us to test a broad set of learning objectives with minimal grading time and providing students with immediate feedback. Second, we write question generators that use randomness to produce a collection of problems, allowing us to give each student different questions and permitting the problem generators to be used semester after semester. Third, because each student has a unique exam, we allow students to schedule their exams at a time convenient to them within a specified day range, providing students flexibility and avoiding the need to manage conflict exams. Finally, because exam scheduling and proctoring is completely handled by the CBTF, once faculty have their exam content, it is no more effort to run more frequent, smaller exams, which reduces anxiety for some students, as well as offering second-chance exams to reduce failure rates by allowing struggling students an opportunity to review and demonstrate mastery of concepts that they missed on an exam. In this talk, I'll discuss the basic operation of our CBTF and the key components that make it work. I'll present findings on aggregate student behavior in the CBTF and data on increased learning gains and reduced failure rates in specific courses. I'll discuss our mechanisms and policies for maintaining security, supporting testing accommodations, and minimizing faculty disruption. Finally, I'll present findings from surveys of faculty and students and discuss the cost of operating the CBTF and how it compares to traditional exams and online services.

Bio:

Craig Zilles is an Associate Professor in the Computer Science department at the University of Illinois at Urbana-Champaign. His current research focuses on applying computing and data analytics to education. Previously, his research focused on the interaction between compilers and computer architecture, and he developed the first algorithm that allowed rendering arbitrary three-dimensional polygonal shapes for haptic interfaces (force-feedback human-computer interfaces).


Faculty Host: Professor David Kaeli, Department of Electrical and Computer Engineering kaeli at ece.neu.edu<mailto:kaeli at ece.neu.edu>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Craig Zilles Flyer.pdf
Type: application/pdf
Size: 389938 bytes
Desc: Craig Zilles Flyer.pdf
URL: <http://lists.ccs.neu.edu/pipermail/colloq/attachments/20191001/306435fe/attachment-0001.pdf>