Detecting possible cheating in programming courses using drastic code change

Automated code similarity detection tools are widely used in programming classes to detect students cheating who copied from classmates or from a common online source. But today, other ways of cheating are becoming common that do not yield code similar to classmates, such as copying from an online source that other classmates didn’t find, or copying code developed by a hired contractor. Using a cloud-based programming environment that records every code compile/run, we detected code clearly not written by students in our class and noticed running of such code was sometimes preceded by a “drastic change” in their code history — a run whose code is so dramatically different from the previous run as to be unlikely to have been derived normally from the previous run. Some students submitted such code right away, what we call an “initial leap”. Other students tried writing code themselves, gave up, and then copied from an online source or hired a contractor, what we call “gave up”. Among either group, we further noticed that some would sequentially try a variety of copied solutions attempting to find one that works, what we call “solution hopping,” causing even more instances of drastic changes in the student’s code history. Thus, we developed a tool, based on a simple “text diff” algorithm, to detect drastic code changes in student code progressions, and to point instructors to possible cheating cases. We conducted two experiments. The first experiment measured the accuracy of our tool. The tool averaged 100% sensitivity and 100% specificity using real data, and synthetic data. The second experiment studied the prevalence of drastic changes in real student programs in our course. The study showed about 32% of students in the initial leaps group, and 5% in the gave up group, which we manually confirmed as actual cheating. Furthermore, 24% of initial leap students and 47% of gave up students subsequently solution hopped. We plan to make our drastic change detection tool available to the CS community as a free web tool.