By Carl Straumsheim
The plagiarism detection service Turnitin on Wednesday made bold claims about the effectiveness of its product to weed out “unoriginal writing”, but researchers in the field aren’t buying the results.
In a study detailed in the report released Wednesday morning, Turnitin tracked the decrease in "unoriginal writing" -- meaning writing that scored 50 percent or higher on the software’s Overall Similarity Index -- at 1,003 non-profit colleges and universities in the U.S. that had used Turnitin for five years.
Most institutions started experiencing drops in unoriginal writing by the third year of Turnitin use, and by year four, not a single type of institution reported an increase. In the fifth and final year of the study, every class posted a double-digit decrease, ranging from 19.1 percent among four-year institutions with fewer than 1,000 students to 77.9 percent among two-year colleges with 3,000 to 5,000 students. Overall, unoriginal writing decreased by 39.1 percent.
The results were met with skepticism from people such as Thomas S. Dee, a professor in the Stanford Graduate School of Education.
“To be honest, I don’t find this study particularly convincing, and I say that as someone who is quite sympathetic to the idea that there are technological solutions to the plagiarism problem that are readily available and affordable and worth studying further,” said Dee, who is also a research associate for the National Bureau of Economic Research.
Dee described the report as containing “suggestive, descriptive evidence,” not “convincing causal evidence” that Turnitin was primarily responsible for the drop in unoriginal writing.
“It’s a kind of selection bias story,” Dee said. “They’re looking into the prevalence of unoriginal content over time -- but think about the professors who might be picking up Turnitin. They may be the ones who feel they have the biggest problem with plagiarism in their class, so you’re going to get a very high baseline of unoriginal content. The profs who were late adopters could be those who see only marginal benefits. That would also create the appearance of a treatment effect when none may exist.”
Dee also said the adoption of Turnitin may have coincided with new policies meant to crack down on plagiarism. “Are we really observing the effect of the website here or the additive -- possibly multiplicative -- effect with other institutional changes?” he said.
...