Skip to main content

Turnitin AI Detector

Turnitin AI Detector

Overview

By now, most of us have heard about ChatGPT, the artificial intelligence (AI) program that can generate text when given a question or prompt. It has raised significant concerns within academia given that students could potentially use it to generate essays, discussion posts, and other writing that they are expected to produce themselves. Unlike the familiar forms of plagiarism that involved copying from existing texts, the text that ChatGPT generates is original and unique, so it cannot be detected by matching it against existing texts using a program like Turnitin’s originality checker.

However, Turnitin has now added a new feature designed to detect text generated by AI, called AI Writing Detection. When a paper is run through Turnitin, in addition to the usual “originality report” (the tool that detects when text in the student’s submission matches existing text), you will now see a blue box labeled “AI” that has a percentage score. That score indicates how much of the text Turnitin believes to have been generated by artificial intelligence. When you click on that, a new window opens up highlighting in blue text believed to have been AI-generated.

Turnitin AI Detection report percentage.

Important Considerations

What does this mean? We would like to raise a couple of important points about this feature.

  1. Students do not see the report.
    1. Unlike the originality checker, which an instructor can opt to allow a student to review prior to a final submission, students will not be able to see the AI Detector’s report.
       
  2. This is NOT proof of cheating.
    1. Turnitin claims that the rate of false positives is less than 1% when more than 20% of the paper is determined to have been AI-generated. This is based on comparing hundreds of thousands of AI-generated and human-written texts. However, unlike its originality checker that can show an exact match between a portion of the submitted writing with existing text, there is no surefire way to prove that the text identified by the AI detectors was, in fact, generated by AI. ChatGPT and other LLMs generate new, unique text each time they are given a prompt, so it’s not possible to replicate what was previously generated by, say, entering an assignment prompt. Thus, if text is identified by Turnitin as likely to have been AI-generated instructors must review it themselves. If it seems that Turnitin’s determination is accurate, you will still need to decide how to proceed (more on this below). 

How to Review the Report

If Turnitin highlights portions of a student’s text as likely to have been AI-generated, there are some ways to enhance your confidence in that determination:

  • Compare the writing style in the non-highlighted portions of the paper with the style in the highlighted portion.
  • Compare the writing style of the highlighted text with text you know (or have good reason to believe) represents the student’s own original writing, such as emails or discussion posts.
  • Look for repeated mistakes of grammar and mechanics in the paper. Do they persist or disappear in the highlighted portion? If they disappear, that may indicate the use of AI.
  • ChatGPT is notorious for making up bogus citations. If there are citations in the highlighted portion, run it through the library’s website or WorldCat to see if the cited text actually exists (often the author and/or journal are real, but the cited text is not).
  • Consider whether it makes sense for the student to have used ChatGPT for that portion of the assignment. For example, does it contain personal anecdotes or reflections, refer to course lectures or discussions, or echo things the student has expressed in class? While features certainly don’t rule out the use of ChatGPT, they may lead one to question the accuracy of the AI determination.

How to Proceed with a Positive Determination

If Turnitin has detected AI-generated text, and your review validates that determination, there are some strategies to pursue in response.

First, invite the student to a formal interview and ask them some probing questions to try to determine whether and to what extent they used ChatGPT, such as:

  • “What websites did you go to when writing your paper?”
  • “How long did it take you to write this essay?”
  • “Please could you speak for a few minutes on the subject of the essay?”
  • “Please explain your writing process.”
  • “Could you tell me the definition of [interesting or unusual word used]?”
  • “What sources did you use for this essay? Did they influence your writing?”
  • “For this essay, can you tell me whether you approached it in a different way than previous assignments?”

If your suspicions seem founded, you can proceed to ask directly whether they used a website to generate any portion of their essay. If they deny it, but you have reason to suspect they are not being honest, you may inform them that an AI detection tool has found that X% of the text was found likely to have been generated by AI, and that you will be providing that report to the Office of Academic Affairs.

Further Resources