What distinguishes AI from other programs is that it is programmed to do things that appear intelligent - such as writing texts.
Getting a computer to compose a text is nothing new since that has been done for decades. What is new and what has led to debate is that computer tools that create text that is so good that it can fool an unwary human reader have become more widely available. The best-known is ChatGPT, but there are several others, and development is progressing very quickly.
The university's assessment of students is based largely on their written work. Therefore, many teachers are concerned that students can cheat using AI as a form of ghostwriter. In principle and legally, this is no different from using human ghostwriters, something that has always occurred. If you allow AI to write your take-home exam, then you are cheating in the same way as if you asked your smart cousin to write it, and the types of assignments that can be AI-cheated are the same as those that you can cheat on with a cousin. No doubt there are a great many homework assignments that were written by cousins, spouses, friends, and even professional ghostwriters. AI has only democratised this cheating, making it accessible even to those without smart cousins and underprivileged groups who cannot afford human ghostwriters.
That said, how should a teacher deal with AI cheating? We can distinguish between proactive measures, which is when the teacher designs the assessment in a way that makes AI cheating more difficult, and reactive measures, which is when the student has completed the assessment and cheating is suspected.
Proactive Measures 1
Change the form of assessment in the course syllabus
Assessment in a course should always be well-thought-out and enable the examiner to assess the students' achievement of the objectives. The risk of cheating of various kinds should be considered when designing the forms of assessment. However, this does not mean you need to eliminate all forms of assessment that may be easy to cheat on. Sometimes, the pedagogical benefits outweigh these.
Forms of assessment that are very difficult to cheat on with AI:
- Supervised exam on campus
- Oral assessment on campus
- Practical exercises where a student must physically do or produce something.
- Field trips, labs, creation using physical materials
Forms of assessment where it is somewhat difficult, but not impossible, to cheat with AI:
- Online exam
- Oral assessment online
- Examining seminars
- Film
- Supervised writing, where the supervisor follows the writing process
Forms of assessment where it is easy to cheat with AI (or with human ghostwriters):
- Unsupervised written assignments (take-home exams, reflections, reviews, reports…)
- Digital creation.
One way to prevent AI cheating is, of course, to change the form of assessment to one that is more difficult to cheat on. However, this means changing the course syllabus, which is nor permitted in an ongoing course.
Proactive Measures 2
Change the assignment to make it more difficult for AI
AI is good at creating certain types of texts and answering certain types of questions, but significantly less so when it comes to others. You can therefore prevent a great deal of AI cheating simply by changing questions, or similar, in an exam. Avoid the types of questions that AI is good at; instead, ask questions that are difficult for AI.
AI is quite good at:
- Open general questions:
- Explain Piaget's stage theory of child development
- How does inflation occur according to monetarism?
- What are the main advantages and disadvantages of wind power as an energy source?
- Some types of problem-solving:
- Legal cases
- Medical diagnoses based on symptom descriptions
- Creating digital images and sound/music based on a general description, or in a specific style
- Creating a picture of a child petting a dog
- Writing a song in a specific genre
- Some programming assignments
Currently, AI is not good at:
- Relating to somebody’s experiences
- Relating to things like lecture content or other material that is not available online
- Talking about current issues – AI takes time to train, and therefore it is not trained on the latest news: for example, “Reflect on the ongoing debate about hungry schoolchildren and porridge.”
- Managing sources – asking AI for sources can be completely hit and miss
- Factual information – pure factual errors are common in AI texts
- Idiomatic expressions – AI likes to interpret these literally
- Some types of problem-solving:
- Scientific and technical problems
- Other problems that require figures and calculations
In general, AI is better at “standard tasks” - that is, tasks that are commonly used in a similar form in a subject. The more unusual the task, the more difficult it is for AI. Therefore, it is possible to make AI cheating more difficult by being creative when constructing the task. The more original the task, the more difficult it is for AI. Better still, it becomes more difficult for students to find texts to plagiarise.
One tip if you are in doubt is to simply test your exam questions in ChatGPT. If it gives good answers, then you need to review the question. This also gives you a good idea about how AI would write.
Reactive Measures
Determine if a text is AI-written
Unfortunately, there is no completely sure way to prove that a text is AI-written. However, there are quite a few characteristics that can speak either for or against AI cheating. However, to begin with, there are some methods and arguments that should not be used:
Ask AI if it wrote the text. It is perfectly possible to enter a text into, for example, ChatGPT and ask if it has written the text. It will often answer yes – but that answer is completely useless, because it will often not give a true answer.
Various online tools are available that are said to show whether a text is AI-written. These have various levels of accuracy and all too often yield false positives – false accusations, which means they cannot be used in disciplinary cases.
“The text is too good for it to have been written by this student.”
If you are a teacher and find it difficult to believe that a particular student has written such a good text, it may be a good idea to check the text more thoroughly: your gut instinct is not enough to report the matter. Furthermore, this argument may be perceived as discriminatory.
To assess whether a text is AI-written or not, you can use the list from the previous section that tells you what AI is good or bad at. If a text contains good material of the type that AI is bad at writing, it is likely written by the student. Language processing can also be a clue: AI normally writes correct and careful language that lacks colloquialisms or dialect. You can ask ChatGPT to write in dialect, but the results are usually exaggerated and unnatural.
Signs that a text is written by a student:
- Linguistic errors, spelling mistakes, colloquialisms, dialect
- For students with a foreign background: the kind of mistakes that second language speakers tend to make are, for example, with prepositions and word order.
- Idiomatic expressions used correctly
- Relevant source references with correct page references
- in particular, references to the course literature and other "local" literature
- Errors in structure are more common in student-written texts than in AI texts
- Credible descriptions of personal experiences
- especially if there are concrete details that can be linked to VFU or similar
- References to what a named teacher has said
- Links to current events
Signs that a text may be AI-written:
- Correct neutral impersonal language
- Note, however, that well-performing students can also write like that!
- General text, without concrete connections to the course or personal experiences
- Made-up or in some other way factually incorrect references
- Errors with the likes of structure, format and references indicate the student is the author.
- Other kinds of fabrications and factual errors
- A human writer can make mistakes with facts, but only an AI will fabricate information in an exam text.
- Idiomatic expressions are interpreted literally.
- Note, however, that even human writers can miss idioms especially if the language is not their native language.
- The text is completely devoid of things that AI is not good at.
- Large stylistic differences between sections of text, where what is easy for an AI is written in an AI-typical manner, while what is difficult for AI is written in a human way
- Note, however, that the same effect can occur through legitimate language review of parts of a text.
It is rarely possible to determine that a text is AI-written through its positive qualities since a good student is also good at what AI is good at. What you can catch AI on are its mistakes, and above all its fabrications. It is seldom the case that students make up references or factual information. AI also makes other kinds of linguistic mistakes, especially when the language is not intended to be interpreted literally.