Assessing what AI Can’t Do

Authentic activities that focus on the student’s developing dialog with course texts produce writing that no AI-generated text could substitute for.
by LSA Learning & Teaching Technology Consultants

The media is gleefully viewing the latest round of advance in AI with great alarm, declaring the death of the essay and possibly the collapse of academia in general. Many instructors are understandably alarmed, themselves, and wondering how to maintain the integrity of their courses and assignments if an AI engine like ChatGPT can construct a complex essay that, at least on first glance, appears germane. A deeper read of the first-hand reports demonstrates that many details are still obviously incorrect, but it is reasonable to ask, at this point, what can we do to prevent students who may be tempted from using AI engines to produce lack-luster essays that might slide by as the product of a student who doesn’t care about the course?

Fortunately, the answer to that is already in the room. Any instructor who focuses on teaching processes already uses activities and assessments that no AI can duplicate. Burke Nixon, for example, a writing instructor at Rice University, cogently points out all the ways in which authentic activities that focus on the student’s developing dialog with course texts produce writing that no AI-generated text could substitute for. A scaffolded term paper, one that includes submission of the steps that build up to the final product, such as annotated bibliographies, thesis statements, and drafts, both engages the students in learning the process of good research and critical thinking and also familiarizes the instructor(s) with each student’s genuine voice. While an AI can certainly generate a generic essay just as easily as a paid human “tutor” can, even that human can’t easily duplicate another person’s writing voice unless they have multiple and extensive previous examples to work from, something that a student paying to have someone else write their paper is very unlikely to have on hand. The same is true of an AI. Or as Ken Mondschein pithily puts it in his account of evaluating ChatGPT’s capabilities, “artificial ‘intelligence’ is, in fact, quite stupid.”

(Indeed, the artist Ursula Vernon recently posted a very entertaining series of tweets, documenting her attempts to get an art AI to generate a specific sort of image. The results were deeply underwhelming, and thoroughly sustained her argument that commission artists have nothing to fear from AI for a good, long while.)

Here are some additional steps that instructors can use to AI-proof their essay assignments:

  • Assign many small, low-stakes writing activities (less than one page). This will develop instructor familiarity with each student’s writing voice, as well as developing the students’ ability to write concisely and reducing their fear of writing.

  • Include the build-up stages (research, drafts, etc.) explicitly in the assignment, offering both feedback and a small number of points for each stage. Both evaluation of research and revision of drafts require human analysis.

  • Require that students engage directly with the course texts, and ideally with the readings and interpretations of other students in the class. Such timely and unique material cannot be generated by AI.

  • Ask for genuine analysis and nuance, in your writing assignments. (The Learning and Teaching Consultants can help you with this, if you are concerned about your existing prompts.)

Measures to avoid include:

  • Requiring all writing to be done by hand. Many students are not taught longhand writing any more, and will not be able to write quickly and clearly enough, especially for longer assignments.  Such assignments would both present a genuine barrier to evaluating learning and unnecessarily tax the instructor to read. If you particularly want live responses, consider assigning oral presentations with a question period included. 

  • Seeking a purely technological solution. Evaluating the humanity of an essayist requires human judgment from the instructor, just as evaluating the merits of a research source requires it from the student. Consider, too, that it is very likely that the essay-writing AIs were trained using exactly the vast store of student essay data that plagiarism detection software collected over the past decade or so. Let us not invite yet more trouble.

On the other hand, if you are interested in trying out in-class writing activities with lockdown technology, so that students can type and use word-processing tools without being able to leave the page to use any additional tools, contact LTC to be included in our upcoming pilot of Tophat lockdown.

If you would like to discuss how best to adjust your essay assignments, please feel free to reach out to the Learning and Teaching Consultants! We will be happy to assist you.

 

Works Cited

Kearns, S. (2022, December) AI Writes Academic Essay, Raises Questions About Academic Integrity, Standards. CNS News. https://www.cnsnews.com/commentary/shelby-kearns/ai-writes-academic-essay-raises-questions-about-academic-integrity

Mondschein, K. (2023, January) Avoiding Cheating by AI: Lessons from Medieval History. Medievelists.net. https://www.medievalists.net/2023/01/chatgpt-medieval-history/?fbclid=IwAR1mLOuyt9YVzAAc-1FmTfkaYwj7mln_WYvVmpHBh-zEiRGSODmxQvX0lSA

Nixon, B. (2023, January) Opinion: No, AI chatbots haven't killed the college essay. Here's why. Houston Chronicle. https://www.houstonchronicle.com/opinion/outlook/article/ChatGPT-AI-technology-writing-obsolete-college-17680035.php

Ursula, V [@UrsulaV]. (2022, December 14) Alrighty gang, it’s time for another installment of “Can The AI Actually Do That?” with our fictional stunt artists, the Fabulous Veggiesmiths! [Tweet-thread]. https://twitter.com/UrsulaV/status/1603153432315043843
 

Further Reading

Watkins, M. (2022, December) Guest Post: AI Will Augment, Not Replace. Inside Higher Ed. https://www.insidehighered.com/blogs/just-visiting/guest-post-ai-will-augment-not-replace

Watkins, R. (2022, December) Update Your Course Syllabus for chatGPT. Medium.com. https://medium.com/@rwatkins_7167/updating-your-course-syllabus-for-chatgpt-965f4b57b003

 

 

Email
Release Date: 01/12/2023
Category: Learning & Teaching Consulting; Teaching Tips
Tags: Technology Services
Technology Services Contact Center Chat