From the CRLT Blog

ChatGPT: Implications for Teaching and Student Learning

January 9, 2023
1 comment
crlt

ChatGPT

Recently, the artificial intelligence app ChatGPT has been making headlines in the higher education media and beyond. Some have taken an alarmist approach, such as a recent Atlantic piece titled “The College Essay Is Dead.” Others have been more sanguine, examining the limitations of the app as well as offering suggestions for how it can inform student learning and writing. Below is an FAQ about ChatGPT that includes links to useful resources.

What is ChatGPT?

Developed by Open AI, ChatGPT (Generative Pre-trained Transformer) is a machine learning tool that can sift through large amounts of data available on the web to generate responses to user prompts. Responses are often quite plausible across a wide range of topics, and the power of this artificial intelligence tool has caused widespread concern that students will use it to plagiarize assignments, generating answers to essay or exam prompts rather than spending time developing their own responses.

ChatGPT itself responded as follows when given the prompt “Write a description of OpenAI GPT-3” 

GPT-3 has received a lot of attention in the media and in the tech industry due to its impressive language processing capabilities. It has the ability to generate coherent and fluent text that is often difficult to distinguish from text written by humans. However, it is important to note that, like any machine learning model, it has limitations and is not a replacement for human intelligence.

A more technical explanation is available here.

What are its limitations?

While powerful, there are limitations. ChatGPT can respond credibly on a range of topics but its responses tend to be formulaic and can include incorrect information. In addition, it cannot access material behind a firewall, such as articles from JSTOR, and it only references material from 2021 or earlier. Nor does it have access to experiences not stored on the Web, including the content you present and discuss with students in your classroom. Daniel Lameti notes in his article “AI Could Be Great for College Essays,” that while ChatGPT does particularly well with very general prompts such as "contrast capitalism and socialism",

few professors ask students to write papers on broad questions like this….And the more you make the question like something a student might get—narrow, and focused on specific, course-related content—the worse ChatGPT performs. I gave ChatGPT a question about the relationship between language and colour perception that I ask my third-year psychology of language class, and it bombed. (“AI Could be Great for College Essays”).

You might consider running your own experiment by signing up for a ChatGPT account and entering your assignment prompts to see how ChatGPT would fare in response to assignments in your course.

How can I design assignments to discourage use of ChatGPT?

While there have been reports of programs developed to detect use of ChatGPT, it is likely that there will be technologies developed to thwart such detectors. Rather than pursuing such approaches, you can adopt strategies that discourage use and promote learning. One important strategy for deterring use of ChatGPT or other similar technologies is to create assignments that require students to show stages of their work (outlines, rough drafts, etc.). These strategies can lead to deeper learning, provide instructors with more regular insight into student work, and increase the likelihood that the final product reflects student efforts rather than a copy of others’ work (human or artificial). The following examples offer a starting point. See CRLT’s Occasional Paper on academic integrity for a fuller discussion of these and similar approaches.

  • Sequence major assignments to include project proposals/outlines, multiple drafts, annotated bibliographies.
  • Ask students to reference course materials (especially those housed in Canvas) and discussions, which would be impossible for ChatGPT to access and would also promote valuable integration skills.
  • Specify the types of source materials students should use, including some that are very specific to the assignment, such as field specific journal articles that require authentication, data collection and analysis when relevant, or client assessment for field assignments.
  • Ask students to engage in and submit a reflection about what they have learned from completing the assignment. Sample prompts include: a) Discuss the most challenging and most rewarding aspects of your project. b) What was the most surprising thing you learned in the course of this project? c) If you had the chance to do it again, what one thing would you have done differently on this project?
  • Discuss academic integrity with students, including what constitutes original work and plagiarism in your field, what type of assistance is and is not permitted. It is also worth pointing out that OpenAI’s terms of use include the following among its restrictions: users may not “represent that output from the Services was human-generated when it is not.”
  • Teach citation skills, sharing An Introduction to Academic Integrity, a U-M Library website designed for students. This Sweetland Center for Writing web page on Teaching Citation and Documentation Norms has additional suggestions and exercises.

In a recent email exchange on this topic, Moni Dressler, head of LSA’s Office of Academic Technology, suggested the following additional practices:

  • Have students work on peer editing and peer commentary as part of the evaluation/writing process, so that they have to comment and make suggestions and respond to other students' writing.
  • Have students write and submit a Google Doc where you are added as an editor so that you can see the document history.
  • Focus on research skills and the expression of original thought, rather than creating a synthesized document.

It is also worth considering why you assign writing in your classes and discussing that explicitly with students. A recent article in the Chronicle of Higher Education discussed the power of such an exchange.

For most professors, writing represents a form of thinking. But for some students, writing is simply a product, an assemblage of words repeated back to the teacher….[According to] John Warner, a blogger and author of two books on writing, "If you can create an atmosphere where students are invested in learning, they are not going to reach for a workaround. They are not going to plagiarize. They are not going to copy, they are not going to dodge the work. But the work has to be worth doing on some level, beyond getting the grade.”

What about using graded, hand-written, in-class assignments?

Some recent articles have suggested only assigning in-class, hand-written writing as a way to avoid plagiarism associated with ChatGPT. While this would certainly solve the problem of students using ChatGPT, such exercises could be very problematic for a range of students, such as students who need extra time to complete assignments as well as students whose first language is not English who need additional time and resources to write. Moreover, during the pandemic semesters, we heard from students across the university that they appreciated the move away from timed high-stakes assignments. Increasing the use of graded, in-class, written assignments would be a step backward, and would severely limit the genres of writing students engage with.

How can I incorporate ChatGPT into the writing process?

These types of apps are not going away and will no doubt proliferate in the future (e.g., Magic Write), so encouraging students to use them productively could be seen as an important contribution to information literacy. For example, consider using ChatGPT as a tool within the writing process. You could ask students to use the app to brainstorm topics for a paper or to comment on/edit the app’s response to a prompt, either individually or as a class, by pasting the response into a Google Doc and using the commenting function. Questions might include where arguments are unconvincing, what information is incorrect or missing, and how the writing could be made more persuasive or engaging. This article in Inside Higher Education describes a collaborative project among faculty at the University of Mississippi to develop productive strategies for using ChatGPT with students.

COMMENTS

pfa

Wed, 01/11/2023 - 01:36pm

While this is interesting and a helpful start, it doesn't include some important pieces of information. 

For limitations, they should really include that ChatGPT tends to falsify citations, including fictional articles written by well known authors, actual DOIs that are for completely unrelated articles not cited, and similar problems. Often the fake citations are too good to be true. I've had researchers come to me asking me to help them find something cited by ChatGPT because it's exactly what they've been looking for, but ... it doesn't exist. It's also been caught creating fictional scientific specialties in esoteric domains. Basically, depending on your prompt, it tries to assure you that what you are looking for exists, whether or not this is true. 

While the blogpost indirectly references GPTZero, it did not provide the link or name it, and there are related tools people are using which don't have fancy websites. Some of these are quite interesting, and ChatGPT is also working to add "watermark" code hidden in its output to facilitate identifying their products. 

Some of the strategies recommended here to avoid ChatGPT-generated student writing have already been tested and failed. 

"Have students work on peer editing and peer commentary as part of the evaluation/writing process, so that they have to comment and make suggestions and respond to other students' writing.
Have students write and submit a Google Doc where you are added as an editor so that you can see the document history."

ChatGPT is being used to create commentary and to grade assignments, and when supplied with a rubric and original draft will revise to meet the new specifications. The final suggestion is the best:


"Focus on research skills and the expression of original thought, rather than creating a synthesized document."

The AI tools focus on a lowest common denominator approach, not creativity. What is it that humans do well, and what are approaches that emphasize the human aspects of communication, literacy, creativity? Assignments that focus on this are where the future of education lies. Where I've been finding the most useful ideas for innovative assignments is with the Twitter hashtag #unessay. I encourage people to explore this, and also #AIEducation and #AIinEdu.

My two cents ...