ETUG Spring Workshop 2023 – Day 2.
12. Carrie Fry, R. John Robertson | Risks, Realities, and Ruminations on the Probabilistic Web

Recorded June 2, 2023

Current AI tools use large language models (LLM) to create original text, using probability to build it word-by-word. What does this mean for determining if something is real? If AI output is based on how often words are found together in the resources the tool has trained on, is it only as real as what is freely available online? How does this question of what is real intersect with other tools, techniques, and technologies? Using AI-produced academic papers and their reference lists as our case study, we will examine how tools like ChatGPT may alter our understanding of our current information landscape and allow us to interrogate a path forward. Reference lists create an accessible test case, as fact-checking is a little more straightforward than for the paper itself. ChatGPT assembles references that may or may not exist, merged references that contain real DOIs or article titles blended together beside completely invented references. What happens if judging something as real becomes too much of a barrier? Readers might not have access to the evidence to refute incorrect citations as most scholarly sources are locked behind paywalls. Then too, reviewing sources could be too time-consuming; most professors only give the references a cursory skim, unless they notice a problem. Is it reasonable to expect them to take time out of grading to research the references closely. While these decisions could be off-loaded to other detection tools, they are not keeping up, for example, Turnitin’s AI checker returns at least 1% false positives. Finally, is there hope for a semantic web, where AI tools recognize and evaluate their sources and know what a citation is, not just what it should look like?

Attachments: Transcript (Word, PDF, Text), Slides (PDF)

Note: If you notice an error in the closed captions (or transcript file), please email support@bccampus.ca with details of the error, the time stamp, and your suggested correction, if any.