This is the second post in the To Ask or Not Ask the Question in ChatGPT series. The first post focused on using ChatGPT to ask questions and gather information, while this post highlights best practices when asking ChatGPT to ask questions, starting the research process, and teaching strategies to introduce these ideas to students.
Best Practices for Information Gathering
Prompt Writing and Questions
It’s best practice to include specifics and context when writing an effective prompt. However, we need to ensure our students don’t confuse specificity with leading when asking questions. Leading questions increase confirmation bias and the chance of hallucinations—or, more bluntly, the chance that ChatGPT will just make things up. To avoid this, students need to ask open-ended questions. The good news is that these aren’t new discussions; we’re simply applying them to GenAI tools.
Follow-Up Questions
We can’t identify (or evaluate) the sources ChatGPT uses in its responses, nor can we assess the accuracy and diversity of the data it is trained on. Therefore, we need students to understand that they must critically analyze ChatGPT’s responses.
They should start with a strong, open-ended prompt. Then read the response carefully and ask follow-up questions that expand upon or challenge the initial response. Is anything missing or contradictory? Are there gaps in the perspectives in the response (i.e., people of color, disabled, conservative or progressive…)? Prompt ChatGPT to include those perspectives or address contradictions. Students can also try asking two opposing leading questions, such as: “Since social media is harmful to teenagers, how does…?” and “Since social media is beneficial to teenagers, how does…?” By comparing the answers, students can identify biases and identify future areas for deeper research. They need to treat ChatGPT as a conversation, not as a one-off Google search.
Case Study: Star Trek and Baseball
You didn’t think I’d skip a Star Trek reference in an AI discussion, did you? Let’s look at how question phrasing and follow-up affect ChatGPT’s response in these prompts:
“How is baseball used as a symbol for human creativity in Star Trek?” (ChatGPT)
ChatGPT’s response is “okay,” but the response assumes three things based on assumptions in the prompt: baseball appears in Star Trek, holds symbolic significance, and symbolizes creativity. In reality (and take my word as a Trekkie for it), baseball only appears meaningfully in the tv series Star Trek: Deep Space Nine (DS9), where it’s important to only one main character, has several possible symbolic meanings, and features prominently in just three episodes. None of those nuances are in ChatGPT’s response because the prompt is too leading, and I didn’t ask any follow-up questions.
A better question might be:
“Does baseball play any significant symbolic role in Star Trek?” (ChatGPT)
This wording invites more flexibility in ChatGPT’s response, but the response still isn’t without its flaws. ChatGPT wrote that “Baseball plays a symbolic role in Star Trek, particularly Star Trek: Deep Space Nine,” and after a close reading, it’s clear that all the examples come from DS9. It took a follow-up question for ChatGPT to clarify that baseball is not “particularly” explored in DS9, but instead, it is only explored within DS9.
ChatGPT and the Research Process
Once students understand how to write an effective prompt, craft open-ended questions, and understand the need for follow-up, they can explore ways that ChatGPT can help in the early stages of their research.
Exploring and Brainstorming
We’ve all seen it: a student has a broad idea but no idea how to develop it for their assignment. This initial exploration into a topic is something that ChatGPT can help with. Try this prompt:
“I’m working on a [type of assignment] for a [grade level] [course type] class and I need help coming up with a topic. I’m interested in topics related to [general topic]. Please give me a list of 10 topic ideas related to that.”
“I’m working on a 10-page literature review on climate change for an upper-division marine biology class. I’m interested in marine mammals. Please give me 10 topic ideas.”
Students can then follow up by asking for additional ideas, for broader or narrower alternatives, or to further refine a topic they found interesting. Since they won’t know if a topic is “just right” until they start searching for sources and engaging with the information, it can be helpful for them to see multiple variations of a topic.
Finding Search Terms
Since ChatGPT isn’t a credible source, students’ next step should be asking for help finding reliable information in a library or an academic database. These prompts should be very specific because search strategies differ depending on the type of resource and search platform. So they should try a prompt like this:
“I’m researching [topic] for a [grade level] [assignment]. Act like a librarian and help me with search terms and tips to find [types of resources] in [search tool].”
“I’m researching how climate change affects dolphin lifespans and population sizes. Act like a librarian and show me a list of search terms I could use to research this topic and find peer-reviewed sources in my college library catalog or academic database.”
If the suggested terms don’t work, students can connect with a CI librarian (we even have 24/7 chat!) or follow up in ChatGPT with, “These keywords returned results about X; I want ones about Y.” ChatGPT doesn’t replace libraries or teachers, but it can be a great tool at the beginning to help students get past that initial phase of starting a project.
Teaching Tips
AI and AI literacy are vast concepts that can be intimidating to introduce in the classroom. Here are a few tips to help:
- Start with something fun, yes, even Star Trek! Or try Terminator if you’re an AI skeptic. Starting lessons with laughter and fun is a great way to start a harder discussion.
- Show what GenAI can’t do. Highlighting “easy” tasks that it can’t do can help students question the complex things they think it can do. For example, until very recently, ChatGPT was unable to create an image of a full glass of wine (ChatGPT).
- Promote active learning with ChatGPT Edu. Have students ask ChatGPT questions about a class topic, a niche personal interest, or a controversial current event. Then students evaluate the responses, ask follow-up questions, and present their findings. Ask them to address:
- What did ChatGPT get right or wrong?
- Was the answer balanced or incomplete?
- What follow-ups improved the response?
- Could they lead ChatGPT to generate a hallucination?
- Did leading vs. open-ended questions change the response?
- Can ChatGPT assist in identifying credible external sources beyond the library?
- Could ChatGPT help them find credible sources in the library?
Finally, don’t start from scratch. The practices discussed here— avoiding confirmation bias, evaluating credibility, using popular sources for background information, and crafting open-ended questions —aren’t new. We have years of information literacy lesson plans that can be easily adapted for GenAI tools.
~
AI sparks strong reactions, and that’s okay. You don’t have to love it or use it, and ethical concerns should be part of the conversation. But AI isn’t going anywhere, and we need to prepare students to navigate this reality. I like to view AI as an opportunity for us to reinforce our students’ information literacy skills, rather than an obstacle we have to overcome.
References
ACRL Framework for Information Literacy Sandbox. (n.d.). Association of College and Research Libraries. https://sandbox.acrl.org/
Framework for Information Literacy for Higher Education. (2016). Association of College and Research Libraries. https://www.ala.org/acrl/standards/ilframework
Generate Topics for Your Research Paper with ChatGPT. (2024, Mar 28). Baylor University. https://libguides.baylor.edu/students-chatGPT/generate-topics-chatGPT
Integrating AI to Guide Learning. (n.d.). Miami University. https://miamioh.edu/howe-center/hwac/resources-for-teaching-writing/integrating-ai-to-guide-learning.html
McKay, P. (2024, Oct 23). How Does ChatGPT Answer Reference and Research Questions? University of Florida Business Library. https://answers.businesslibrary.uflib.ufl.edu/genai/faq/416425
Walker, S. (2024, Oct 29). Star Trek: Deep Space Nine Shows Us That Baseball Is a Game Worth Playing. Startrek.com. https://www.startrek.com/news/game-worth-playing-deep-space-nine-baseball
