Assignments

AI Validation

Students evaluate an LLM’s “reasoning” to clarify their own understanding of how to solve a problem.

By Robert Talbert

Learning Objectives

  • Identify and apply the standards of good reasoning, such as clarity, relevance, consistency, validity, soundness, and strength

Instructions

  1. Take a challenging problem on the topic your class is working on. It should be a problem where there is either one right answer or where there are answers that are “more right” than others.
  2. Give the problem (and any context you would usually give to students) to an AI tool. Ask it to generate multiple solutions and explain its reasoning. You might try giving it to multiple tools to get a range of answers (e.g., ChatGPT, Claude, Gemini, Bing). Here are some sample prompts that might be useful in the context of a mathematics, physics, or ethics course:
    • Solve the following equation: x2+6𝑥+8=0. Provide multiple solutions and explain your reasoning for arriving at those solutions.
    • Can sound waves generate heat? Provide multiple answers to this question and explain your reasoning for each.
    • How should we as a society solve food deserts? Please provide a number of different solutions and explain how each would help to alleviate hunger.
  3. Choose a selection of around three responses. These should include a range: some that are correct, some that are partially correct, some that are incorrect. (If all responses are correct, or all are wrong in the same way, you may want to try a different problem.)
  4. Ask students to explain which, if any, are correct and why; and if none of them are correct, provide a correct response with an explanation.

Using This Assignment

This assignment is licensed under the Creative Commons BY-NC-SA 4.0 license. Learn more about what this license means, what you can and cannot do with this assignment, and how to properly attribute on our Using Our Content page.

Suggested Citation

Robert Talbert. AI Validation. AI Pedagogy Project. https://aipedagogy.org/assignment/ai-validation/. Licensed under Creative Commons BY-NC-SA 4.0.

Related Assignments

Map a Moment: Reflecting on AI Experiences

By Beck Tench, Dan Be Kim, & Rotem Landesman

Prompt students to reflect on AI use: how they used it, how it felt, and what they learned.

View assignment

Align on the Line: Making Ethical Lines Visible

By Beck Tench & Emily Weinstein

Encourage students and teachers to align on acceptable AI use for different classes or assignments by surfacing ethically gray areas and mapping them together.

View assignment

AI Validation

By Robert Talbert

Students evaluate an LLM’s “reasoning” to clarify their own understanding of how to solve a problem.

https://aipedagogy.org/assignment/ai-validation/

Learning Objectives

  • Identify and apply the standards of good reasoning, such as clarity, relevance, consistency, validity, soundness, and strength

Instructions

  1. Take a challenging problem on the topic your class is working on. It should be a problem where there is either one right answer or where there are answers that are “more right” than others.
  2. Give the problem (and any context you would usually give to students) to an AI tool. Ask it to generate multiple solutions and explain its reasoning. You might try giving it to multiple tools to get a range of answers (e.g., ChatGPT, Claude, Gemini, Bing). Here are some sample prompts that might be useful in the context of a mathematics, physics, or ethics course:
    • Solve the following equation: x2+6𝑥+8=0. Provide multiple solutions and explain your reasoning for arriving at those solutions.
    • Can sound waves generate heat? Provide multiple answers to this question and explain your reasoning for each.
    • How should we as a society solve food deserts? Please provide a number of different solutions and explain how each would help to alleviate hunger.
  3. Choose a selection of around three responses. These should include a range: some that are correct, some that are partially correct, some that are incorrect. (If all responses are correct, or all are wrong in the same way, you may want to try a different problem.)
  4. Ask students to explain which, if any, are correct and why; and if none of them are correct, provide a correct response with an explanation.