AI Validation

Students evaluate an LLM’s “reasoning” to clarify their own understanding of how to solve a problem.

Assignment Details

AI Theme

Subject

Tools

Skills

Learning Objectives

Identify and apply the standards of good reasoning, such as clarity, relevance, consistency, validity, soundness, and strength

Instructions

  1. Take a challenging problem on the topic your class is working on. It should be a problem where there is either one right answer or where there are answers that are “more right” than others.
  2. Give the problem (and any context you would usually give to students) to an AI tool. Ask it to generate multiple solutions and explain its reasoning. You might try giving it to multiple tools to get a range of answers (e.g., ChatGPT, Claude, Bard, Bing). Here are some sample prompts that might be useful in the context of a mathematics, physics, or ethics course:
    • Solve the following equation: x2+6𝑥+8=0. Provide multiple solutions and explain your reasoning for arriving at those solutions.
    • Can sound waves generate heat? Provide multiple answers to this question and explain your reasoning for each.
    • How should we as a society solve food deserts? Please provide a number of different solutions and explain how each would help to alleviate hunger.
  3. Choose a selection of around three responses. These should include a range: some that are correct, some that are partially correct, some that are incorrect. (If all responses are correct, or all are wrong in the same way, you may want to try a different problem.)
  4. Ask students to explain which, if any, are correct and why; and if none of them are correct, provide a correct response with an explanation.

Related

Debating the Ethics of Generative AI

Facilitate a debate between students about the potential harms and benefits of using generative AI tools in the classroom.

...

AI Misinformation Campaign

Guide students to misuse AI for a misinformation campaign in order to expose AI’s shortcomings and threats while encouraging them to think critically about persuasion, appeals in composition, and credibility.

...