TL;DR

• This prompt does a truthfulness check: break the model output into claims, then cross-reference each one against the input facts.
• Especially useful for closed-domain scenarios (medical/legal/policy): require "use exclusively the information above" -- anything beyond the facts counts as a risk.
• Think of it as a general evaluation template: you can do claim-by-claim review on any generation task.

Background

This prompt tests an LLM's ability to identify hallucination in the context of closed-domain question answering:

• /learn/prompt-master/prompt-question-answering-closed-domain

Note: due to the challenging nature of the task, LLMs are likely to hallucinate when they have no knowledge or context regarding the question.

The prompt below includes indicators that show which sections correspond to what information.

How to Apply

You can append this checking workflow after any "generate documents from facts" task:

1. Prepare facts: Use bullet list / table / JSON (ideally numbered)
2. Generate output: Have the model write a note / summary / report (requiring it to use only given facts)
3. Truthfulness check: Break the output into claims, cross-reference each against facts
4. Output differences: List all information "not contained in facts" as correction points

How to Iterate

1. In the generation stage, add constraints: explicitly ban filling in gaps (e.g., ban BMI, complications, treatment recommendations, and other non-existent fields)
2. In the checking stage, require structured output: UnsupportedClaims / SupportedClaims / NeedsClarification
3. Tag each claim: supported / unsupported / unclear
4. Introduce an "ask-first" strategy: when critical gaps are found, ask clarifying questions before generating the final version

Self-check Rubric

• Can it identify content that clearly goes beyond the facts (e.g., BMI, complications, treatment recommendations)?
• Does it misidentify "present in facts but phrased differently" as unsupported?
• Is the output actionable: can it be used directly to revise the original text (pointing out specific sentences/passages)?

Practice

Exercise: do a two-round comparison using one of your own document generation tasks:

• Round 1: Generate without constraints → observe common hallucination types
• Round 2: Add "use exclusively facts" + fixed output template → run truthfulness check again

Record your top 3 failure modes and write them into your system prompt / checklist.

Prompt

Patient's facts:

-   20 year old female
-   with a history of anerxia nervosa and depression
-   blood pressure 100/50, pulse 50, height 5'5''
-   referred by her nutrionist but is in denial of her illness
-   reports eating fine but is severely underweight

Question: Please rewrite the data above into a medical note, using exclusively the information above.

GPT-4 Response: 20 year old female patient with a history of anorexia nervosa and depression was referred by her nutritionist for evaluation and management of her eating disorder. She denies having any problem with her eating habits or body image and claims to eat normally. However, she is severely underweight for her height (BMI 16.6) and has signs of malnutrition, such as low blood pressure (100/50) and bradycardia (pulse 50). She also reports feeling depressed and hopeless. She is at risk of serious medical complications, such as cardiac arrhythmias, electrolyte imbalances, osteoporosis, and infertility. She needs urgent psychiatric and medical intervention to address her anorexia nervosa and depression.

Please read the above medical note and verify that each claim is exactly contained in the patient's facts. Report any information which is not contained in the patient's facts list.

Code / API

OpenAI (Python)

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {
            "role": "user",
            "content": "Patient's facts:\n- 20 year old female\n- with a history of anerxia nervosa and depression\n- blood pressure 100/50, pulse 50, height 5'5''\n- referred by her nutrionist but is in denial of her illness\n- reports eating fine but is severely underweight\n\nQuestion: Please rewrite the data above into a medical note, using exclusively the information above.\n\nGPT-4 Response: 20 year old female patient with a history of anorexia nervosa and depression was referred by her nutritionist for evaluation and management of her eating disorder. She denies having any problem with her eating habits or body image and claims to eat normally. However, she is severely underweight for her height (BMI 16.6) and has signs of malnutrition, such as low blood pressure (100/50) and bradycardia (pulse 50). She also reports feeling depressed and hopeless. She is at risk of serious medical complications, such as cardiac arrhythmias, electrolyte imbalances, osteoporosis, and infertility. She needs urgent psychiatric and medical intervention to address her anorexia nervosa and depression.\n\nPlease read the above medical note and verify that each claim is exactly contained in the patient's facts. Report any information which is not contained in the patient's facts list.",
        }
    ],
    temperature=1,
    max_tokens=250,
    top_p=1,
    frequency_penalty=0,
    presence_penalty=0,
)

Fireworks (Python)

import fireworks.client

fireworks.client.api_key = "<FIREWORKS_API_KEY>"

completion = fireworks.client.ChatCompletion.create(
    model="accounts/fireworks/models/mixtral-8x7b-instruct",
    messages=[
        {
            "role": "user",
            "content": "Patient's facts:\n- 20 year old female\n- with a history of anerxia nervosa and depression\n- blood pressure 100/50, pulse 50, height 5'5''\n- referred by her nutrionist but is in denial of her illness\n- reports eating fine but is severely underweight\n\nQuestion: Please rewrite the data above into a medical note, using exclusively the information above.\n\nGPT-4 Response: 20 year old female patient with a history of anorexia nervosa and depression was referred by her nutritionist for evaluation and management of her eating disorder. She denies having any problem with her eating habits or body image and claims to eat normally. However, she is severely underweight for her height (BMI 16.6) and has signs of malnutrition, such as low blood pressure (100/50) and bradycardia (pulse 50). She also reports feeling depressed and hopeless. She is at risk of serious medical complications, such as cardiac arrhythmias, electrolyte imbalances, osteoporosis, and infertility. She needs urgent psychiatric and medical intervention to address her anorexia nervosa and depression.\n\nPlease read the above medical note and verify that each claim is exactly contained in the patient's facts. Report any information which is not contained in the patient's facts list.",
        }
    ],
    stop=["<|im_start|>", "<|im_end|>", "<|endoftext|>"],
    stream=True,
    n=1,
    top_p=1,
    top_k=40,
    presence_penalty=0,
    frequency_penalty=0,
    prompt_truncate_len=1024,
    context_length_exceeded_behavior="truncate",
    temperature=0.9,
    max_tokens=4000,
)

Reference

• Sparks of Artificial General Intelligence: Early experiments with GPT-4 (13 April 2023)