APE
Automatic Prompt Engineer: auto-generate and select instructions
Automatic Prompt Engineer: auto-generate and select instructions
A reusable prompt, example set, evaluation record or safety rule with its task boundary preserved.
Run at least one representative case, inspect the result and record what still requires human review.
Image source: Zhou et al. (2022)
Zhou et al. (2022) proposed Automatic Prompt Engineer (APE), a framework for automatic instruction generation and selection. The instruction generation problem is framed as natural language synthesis -- using LLMs as black-box optimizers to generate and search candidate solutions.
The first step involves a large language model (as an inference model) that receives output demonstrations to generate instruction candidates for the task. These candidates guide the search process. A target model executes the instructions, and the best instruction is selected based on computed evaluation scores.
APE discovered a zero-shot CoT prompt that's better than the human-designed "Let's think step by step" prompt (Kojima et al., 2022).
The prompt "Let's work this out in a step by step way to be sure we have the right answer." triggers chain-of-thought reasoning and improved performance on MultiArith and GSM8K benchmarks:
Image source: Zhou et al. (2022)
This paper touches on an important topic in prompt engineering: automatically optimizing prompts. While we don't go deep into this here, here are some key papers if you're interested:
- Prompt-OIRL - Uses offline inverse reinforcement learning to generate query-dependent prompts.
- OPRO - Introduces the idea of using LLMs to optimize prompts: telling LLMs to "take a deep breath" improves math performance.
- AutoPrompt - Proposes a gradient-guided search method for automatically creating prompts for various tasks.
- Prefix Tuning - A lightweight fine-tuning alternative that prepends trainable continuous prefixes for NLG tasks.
- Prompt Tuning - Proposes learning soft prompts through backpropagation.
📚 Related resources
❓ Common questions
Open a question to review the practical answer.
What is APE and why call it "Automatic Prompt Engineer"?
Zhou et al. 2022 reframe instruction generation as natural-language synthesis under black-box optimisation. One LLM acts as the inference model — it reads input-output examples and proposes candidate instructions; a target model then executes each candidate and scores it; the highest scorer wins. No human writes the prompt — hence "automatic prompt engineer".
What was the best zero-shot CoT prompt APE discovered?
The headline result: APE discovered "Let's work this out in a step by step way to be sure we have the right answer." — and it beat Kojima 2022's hand-crafted "Let's think step by step" on both MultiArith and GSM8K. Evidence that LLMs can optimise their own prompts, and that human-written ones aren't necessarily optimal.
What's APE's edge over hand-crafted prompt engineering?
Three: (1) systematic search — hundreds of candidate instructions instead of two or three written on intuition; (2) no engineer-side bias about how the task "should" be phrased; (3) iterative refinement against an evaluation score. Costs: you need a labeled input-output set as the eval rig, and every pass burns a lot of tokens. Worth it when the prompt will be reused at scale once optimised.
What other automatic prompt optimisation methods exist besides APE?
The paper points to several: Prompt-OIRL (offline inverse RL for query-conditional prompts), OPRO (have the LLM "take a deep breath" to optimise math), AutoPrompt (gradient-guided search), Prefix Tuning (lightweight fine-tune alternative), Prompt Tuning (soft-prompt learning via backprop). They split into two camps: search-based (APE / OPRO) vs gradient-based (AutoPrompt / Prefix).
Can I use APE without serious GPU resources?
Yes, but pick the right variant. APE in the original paper hits commercial APIs to evaluate candidates — no GPU needed, just budget for N inference calls. GPU is the price for soft-prompt methods like Prefix Tuning / Prompt Tuning, which need backprop. Small teams can run APE-style search end to end: write 50-100 candidates → score on a small eval set → pick the winner, all on plain API calls.