logo
10

Function Calling & Tool Use

⏱️ 35分钟

Function calling turns LLMs into orchestration engines. This chapter outlines patterns for stable tool use.

1) When to Use

  • Structured actions: DB queries, API calls, code exec, search.
  • Constrained outputs: prefer tool calls over free-text for reliability.
  • Auditable actions: log tool name/args/results.

2) Tool Schema Design

  • Clear names/descriptions; types for every param; enums for constrained values.
  • Required vs optional fields; defaults kept server-side.
  • Validate inputs server-side; reject/repair before execution.

3) Prompting for Tools

  • System: “Prefer calling tools when helpful; don’t guess params; ask for missing info.”
  • Few-shot: include examples of good tool calls and refusals.
  • Disallow hallucination: remind model to refuse if no suitable tool exists.

4) Execution Loop

while not done:
  ask model → get tool call(s)
  validate/repair args
  run tool in sandbox with timeout
  append tool result back to model
  stop if final answer / max steps / time budget

5) Safety & Limits

  • Timeouts per tool; circuit-break noisy tools.
  • Allowlist domains/APIs; no raw shell without sandbox.
  • PII stripping before tool calls; redact secrets from logs.
  • Idempotency for mutating tools; confirmation steps for risky actions.

6) Error Handling

  • Distinguish user errors (bad params) vs system errors (tool down).
  • Provide concise tool error back to model; let model replan or ask user.
  • Retry with backoff for transient failures; limit total attempts.

7) Multimodal Tools

  • Tools that accept files/URLs: validate size/type; pre-process (OCR/transcript).
  • Return handles/IDs instead of raw blobs; store artifacts with TTL.

8) Testing & Evals

  • Contract tests: schema compliance, required params present.
  • Golden cases: correct tool selection, refusal when no tool fits.
  • Load/chaos: inject tool errors and ensure graceful degradation.

9) Minimal Checklist

  • Strong schemas + validation + allowlists.
  • Sandbox + timeouts + retries + circuit breakers.
  • Logs: tool, args (scrubbed), duration, success/fail, tokens.

📚 相关资源