Claude 3
Claude 3 overview
#TL;DR（中文）
- 是 Anthropic 的一组code
Claude 3（Haiku/Sonnet/Opus），常见定位是：Haiku 追求速度与成本，Opus 追求更强的综合能力。codeLLM - 适合做：长文本理解（context window）、结构化输出（例如 JSON）、写作与分析，以及部分 任务。code
coding - 选型建议：用你自己的 （10-50 条样本）对比不同 models 的正确性、格式稳定性、以及 hallucination 风险，再决定用哪个版本。code
evaluation
#中文导读（术语保留英文）
阅读这页时建议重点关注：
- context window（以及长上下文场景的可靠性）
- structured output 能力（是否容易 format drift）
- hallucination 与 factual question answering 的表现
如果要落地到产品里，建议把输出格式写死（schema）并加
self-check#Original (English)
Anthropic announces Claude 3, their new family of models that include Claude 3 Haiku, Claude 3 Sonnet, and Claude 3 Opus.
Claude 3 Opus (the strongest model) is reported to outperform GPT-4 and all other models on common benchmarks like MMLU and HumanEval.
#Results and Capabilities
Claude 3 capabilities include advanced reasoning, basic mathematics, analysis, data extraction, forecasting, content creation, code generation, and converting in non-English languages like Spanish, Japanese, and French. The table below demonstrates how Claude 3 compares with other models on several benchmarks with Claude 3 Opus outperforming all the mentioned models:
Claude 3 Haiku is the fastest and most cost-effective model of the series. Claude 3 Sonnet is 2x faster than previous iterations of Claude and Opus is as fast as Claude 2.1 with more superior capabilities.
The Claude 3 models offer support for 200K context windows but can be extended to 1M tokens to select customers. Claude 3 Opus achieved near-perfect recall on the Needle In A Haystack (NIAH) evaluation which measures the model's ability to recall information in a large corpus and effectively process long context prompts.
The models also have strong vision capabilities for processing formats like photos, charts, and graphs.
Anthropic also claim that these models have a more nuanced understanding of requests and make fewer refusals. Opus also shows significant improvements in factual question answering in open-ended questions while reducing incorrect answers or hallucinations. Claude 3 models are also better than the Claude 2 models at producing structured outputs like JSON objects.
