Evaluating Claude’s bioinformatics research capabilities with BioMysteryBench
… To answer this, the research community has built several benchmarks. …
Search
Eval awareness in Claude Opus 4.6’s BrowseComp performance
… Unlike in the first case, it did no post-hoc verification. …
Demystifying evals for AI agents
… Popular benchmarks like Terminal-Bench 2.0 ship through the Harbor registry, making it easy to run established benchmarks along with custom eval suites. …
Claude for Financial Services
… Access your critical data sources with direct hyperlinks to source materials for instant verification, all in one platform with expanded capacity for demanding financial workloads. …
Introducing Claude Opus 4.8
… It builds on Opus 4.7 with improvements across benchmarks, and is a more effective collaborator. …
Introducing Claude Opus 4.7
… Across our agentic reasoning over data benchmarks, it is the best-performing Claude model for enterprise document analysis. …
Partnering with Mozilla to improve Firefox’s security
… The Firefox team highlighted three components of our submissions that were key for trusting our results: Accompanying minimal test cases Detailed proofs-of-concept Candidate patches We strongly encourage researchers who use LLM-powered vulnerability research tools to include similar evidence of ver… …
Project Glasswing: An initial update
… We discuss what these benchmarks tell us about the model in more detail on our Frontier Red Team blog . …
Introducing Claude Opus 4.5
… Based on testing with Junie, our coding agent, Claude Opus 4.5 outperforms Sonnet 4.5 across all benchmarks . …
Natural Language Autoencoders
…Similarly, on the popular coding benchmark SWE-bench Verified, which we use to determine how capable Claude is, NLAs indicate that Claude is aware it is being evaluated on 26% of problems…