…Coding products that make complex changes to multiple files each time. Search tasks that involve gathering and analyzing information from multiple sources for possible relevant information. Workflow: Evaluator-optimizer In the evaluator…
…Related content Making Claude a chemist Coding agents in the social sciences Results from a survey of 1,260 social scientists about AI and coding agent use. Project Glasswing: An initial update…
…likely to replicate in the real world, we simulated longer conversations that real users might naturally have with AI models, and tested whether drift over time led to concerning behavior. To assess…
…We believe this previously unobserved technique is made possible by increases in model intelligence and more capable tooling, notably code execution. This finding raises questions about whether static benchmarks remain reliable when…
…Still, many real-world scientific tasks demand more than that. They require reading papers, querying databases, running experiments, coding and analysis. Now that models can do many of these things, benchmarks have…
…Related content Making Claude a chemist Coding agents in the social sciences Results from a survey of 1,260 social scientists about AI and coding agent use. Project Glasswing: An initial update…
…Our Policy and Safeguards teams turned that list into a classifier that could identify concerning nuclear queries in real-time. Think of a classifier as a specialized labeller, like the one that…
…list of numbers within an impossibly tight time constraint. Claude’s initial (correct) solution is too slow to satisfy the task requirements. It then realizes that all of the tests being used…
…It took us two years to release our first commercial model, and just three more to develop models that can discover severe cybersecurity vulnerabilities , take on a wide range of real work…
…In this year’s US midterm elections, our banner will direct users to TurboVote, a nonpartisan resource from Democracy Works that provides reliable, real-time information about those topics. We’ll implement…