Anthropic sues US over national security blacklist
…refusal to let the government strip its safety guardrails , a move that would have allowed its AI to be used for fully autonomous weapons and domestic mass surveillance. A day after Anthropic…
Search
All sources
theverge.com
11
anthropic.com
6
tomshardware.com
5
cnet.com
5
techcrunch.com
4
restofworld.org
4
wired.com
4
spectrum.ieee.org
4
theregister.com
3
gamerant.com
3
en.wikipedia.org
3
9to5google.com
2
People also ask
What are the current safety measures?
Current guardrails are a mix of industry best practices, voluntary screening and government regulation. Many synthesis companies willingly participate in sequence screening initiatives that flag orders matching known pathogens or dangerous sequences. Some generative AI companies also apply content and model-safety policies that block prompts about biological-harm instructions. But in the open letter, experts across the fields of science, technology, public policy, academia and law are proposing tougher measures. The public letter shows a consensus that the risks of AI-developed bioweapons des
AI Leaders Call for Rules on Synthetic DNA to Limit Bioweapons RiskWhy does this matter?
If an AI system is too cautious, it might refuse legitimate nuclear engineering coursework. Too permissive, and it could inadvertently assist bad actors. Our classifier appears to strike the right balance. In preliminary testing with synthetic data, we achieved a 94.8% detection rate for nuclear weapons queries and zero false positives (overall, 96.2% of the classifier’s labels in this test were accurate as shown in Figure 2), suggesting this system would not flag legitimate educational, medical, or research discussions as concerning. This precision matters because nuclear conversations in AI
Developing Nuclear Safeguards for AIVideos
More videosTrump signs executive order to review AI models before they’re released
…OpenAI and Anthropic had already agreed to share their models with CAISI back in 2024 under President Joe Biden as part of Biden’s push for AI safety guardrails. But until recently…
Google Moves Forward With Pentagon AI Deal Despite Employee Pushback
…and national-security AI has shifted since then. Last year, Google removed a previous language from its AI principles that said it would not pursue technologies likely to cause overall harm, weapons…
Microsoft, Nvidia claim AI speeds approval of nuclear plants
…by gutting the safety rules and skipping full environmental reviews for new reactors. AI, we're told, is expected to help by making highly complex work repeatable and predictable, and slashing development…
Top stories
Paper page - When No Benchmark Exists: Validating Comparative LLM Safety Scoring Without Ground-Truth Labels
…Sushant Gautam , , , , , , , , Abstract Comparative safety scoring without labeled benchmarks relies on scenario-based audits with validity chains measuring responsiveness, variance dominance, and stability to establish deployment evidence. AI-generated summary Many deployments…
LLMs and biorisk
…initial focus of our evaluations and the corresponding safety measures was on biological weapons. In this post, we want to expand on our perspective on AI and biological risk (biorisk). It is…
Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable | TechCrunch
…The restrictions on biology come from a similar concern around developing biological weapons . When the AI giant released Mythos in April, it restricted the model to a limited number of companies and…
Discussions and forums
Anthropic Plots Major London Expansion
…Anthropic refused to allow its models to be used in mass surveillance and autonomous weapon systems, leading to an ongoing legal battle between the AI lab and the Pentagon. As part of…
Anthropic sues US government over supply chain risk designation - Engadget
…two of OpenAI's most important safety principles are "prohibitions on domestic mass surveillance and human responsibility for the use of force, including for autonomous weapon systems" — the same issues that got…
White House reportedly considers mandatory government vetting of AI models before release — executive order under discussion
…AI adviser, and Ben Buchanan, a former Biden White House AI adviser, co-authored a New York Times op-ed calling on Congress to mandate third-party audits of AI developers' safety…