AI safety: latest AI news

AI safety: incidents, evaluations, alignment research and alarming findings, linked to the papers and reports.

2 Nov Eight independent AI evaluation groups form a forum to set ground rules for testing frontier models
2 Nov Startups such as Hirundo are editing Chinese open-weight models to strip censorship for Western businesses
2 Nov WIRED finds about 400 apps posing as ordinary AI tools that hide sexually abusive deepfake generators
2 Nov TÜV groups propose independent pre-launch testing of frontier AI and agents, led by Germany's AI safety institute
1 Nov Microsoft AI chief Mustafa Suleyman says labs must not build conscious superintelligence
1 Nov VicOne reports 28 zero-day findings in robotics AI software in under two days using AI-assisted research
1 Nov UK shoppers describe being wrongly flagged as shoplifters by AI facial-recognition systems in stores
1 Nov US rescuers describe hikers getting into trouble after planning trips with AI chatbots
1 Nov Anthropic finds Claude agents can write training data that passes its monitor yet makes a research model misaligned
1 Nov Observer investigation says OpenAI's team reviewing flagged ChatGPT chats is small and its backlog ran months
1 Nov Study: Gemini answered 83% of SAT questions from images distorted to mimic a smart-glasses camera
31 Oct Former Google DeepMind safety researcher publishes Delhi talk warning Indian policymakers about AI extinction risk
31 Oct LessWrong post disputes a paper's finding that sparse autoencoders recover only 9% of true features
31 Oct Scott Alexander argues the AI safety movement should stop demanding unilateral pauses by labs, staff and countries
31 Oct Northflank says customers' AI API keys were abused after a breach, one of nine AI platform incidents Wiz counts
31 Oct Florida election sites pulled voter lookup tools during early voting over PHP flaws OpenAI found
31 Oct Daughter says ChatGPT told her mother, dying of colon cancer, she was more likely looking at years
31 Oct Cisco Talos details VOLTA infostealer that asks Claude, Kimi or Gemini to rank what each victim is worth
31 Oct Davidad discusses 'emergent wisdom' in language models as the mirror image of emergent misalignment
31 Oct Cursor says a malicious Claude Code 2.1.313 extension in its marketplace tried to plant an SSH backdoor
31 Oct Anthropic finds Claude can write training data that passes a production-style monitor yet makes a model misaligned
30 Oct CISA official calls Chainguard's early release of PHP2Shell fixes 'dangerous'
30 Oct Chainalysis says an attacker used 'offensive agentic techniques' to breach it and reach customer names and emails
30 Oct OpenAI says ChatGPT will end chats fixated on violence and limit weapons detail for teens
30 Oct OpenAI tightens ChatGPT limits on mass-attack details and teen weapons queries, plans safety report
30 Oct Theranos whistleblower Tyler Shultz calls AI-lab NDAs and non-disparagement deals 'likely illegal'
30 Oct SpyCloud finds Claude Code, Codex and other AI coding-tool tokens in recaptured infostealer logs
30 Oct Study finds self-improving agents can silently delete overlapping safeguards when edits are judged on task success
30 Oct University of Florida study maps the forums and resources sustaining AI-generated non-consensual nude images
29 Oct Epoch AI finds open-weight GLM 5.3 built file-stealing exploits for six Firefox releases using unshipped patches
29 Oct Eclypsium shows compromised server hardware can alter a running model's weights so it gives dangerous answers
29 Oct WIRED: Accenture contractor allegedly linked a secure Pentagon system to a personal Claude Code agent for two months
29 Oct CrowdStrike says China-nexus VAPOR PANDA ran AI agents on DeepSeek against more than 200 organisations
29 Oct Anthropic has Claude grade the safety of each of its own conversations as an offline misuse monitor
29 Oct OpenAI expands misalignment monitors; a replay flags its Hugging Face sandbox escape 30 hours before the breach
29 Oct 20 House Democrats press OpenAI over the firing of three safety researchers and its security incidents
29 Oct Researchers report code-execution and file-theft flaws in the browsers built into five AI coding assistants
29 Oct RBC, BMO, TD and CIBC block personal AI agents such as Meta's Muse from accessing online banking
29 Oct Cursor and Microsoft detail a fake Claude Code extension that planted an SSH backdoor on developer machines
28 Oct Crafted ChatGPT share link could run code in a victim's session and read private chats until OpenAI's fix
28 Oct Anthropic says an unreleased Claude model built working exploits for Chrome, Safari, Android, Windows and Linux
28 Oct Anti-bot script names its functions after ricin and lethal doses so AI models refuse to deobfuscate it
28 Oct Researchers show a shared ChatGPT link could run code in a victim's session and read private chats