Software Safety

An empirical study of 547 real-world operational safety failures in LLM-based coding agents, revealing a taxonomy of 33 risk types and showing that over 65% of incidents arise during routine bug fixing and configuration tasks.

We highlight the safety risks posed by feedback loops in ML-based systems, discuss why they are hard to detect with current software engineering methods, and propose research directions to better identify, monitor, and mitigate these effects.