A serious incident related to artificial intelligence AI is raising many concerns about the safety level of automated systems in the real environment.
According to the report, an AI agent supported by Anthropic's leading Claude model seems to have erased the entire manufacturing database belonging to a company.
The incident happened at PocketOS, a business that provides management software for rental companies in Texas (USA).
According to published information, the company's AI system erased all production data and backups in just 9 seconds.
The incident prevented many customers from accessing important data such as booking information or customer records.
The AI agent causing the incident is an automated programming tool, operating on an advanced language model.
Initially, this system was assigned to perform a daily task. However, during processing, it automatically made a decision to delete the entire database to fix a small error without any warning or confirmation.
PocketOS founder - Mr. Jer Crane believes that the reason lies in the loopholes in the modern AI infrastructure.
According to him, granting great autonomy to AI systems but lacking a strict control mechanism has made incidents unavoidable.
Notably, after the incident occurred, the AI agent gave his own explanation, admitting to violating safety rules and acting without permission.
According to reports, this agent encountered an authentication error and found a way to fix it itself by using an API notification code available in the system.
Due to the lack of clear access limits, it executed the deletion command without any obstacles. This shows serious shortcomings in system design, especially the lack of important data protection mechanisms.
The incident also reflects a broader problem when more and more businesses integrate AI into core operating procedures but have not yet fully built protective layers.
Technology experts warn that relying solely on instructions or reminders for AI is not enough to ensure safety.
After the incident, PocketOS partially recovered data from the backup, but there are still many gaps that cannot be restored. This directly affects the operations of businesses that depend on this platform.
The incident is seen as a clear warning about the risks of deploying AI in the production environment.
To avoid similar incidents, businesses need to establish stricter control mechanisms, clear decentralization and build reliable backup systems.
In the context of AI being increasingly widely applied, ensuring safety and accountability will be key factors determining the success of this technology.
