AI follows good habits a little too literally
Telling AI to back up files before editing them is good practice. On a live web server, it quietly made private source code readable to anyone.
When you let AI work directly on a live website, you give it rules to follow. On a website I worked on, one of those rules was to back up a file before you change it, and the AI followed that rule perfectly. Every edit left a copy of the original next to it, with the date in the file name.
The thing though, a file on a web server is in a public folder and can be downloaded by anyone who guesses its name. The live files were safe because the server runs them instead of showing them, but the backups had a different file ending, so the server simply handed them out as plain text, source code included.
Over a few weeks well over a hundred of these copies piled up, and nothing broke and nothing looked wrong, so that was nothing I noticed until I sent out a security AI agent to look for this kind of thing.
The copies were deleted and the rule was rewritten to keep backups on my own computer. The review also added the security headers the site had been missing.
The lesson
AI follows instructions very precisely, including in places where the instruction has holes in it. This type of thing has ended up in the news several times this year, often in far more serious cases.
The lesson is to always find holes in your instruction yourself before the AI falls into them.
So review what AI has done, not just what it has written. Schedule security reviews the way you schedule content audits, regularly.