A freelance developer was building a small booking system for a client, a physiotherapy practice. Late one evening he pushed his work to a public code repository so that a friend could look at a bug. In the push was a configuration file holding two things that should never have left his machine: the database password for the staging server and the API key for the email service that sent booking confirmations.
He noticed about an hour later, while rereading the commit. He deleted the file, committed the deletion, pushed again, and went to bed relieved that the exposure had been short. By the next morning the email account had sent several thousand messages he had not written, and the provider had suspended it. This is a fictional composite, but the speed of the sequence is accurate to how these incidents unfold.
It is useful to go through it carefully, because most people's instinct in that first hour is the same as his, and it is the wrong one.
What was in the file
The file was an ordinary application config, the sort every framework has: database host, name, user and password, then a block of third-party settings including the email service's API key. Developers write these files to make local setups easy, and the habit of committing them comes from the same convenience. Normally the file is listed in .gitignore so it never reaches the repository. This one had been created before the ignore rule was added, and Git does not forget a file it already tracks.
DB_HOST=db.example.com
DB_USER=booking
DB_PASSWORD=...
MAIL_API_KEY=...Two secrets, two different kinds of damage. The database password allows reading and changing the practice's appointment records, which include names and contact details. The API key allows sending email as the practice, and any holder of the key can use it from anywhere, with no further proof of identity.
How fast the scanners were
Public repositories are watched. Some of the watchers are helpful: code hosting platforms scan pushes for known key formats and notify the owner or the provider. Others are not. Criminal scanners follow the stream of public pushes in near real time, pull out strings that look like credentials, test them against the relevant service, and put working ones to use. The gap between publication and first use can be a few minutes.
Why deleting the file did not help
Git is a history database. Every commit is a permanent snapshot, and a commit that deletes a file only adds a new snapshot in which the file is absent. The earlier snapshot, with the secrets inside it, stays in the history. Anyone can run git log, check out the older commit, or view it on the hosting site, and see exactly what was there.
Several copies existed by morning. The hosting platform keeps the old commits reachable by their hash even after a branch is rewritten. Anyone who had cloned or forked the repository in that hour had their own copy. Search engines and archive services may have kept pages. A leak on the public internet cannot be recalled; it can only be made useless.
The order of response
When the developer woke up to the suspension notice, he did the sequence a support engineer would have recommended, though in a more panicked order than ideal. The right order is below.
- Revoke the email key. In the provider's dashboard, delete the exposed key and create a new one with only the permissions needed to send. The old key stops working immediately.
- Change the database password, then update the application's configuration. Check the database's own users for any he did not create, and check whether it accepted connections from the internet at all. It should only accept them from the application server.
- Look for damage. Review the email provider's logs for what was sent and to whom. Check the database's recent queries or logs for unusual access, and compare table counts to a recent backup.
- Tell the client, with facts: what was exposed, for how long, what has been rotated, and what was or was not accessed. If personal data may have been read, the client may have legal duties to report it, and that decision is theirs, with advice.
- Clean the history, after the secrets are dead, so the repository stops advertising them.
The history rewrite uses a purpose-built tool, because plain Git commands are awkward for this. Something like git filter-repo --path config.env --invert-paths removes the file from every commit, after which a forced push replaces the remote branches. Everyone who cloned has to re-clone, and the hosting platform's support may need to purge cached views. It cleans the record; it does not restore secrecy.
The aftermath
The mail provider had suspended the account for abuse, and getting it reinstated took three days, a call with their trust team and a plain statement of what had happened. In the meantime the practice's appointment reminders did not go out, which was the visible harm. Some of the several thousand messages were phishing, sent as the practice's name, and a few recipients replied to the practice asking what they were. The messages' return addresses also damaged the sending reputation of the practice's domain, and its normal mail was filtered more aggressively for about a fortnight.
The client was understandably unhappy, and then, to his credit, practical. The developer wrote a short incident note: timeline, exposure, actions, and changes. The client kept him on. A frank account, delivered the same day, did more for the relationship than any attempt to soften it would have.
The habits he adopted
He now keeps secrets in environment variables set on the server, or in a secrets file outside the project directory, and the repository holds only a template such as config.example.env with empty values. The real file is listed in .gitignore before the project's first commit. A pre-commit hook runs a secret scanner, such as gitleaks, on every commit and refuses it if it finds anything that looks like a key. He also turned on the hosting platform's own push protection, which stops a push containing a recognised key format.
The less visible changes matter just as much. API keys are now scoped to the least permission they need, so a send-only key cannot read account settings. The database user used by the application cannot drop tables. The database is not reachable from the internet. Keys have owners, and rotation is a habit instead of an emergency.
The takeaway
A pushed secret is a spent secret. Rotate first, rewrite history second, tell the client third, and set things up so the next commit cannot contain a key in the first place.