Hosting Autopsy / The auto-reply that answered itself eleven thousand times

The auto-reply that answered itself eleven thousand times

HOSTING AUTOPSY

6 min read · 1,347 words

This is a composite case written by the editors. It is built from patterns that come up often in support work and is not the account of a particular named person or company.

A mid-sized company, the sort with a shared mailbox for general enquiries and a helpdesk system bolted on the side, set up an out-of-office reply on that mailbox. Separately, the helpdesk system had been configured to send an automatic acknowledgement to anything that arrived at the same address. Each automation was reasonable on its own. Nobody tested the pair.

Late on a Friday, someone went on leave and switched the out-of-office on. The first message to hit the mailbox that evening was answered by the helpdesk, the helpdesk's answer was answered by the out-of-office, and the two carried on like that until Monday. Over the weekend the loop produced about eleven thousand messages, filled several mailboxes, and persuaded the mail provider to throttle the whole domain for sending too much.

On Monday morning, legitimate customer mail was delayed for hours. This is a composite, but the mechanics are real and I have seen the same shape more than once.

The two automations

The out-of-office was a standard auto-responder on the mailbox: any message arriving gets a polite "I am away until the 14th, please contact the sales desk" reply. The helpdesk was a ticketing tool that polled the same mailbox and, for every new message, created a ticket and sent the sender a note saying "We have received your request, reference TKT-0000".

The trouble is what each of them considers "a message from a person". Neither checked. The out-of-office saw a message from the helpdesk and thought: someone has written to me, I should reply. The helpdesk saw the out-of-office reply and thought: a customer has written in, open a ticket and acknowledge it. Each reply was a perfectly valid trigger for the other.

Out-of-office reply on the shared mailbox Helpdesk acknowledgement on the same address "We are away" "We got your request"
Each reply counts as a new incoming message for the other system, so the cycle has no natural end.

Friday to Monday

The timeline, reconstructed from the mail logs afterwards, went roughly like this. All the numbers are illustrative of the shape rather than exact.

Total: around eleven thousand messages. That is not a huge volume for a mail server, but it is a very unusual pattern for one sender and the receiving side treated it that way.

What people noticed first

Nobody noticed a loop. The symptoms were all indirect. A salesperson asked why customers were saying their quotes had not arrived. Someone in accounts found that a supplier's invoice had bounced. The helpdesk queue showed a ticket count several times higher than a normal Monday, nearly all of them with subjects beginning "Auto: Out of office" and "Re: Auto: Out of office".

The delay in legitimate mail was the real damage. When a provider throttles a domain, it does not pick out the good messages. Customer mail sat in the same outbound queue as the loop, behind thousands of automatic replies, and each was retried on the provider's slow schedule.

1,800 6,200 3,000 normal Fri night Saturday Sunday Monday Loop messages per day (illustrative)
Volume peaks on Saturday, drops once the provider throttles the domain, and ends only when a person intervenes.

Wrong turns on Monday

The first guess was a spam attack. It looked like one: thousands of messages, a full mailbox, a provider complaint. The team changed the mailbox password and blocked a couple of sender addresses, which did nothing, because the senders were their own systems.

The second guess was a compromised account being used to send bulk mail. That is the standard suspicion when a provider throttles you, and it is worth ruling out, so they did. The sign-in logs were clean. The sending IP was the company's own mail server. Every message in the outbound queue had an empty body apart from a stock sentence.

It was only when somebody opened two adjacent messages and read the subjects that the pattern showed: "Auto: Out of office: We have received your request (TKT-4412)" followed by "We have received your request: Auto: Out of office". The two systems were quoting each other.

How the loop is supposed to be prevented

Mail has had conventions for this since long before helpdesk tools existed. Properly behaved auto-responders do three things. They set an Auto-Submitted: auto-replied header on their own messages, as described in RFC 3834. They refuse to reply to any message that already carries an Auto-Submitted header other than no, or a Precedence: bulk, list or junk header. And they send with an empty envelope sender, written MAIL FROM:<>, so that replies to them are not generated at all.

Auto-Submitted: auto-replied
Precedence: auto_reply
X-Auto-Response-Suppress: All
From: Sales desk <[email protected]>
Subject: Auto: Out of office

Neither of the two systems here did all three. The out-of-office did not look at incoming headers at all. The helpdesk's acknowledgement did not set any marker on its own messages. Either omission alone would have broken the loop; together they guaranteed it.

The fix

The immediate step was to switch off both automations, empty the outbound queue of the loop messages, and let the provider's throttling lift. Then the repair, in order:

  1. Make the out-of-office skip any message with an Auto-Submitted header, a Precedence of bulk, list or junk, or an empty return path.
  2. Make the helpdesk tag its acknowledgements with Auto-Submitted: auto-replied, and have it discard (not ticket) incoming mail carrying that header.
  3. Limit replies per sender: one auto-reply to a given address per day is plenty. Most out-of-office tools have this setting and it was off.
  4. Exclude the helpdesk's own address from the out-of-office rule, as a belt on top of the braces.

They then cleaned up around four thousand junk tickets by filtering on the subject prefix and bulk-closing them, and wrote to the affected customers whose mail had been delayed.

The aftermath was mostly tedium. The provider lifted the throttle after about a day of clean sending, but the domain's reputation with a couple of large mailbox providers took longer to recover, so for the rest of that week some ordinary quotes landed in spam folders. The company added a line to its change checklist: any new automatic mail rule gets a test with a second automatic rule switched on.

Common questions

Could this happen with a mailing list?

Yes, and historically it did a great deal. List software sets Precedence: list and its own headers precisely so that vacation replies are not sent back to the list. A misconfigured list plus an out-of-office is the classic version of this story.

Will the provider's throttle fix it for me?

It slows the loop but does not end it, and it harms your real mail while it runs. Treat throttling as the symptom that something is wrong, not as protection.

Is an empty envelope sender a problem for deliverability?

Not for genuine auto-replies. Bounces use it too, so every receiving server expects it. Do not use it for ordinary mail.

What would have caught it

PreviousThe database backup left in the public folderNextThe contact form that mailed an expired domain

More from Hosting Autopsy

Autopsy

The IPv6 record pointing at nothing

After moving to a new host, a nonprofit updated the A record and thought no more of it. An old AAAA record...

Autopsy

The www that was not there

A consultancy launched a redesigned site and shared the address on a printed brochure as...

Autopsy

The certificate that expired on a Friday evening

An online shop selling hand-made furniture ran for three years on a certificate bought once and renewed by...