Hosting Autopsy / The CDN that served one customer's basket to another

The CDN that served one customer's basket to another

HOSTING AUTOPSY

6 min read · 1,258 words

This is a composite case written by the editors. It is built from patterns that come up often in support work and is not the account of a particular named person or company.

A small boutique selling handmade homeware put its site behind a CDN to make it faster. The setup guide offered an option to cache everything, including the HTML pages, and not just images and stylesheets. The owner turned it on. Speed tests jumped from mediocre to excellent, the site felt snappy, and a pleased message went round the team.

A week later a customer emailed to ask why her basket contained items she had never chosen. She had also seen a first name that was not hers on the page. The owner assumed at first she had mixed up two browser tabs.

She had not. The shop's cart page set a session cookie, but the CDN rule ignored cookies and stored the first response it saw for that address. For a short window, other visitors received that stored copy. This is a composite, with illustrative numbers, but cache leaks of exactly this sort are among the most serious mistakes a small shop can make with a CDN, and they are very easy to make.

What a CDN cache stores, and what it should not

A content delivery network keeps copies of your files on servers close to visitors. When someone asks for an image, the nearest server answers from its copy and your own server is not bothered. That works wonderfully for things that are the same for everyone: logos, product photos, stylesheets, JavaScript.

HTML pages are different, because many of them are not the same for everyone. A product listing is identical for all visitors. A cart page, a checkout step, an account page or a page with "Hello, Maria" in the corner is produced for one person using a session cookie that identifies them. Cache it as if it were a public page, and the next person to ask receives Maria's page.

A CDN is not being faulty when it does this. A rule that says "cache everything" tells it to store anything with a 200 response, whatever the page says about itself. The cache key, the thing used to decide whether two requests are the same, was just the address. Cookies were not part of it.

Visitor A Visitor B CDN cache key: /cart/ only cookies ignored Shop serverbuilds A's cart 1 asks 2 miss 4 B receives A's page 3 stores the response and reuses it
The first response for a URL is stored under the URL alone, so the second visitor never reaches the shop server.

The week it went wrong

The change went live on a Tuesday. By Wednesday the product pages were measurably faster and the owner had noticed fewer abandoned carts. Nobody checked what happened to logged-in or cart pages, because there was nobody to compare against and the pages looked right when the owner tried them.

The reason it looked right is instructive. The owner was always the first to see a page after clearing the cache, or tested on a phone nobody else shared. The first visitor to a URL always gets the right content, as the cache is filled from their request. Only the second and later visitors see the problem, and only while the stored copy remains fresh.

With the cart page set to a default lifetime of a few minutes, the window per leak was short. But the shop had a few hundred visitors a day, and the first customer to open the cart in each window decided what everyone else saw until it expired.

What was exposed

The investigation was careful, because this was now a possible data protection incident. The cart page contained product names, quantities, and, once a visitor had signed in, their first name and the town from their saved delivery address. It did not contain card details, which were handled entirely on the payment provider's pages. It did not contain passwords or the session cookie itself, since the cookie travels in a header and was not part of the body.

Names and partial addresses are still personal data. Using the CDN's logs, the owner could count how many pages had been served from cache with a cart in them, and cross-check them against the shop's order records. About forty visitors had received someone else's page in the week, and six of those pages carried a name.

That made the owner consider whether this counted as a personal data breach under local rules. The correct answer depends on the jurisdiction, the data and the likely harm, so they took advice instead of guessing. Some regimes expect an assessment within a short, fixed period, and a written record of the decision either way. Whatever the conclusion, keeping a record of what happened and what was done is a good habit.

Diagnosis

The customer's email gave the clue, but reproducing the fault took a moment of thought. The owner tried, from two different browsers, to open the cart page. Both showed identical contents, which would never happen with a correctly working cart. Then they looked at the response headers.

$ curl -sI https://example.com/cart/ | grep -i -E 'cache|age|cf-|x-cache|set-cookie'
cache-control: public, max-age=300
x-cache: HIT
age: 212

Three things were wrong. The page was marked public. The CDN reported a HIT, meaning it answered without asking the shop. And the Set-Cookie header that should have accompanied a fresh cart was missing, because the stored response did not include it. That last symptom is why some visitors also found they could not add items: their browser had no session at all.

Request Path is /cart, /checkoutor /account? Session or cartcookie present? Bypass cache:ask the shop Neither: cacheas normal yes yes no
A bypass rule either way: personal paths and any visitor with a session cookie go to the shop, and only the rest are cached.

The fix

The steps, in the order that mattered:

  1. Turn off the cache-everything rule at once, then purge the whole CDN cache so no stored cart pages survived.
  2. Rebuild the rule so that it caches HTML only for anonymous visitors on public paths, and bypasses the cache whenever a cart or login cookie is present.
  3. Add explicit bypass rules for the cart, checkout, account and payment-return paths, whatever the cookies say.
  4. Make the shop send Cache-Control: private, no-store on those pages, so that even a misconfigured CDN has a clear instruction to obey.
  5. Write up the incident, including the numbers and decisions, for the file.

The speed came back almost entirely. Product and category pages for anonymous visitors still came from cache, and those were most of the traffic. The pages that have to be personal now cost a little more, as they should.

Telling the affected customers

The owner wrote to the customers who could be identified as having seen, or had their details seen in, someone else's basket. The message said what happened in two sentences, what information was involved (first name and town, nothing financial), what had been changed, and a direct address for questions. It did not grovel and it did not minimise.

The replies were mostly kind. Two customers asked for their accounts to be deleted, which was done the same day. One, the woman who first wrote in, said she had assumed the shop was hacked and was relieved it was a settings mistake. The owner was careful not to describe it as nothing: it was a real mistake with a real, if limited, effect, and treating it that way is also what the record of the incident said.

What would have caught it

PreviousThe PHP upgrade that broke the checkoutNextThe robots.txt that told everyone to go away

More from Hosting Autopsy

Autopsy

The disk that filled up overnight

A VPS hosted four or five small sites without problems for two years. One morning all of them returned...

Autopsy

The backup that could not be restored

A hobbyist forum had a nightly backup running faithfully for over a year. Every morning the control panel...

Autopsy

The contact form that mailed an expired domain

A small manufacturer's contact form sent enquiries to an address at a domain the owner had used years ago for...