Hosting Autopsy / The migration that forgot the cron jobs

The migration that forgot the cron jobs

HOSTING AUTOPSY

6 min read · 1,326 words

This is a composite case written by the editors. It is built from patterns that come up often in support work and is not the account of a particular named person or company.

A small publishing company moved to a new host over a weekend. The work was done carefully: files copied, database imported, DNS switched, forms tested, a fresh certificate installed. On Monday everything looked right, and the team went back to work.

Two weeks later, scheduled posts stopped appearing. The invoice generator, which ran every night, had not run. The off-site backup script, also a nightly task, had been silent for fourteen days. The cause was straightforward: the old server had a list of scheduled tasks that nobody had written down, and the migration checklist covered files and databases but not that list.

It surfaced only because an editor noticed that a story due on Tuesday had not gone live. Had it been an invoice run, the delay might have gone unnoticed for a month.

The weekend and the fortnight after it

The company was a six-person publisher running a WordPress magazine site, a small subscription shop and a few internal scripts on one shared-hosting account. The move was planned for a quiet Saturday and went to plan. By Sunday evening the new account served the site, mail was flowing and the old account was left running as a fallback.

That last detail matters. The old account kept its crontab, but nothing pointed visitors at it any more, so its jobs ran against a database nobody was using. Nobody saw that either. The old host's account was cancelled at the end of the month, which is when the evidence disappeared for good.

Day 0Move doneNightly jobs silently missingDay 14Story not liveBackups: none
Fourteen days of missed work, and the only visible sign came at the end.

Where scheduled tasks actually live

People say "the cron jobs" as though there were one list. On a typical server there are several places a timer can hide, and a migration that copies the website copies none of them.

PlaceWhat it holdsHow to see it
User crontabJobs for one accountcrontab -l
System cron filesJobs with an explicit usercat /etc/crontab; ls /etc/cron.d
Periodic directoriesScripts run hourly, daily, weeklyls /etc/cron.daily
systemd timersNewer-style schedulessystemctl list-timers --all
Hosting panelJobs stored in the panel databaseThe "Cron Jobs" page
Application schedulerTasks inside WordPress, for examplePlugin or CLI listing

On shared hosting you only get the panel page and your own crontab, which makes the job easier, but the panel's list is easy to forget because it is not in your files.

What the old account was doing

When the old crontab was finally recovered from a panel screenshot, it held four lines. One ran the invoice generator at 01:30. One ran the off-site backup script at 03:00. One cleared a temporary directory on Sundays. The fourth requested wp-cron.php every five minutes.

*/5 * * * * php /home/pubco/public_html/wp-cron.php
30 1 * * * php /home/pubco/scripts/invoices.php
0 3 * * * /home/pubco/scripts/offsite-backup.sh
15 4 * * 0 find /home/pubco/tmp -mtime +7 -delete

None of these looked important in isolation, which is the usual story. Each had been added by a different person at a different time, usually to fix a problem that no longer existed in anyone's memory, and none was recorded anywhere but the server itself.

The fourth line explains the symptom the editor saw. WordPress normally runs its scheduler when someone loads a page, and a developer had turned that off in wp-config.php with define('DISABLE_WP_CRON', true);, relying on the real cron job instead. The config file travelled with the site. The cron job did not. So the new site was told to stop checking for due posts, and nothing else was checking either.

Old serverNew serverSystem cron, every 5 minwp-cron.php runsPosts go live on timeNo cron job createdDISABLE_WP_CRON is trueNothing checks for due posts
The setting moved with the site; the job that made it safe did not.

Wrong turns

The first guess was a plugin conflict, because a scheduling plugin had been updated that week. It was disabled, the story was published by hand, and the next scheduled post failed in exactly the same way. The second guess was the new host's PHP version, which led to an hour of comparing settings that were identical.

The useful question came from the person who had built the site years earlier: "Did we ever have a cron job for this?" Until then, nobody had thought of the scheduler as something that lived outside the site's own files.

How it was diagnosed

With that hint, the sequence was short. Check the constant in wp-config.php: present. Check the new account's cron page: empty. Look at the site's scheduled events with a plugin or wp cron event list: a pile of overdue entries, with timestamps stretching back two weeks. That list is the clearest evidence you can get, because each overdue event shows when it should have run.

The nightly scripts were a separate discovery. Searching the home directory for files touched by nothing since the move turned up invoices.php and offsite-backup.sh, whose last-modified logs ended on the Saturday of the move. Both were referenced in an old runbook nobody had opened.

A word on the "silent" part. A missed job produces no error, no bounce and no log entry on the new server, because nothing ever tried to run. Monitoring tools that watch for failures see nothing to report. You have to monitor for the absence of an event, which is a different habit from watching for errors, and it is the reason heartbeat checks exist.

The fix and the aftermath

The four jobs were recreated in the new panel, and the invoice generator was run once by hand for the missed nights, after checking that it would not double-bill anyone. The backup script ran that night, and the team restored a test file from it the next morning to prove the chain worked.

Two weeks without a backup is a risk that happened to pass without harm. Two weeks of unsent invoices was a cash-flow delay the finance lead had to explain. The company added a "scheduled work" section to its migration template, listing every job, its owner and how to tell that it ran.

The finance lead's sharpest observation, afterwards, was that the invoice job had a human cost the technical team never saw. Customers on thirty-day terms had their clocks start late, and a few queried the delay. Nobody was harmed, but it took a week of polite emails to settle.

A better way to run scheduled work

The rebuilt setup keeps the jobs in a plain text file in the project's own folder, named crontab.txt, so the list is versioned alongside the code. Each job writes one line to a log when it finishes, and a tiny wrapper pings a monitoring address on success. If the ping does not arrive within the expected window, an email goes to a shared mailbox rather than one person's inbox.

30 1 * * * php /home/pubco/scripts/invoices.php && curl -fsS https://monitor.example.com/ping/invoices >/dev/null

The && matters: the ping fires only if the script exits cleanly. That gives you a record of success rather than a record of attempts.

What to look at on your own setup

Before any move, list every scheduler from the table above and copy the output into the migration notes. After the move, repeat the listing on the new server and compare line by line. Then wait for each timer to fire once, rather than assuming. A job that runs nightly takes a night to test, so schedule the move early in the week.

The cron helper is handy for turning a half-remembered schedule into an expression you can read back, and the troubleshooting guide covers the next step when a job runs but does nothing.

What would have caught it

PreviousThe certificate that expired on a Friday eveningNextThe www that was not there

More from Hosting Autopsy

Autopsy

The autoscaler that scaled to meet the bots

A startup used auto-scaling so the site would handle spikes. During one weekend a scraper began requesting...

Autopsy

The contact form that mailed an expired domain

A small manufacturer's contact form sent enquiries to an address at a domain the owner had used years ago for...

Autopsy

The redesign that lost its old addresses

A consultancy relaunched with new page names and a cleaner structure. The designer was proud that the new...