A hosting provider's connection to the internet is not a single cable. It is a set of links to several upstream carriers, plus connections to internet exchange points where networks swap traffic directly. If one carrier has a bad day, traffic flows over the others.
Inside the building, servers connect to top-of-rack switches, which connect to aggregation switches, which connect to routers at the edge. Each layer is normally duplicated, so a failed switch or cable merely reduces capacity. Protocols such as BGP decide how traffic enters and leaves the network, and the configuration of BGP is a delicate matter, as several large outages have shown.
As a customer, the practical consequences are the quality of the provider's transit (does traffic from your visitors take a direct path?), how well it handles floods of malicious traffic, and whether it offers IPv6, which is now a basic expectation.
Inside the building
Follow a request from the server's point of view. The network card plugs into a top-of-rack switch, usually two cards into two switches, so that the loss of one does not cut the server off. Each top-of-rack switch has uplinks to the aggregation layer, often called spine switches in larger designs, and traffic between any two racks crosses one of these. At the edge, a pair of routers speak to the outside world. The usual speeds today are 10, 25 or 100 gigabits per second inside, with multiple links bundled so one can fail without the group dropping.
Duplication is only worth something if the two paths really are separate. Cables that follow the same tray into the same cabinet, or two routers on one power feed, share a fate. Good designs trace the physical path of each link.
The outside links
Upstream carriers, called transit providers, sell access to the whole internet. A provider with two or three of them, from different companies and entering the building by different routes, is protected against the loss of any one. At internet exchanges the provider connects to many networks at once, including large content networks and consumer ISPs, and traffic to those networks takes a short path without touching a transit carrier. The next piece in this series covers peering and transit in more detail.
Which path a packet takes is decided by BGP, the routing protocol networks use to announce which addresses they can reach. Each network tells its neighbours what it can reach, and the neighbours choose the best option under their own rules. It works on trust and filters. A wrong announcement, in either direction, can send traffic into a dead end for the whole internet. This is the "delicate matter" mentioned above: several well-known outages, accidental leaks of routes among them, came from a mistaken BGP change rather than any broken equipment.
A small example of what BGP carries, with documentation addresses: a provider announces the prefix 203.0.113.0/24 from its own autonomous system number to each carrier. The carriers pass it on, each adding its own number to the front of the path, and routers elsewhere choose the shortest or cheapest path they have heard. If the provider withdraws that prefix by mistake, every other router eventually forgets how to reach those servers, which is exactly what makes the mistake so quick and so total.
Floods and filtering
Every host eventually receives a flood of junk traffic. At the sizes that matter, a link of a few hundred gigabits can be filled by an attacker who rents the right network of compromised devices. Providers defend with filtering at the edge, rate limits, and in some cases a scrubbing service that diverts traffic for a targeted address, removes the attack and passes on the rest. Ask what happens if your IP address is attacked: some providers null-route the address for a while, which protects everyone else but takes you offline.
IPv6
IPv6 adds a second set of addresses, such as 2001:db8::10, alongside IPv4 ones like 203.0.113.10. A good provider gives each server an IPv6 address by default and routes it properly. Many visitors, mobile users especially, reach sites over IPv6 whenever the site offers it.
Look at your own configuration
From a Linux or macOS machine:
traceroute example.com
mtr -rwc 50 example.com
dig example.com AAAA +shortRead the hop list for sudden jumps in latency, and look at the names of intermediate routers, which often contain the carrier name. Compare traces from home, from a phone and from a free online tool in another country. The AAAA lookup shows whether the site has an IPv6 address at all. The troubleshooting guide explains how to tell whether a slow route is your problem or theirs.