Load time as a tiebreak, and where it fails

Why this tiebreak is tempting

Unlike every other way of choosing, this one is based on something you actually observed. You opened two, one came back sooner, and you have direct evidence rather than a guess. It feels like the only empirical option available.

It is empirical. The problem is what the measurement is of.

What a load time is a measurement of

It measures the whole path between you and the service on that attempt, plus the state of the service at that moment, plus whatever your own connection was doing. The address is a small part of that and arguably not a part of it at all.

A different attempt, minutes later, runs over a different path. The result does not carry over, which is what makes the observation real and the generalisation false.

The belief it creates

After two or three coincidences, a reader concludes that one of the three is the fast one. The belief is comfortable, it explains past experience, and there is no feedback that would ever correct it, because nobody re-runs the comparison under controlled conditions.

It then does a small amount of damage. The fast one becomes the real one, the others become the slow ones, and the reader has quietly built the ranking that assuming a ranking is about.

The second order problem

Once a reader believes one is fast, slowness becomes evidence. A slow load on the good one now means something is wrong, rather than meaning what it actually means, which is that this attempt took a longer path than the last one.

That is how an ordinary variation gets read as a problem, and it is one of the routes into one slow and one quick.

When speed is worth acting on

Within a single sitting, it is fine to use. If one route is slow right now and you are trying to get something done, use another right now. That is a decision about this attempt and it does not claim anything beyond it.

What does not follow is the next step people take, which is to remember the result and apply it next week.

Why no page can rank them for speed either

A page that publishes response times is publishing measurements taken from wherever that page runs. Your path is different, your circuit is your own, and the study on the circuit being yours is about exactly this asymmetry.

This site publishes no timings at all, for the simpler reason that it never opens the addresses it prints and therefore has nothing to time.

A usable rule

1. Slow right now is a reason to try another right now.
2. It is not a reason to prefer one tomorrow.
3. A route being slow is not evidence about the market, or about the address, or about anybody's competence.
4. If all three are slow in the same sitting, the shared factor is more likely to be on your side, which is quiet on your side.

The measurement that would settle it, and why nobody has it

To rank three routes by speed you would need many attempts, from many places, spread over time, with everything else held constant. Nobody is in a position to run that, and if somebody did, the result would describe their paths rather than yours.

This is worth spelling out because the tiebreak feels empirical, and empirical things feel like they should be settleable by better measurement. Here they are not, because the quantity a reader wants to measure is not a property of the thing they are trying to measure it about.

What this study is not. It is not a claim that the three perform identically, only that a difference you observed on one attempt does not belong to the address.