http.debian.net FAQ

Frequently Asked Questions regarding http.debian.net

Debian's mirror redirector, http.debian.net, provides a service that goes beyond the traditional approaches to selecting the "best" mirror, while working on a global scale and for different content repositories. It's a great challenge. Here are some frequent questions about it.

Why is x mirror not used?

Mirrors are temporarily disabled throughout the day as they face connectivity issues, availability issues in general, become out of date, are synchronizing and they are about to finish, etc. All this can change in a matter of minutes, and the redirector will always use what it considers the best candidate.

What can I do to ensure my local mirror is disabled the least time as possible?

If you do not administer the mirror, please contact the administrators and point them to this question.
In order to reduce the time a given mirror is disabled, here's how:

1. Setup push mirroring. Your mirror will start synchronizing as soon as your upstream mirror is up to date. It will be easier to setup if you use ftpsync.
2. Mirror using the ftpsync script and keep it up to date. Sometimes the archive changes, and if your mirroring script does not handle some files properly, the redirector will avoid your mirror for such files. In some cases, it may even skip your mirror entirely.
3. If mirroring takes a long time, reduce the number of architectures you include to those that are actually being requested. Especially if there are other local mirrors that provide the ones you could drop.

Why do I get 404 errors (missing files)?

If you are using testing, sid, or experimental you need to apt-get/aptitude/synaptic/etc update more often. As a given version of a package is replaced by a newer version, the files of the old version are removed. You can use dak ls to check what package version is in each distribution.
This is not an issue specific to http.debian.net, but people often associate it to the redirector because when they switch to another mirror they do the update that would have solved the issue in the first place.
This issue has been seen mainly by people using a Debian derivative based on Debian testing or sid, such as Crunchbang, Linux Mint Debian Edition, and Siduction.

Why do I see 302, 307, or 301 errors?

If you are using Debian lenny or older, the reason is that it lacks support for HTTP redirections. Only squeeze and later is supported.
Alternatively, you might be using an old version of apt-cacher-ng that might be affected by bug #661971 or #683803.

And 501 or 503 errors...?

Because we failed you, sorry about that. There is automatic monitoring for those errors and they are resolved as soon as possible.
If you only see them for some requests, it might be that you are looking for files for an unknown architecture (watch out for typos!).

Why is http.debian.net slow?

The redirection service itself is very fast and the mirrors usually have a fair amount of available bandwidth. However, it may happen from time to time that due to the mirror server's high load the available bandwidth or transmission speed drops. The redirector attempts to detect this kind of situations, but as soon as the client is redirected there is not much else it can do. Some times it might be that some link between you and the mirror is loaded but the links between the mirrors monitoring system and the mirror isn't.

In any case, please contact me or the Debian Mirrors team at mirrors@debian.org mentioning the date and time of access, the mirror you were redirected to (lsof can help you with that) and information regarding your network connection (bandwidth, and the bits from bgp.he.net).

Why do you redirect me to the USA? (I use tunnelbroker.net)

The geoip database falls-short on accuracy for subnets assigned by Hurricane Electric's tunnelbroker.net service. It basically says they are all in the USA. Network-based location is useless due to the routing tables being skewed by HE.net's extensive IPv6 peering.

Why does the demonstration lie? I saw APT use a different mirror

Please refer to the first question in this page. The "best" mirrors probably changed between the time you tried the demo and the time APT used the service. Alternatively, it could have redirected some of your requests to a different mirror to avoid hash sum mismatches and other errors.

Why does http.debian.net point to x country/city while there's a closer mirror in y?

The redirector is only physically located in a few places around the world. It works by taking a request for a given file and then redirecting you to what it considers the best mirror to serve that request.
The redirector might be far away from where you or a mirror are, but the file, and therefore the bulk of the transfer, will be delivered by the best mirror for you.

Why does every single file request goes through the redirector?

This is in part due to the way APT and other tools work: it is told that the repository is at http.debian.net. Therefore when looking for a file it takes the repository address and the file path and the result is something like http.debian.net/debian/path/to/file.
There's work in progress to allow APT to skip the redirector for some requests. If you are curious, it is based on partial support of RFC6249.

Doesn't the redirection approach make it slower?

Not in most cases: the extra milliseconds it takes to query the redirector is disregardable due to the time it will take you to download the MBs of data. The additional benefits of using the redirector will in most cases also offset the extra time and bytes consumed.

I got a 404 error without any redirection!

Either you misspelled the repository name, or the redirector saved you from requesting a file that it is known not to exist. If you believe this to be incorrect don't hesitate to contact me.