After the release of Debian wheezy traffic jumped to about 1 million requests per day, but as the weeks have passed by traffic has continued to increase to 5 million requests every day.
Even though it is a new record for the redirector it can not yet be compared to openSUSE.org's 20-40 million on their mirrorbrain instance. Let's see how long it takes to get there.
User adoption has increased but it has yet to become the default mirror in several places.
June 12, 2013
June 05, 2013
The "let the tool do the work" update
Over the last few weeks I've been making several changes to http.debian.net to detect mirrors that don't follow Debian's mirroring guidelines and end up causing problems to the end users. The changes will mean less hash mismatches and similar errors.
As I wrote back in December, the redirector is becoming nicer but also stricter. Some of the changes I recently made caused over 30 mirrors to be completely disabled from the redirector. This is not ideal and I don't like having to disable mirrors. They are contributions afterall.
The only thing I can do, and can't stress enough, is encourage people to use an up to date ftpsync script (available at project/ftpsync/ on every mirror) to mirror Debian.
It takes care for you of all the little but important things needed. Really. A mirror that uses ftpsync is easier for the administrator to properly configure, and provides a consistent mirror for the benefit of the users.
Speaking of ftpsync, there is a new version! If you use ftpsync please upgrade it as soon as possible.
Other improvements are on their way. Contributions are welcome (if you like refactoring, there's quite a bit of explicitly-redundant code in check.pl that should now give a better idea of the way it needs to be refactored.)
As I wrote back in December, the redirector is becoming nicer but also stricter. Some of the changes I recently made caused over 30 mirrors to be completely disabled from the redirector. This is not ideal and I don't like having to disable mirrors. They are contributions afterall.
The only thing I can do, and can't stress enough, is encourage people to use an up to date ftpsync script (available at project/ftpsync/ on every mirror) to mirror Debian.
It takes care for you of all the little but important things needed. Really. A mirror that uses ftpsync is easier for the administrator to properly configure, and provides a consistent mirror for the benefit of the users.
Speaking of ftpsync, there is a new version! If you use ftpsync please upgrade it as soon as possible.
Other improvements are on their way. Contributions are welcome (if you like refactoring, there's quite a bit of explicitly-redundant code in check.pl that should now give a better idea of the way it needs to be refactored.)
May 22, 2013
Dealing with bashisms in proprietary software
Sometimes it happens that for one reason or another there's a need to use a proprietary application (read: can not be modified due to its licence) that contains bashisms. Since the application can not be modified and it might not be desirable to change the default /bin/sh, dealing with such applications can be a pain. Or not.
The switchsh program (available in Debian) by Marco d'Itri can be used to execute said application under a namespace where bash is bind-mounted on /bin/sh. The result:
Simple, yet handy.
The switchsh program (available in Debian) by Marco d'Itri can be used to execute said application under a namespace where bash is bind-mounted on /bin/sh. The result:
$ sh --help sh: Illegal option -- $ switchsh sh --help | head -n1 GNU bash, version 4.1.5(1)-release-(i486-pc-linux-gnu)
Simple, yet handy.
May 08, 2013
Almost one million requests per day
In the first 48 hours after its log files were rotated last Sunday, http.debian.net handled almost 2 million requests, for an average of 11 requests per second.
In the last weeks before the release of Debian wheezy the number of requests had dropped slightly below 2 million per week.
Debian is alive.
In the last weeks before the release of Debian wheezy the number of requests had dropped slightly below 2 million per week.
Debian is alive.
May 04, 2013
A single address to get Debian Wheezy while it's hot
Already preparing to install or to upgrade to Debian Wheezy?
You can use the http.debian.net redirector to install Debian Wheezy or upgrade to it from Squeeze and make use of Debian's ever-growing 370-large mirrors network to get it.
APT one liner (to be used in your /etc/apt/sources.list file):
During the installation process you can also choose to use it by manually entering http.debian.net as an HTTP mirror and /debian/ as the path.
Get it while it's hot!
You can use the http.debian.net redirector to install Debian Wheezy or upgrade to it from Squeeze and make use of Debian's ever-growing 370-large mirrors network to get it.
APT one liner (to be used in your /etc/apt/sources.list file):
deb http://http.debian.net/debian wheezy main
During the installation process you can also choose to use it by manually entering http.debian.net as an HTTP mirror and /debian/ as the path.
Get it while it's hot!
May 02, 2013
An ever-growing mirrors network, a year later
A year ago I wrote about Debian's ever-growing mirrors network, so it is time to review the numbers.
Compared to the numbers from last year, today Debian is being served via http by about 370 mirrors world-wide, and is also available via ftp from 330 mirrors. So that's an increase of 40 mirrors in one year!
The number of countries with Debian mirrors also increased to 76, 3 more since last year.
This has only been possible thanks to the sponsors hosting the mirrors.
During this year some sponsors have had to retire their mirrors, sometimes ceasing years of contributions to the project and its community.
A big thanks is deserved to past and current sponsors.
Compared to the numbers from last year, today Debian is being served via http by about 370 mirrors world-wide, and is also available via ftp from 330 mirrors. So that's an increase of 40 mirrors in one year!
The number of countries with Debian mirrors also increased to 76, 3 more since last year.
This has only been possible thanks to the sponsors hosting the mirrors.
During this year some sponsors have had to retire their mirrors, sometimes ceasing years of contributions to the project and its community.
A big thanks is deserved to past and current sponsors.
April 08, 2013
How the world ended up in Costa Rica
Even though I haven't had much time to dedicate to http.debian.net lately, it has been up and running, or should I say serving?
Part of its job is to detect mirrors that have temporary issues or are entirely gone, down, unavailable. It does so, and many other things, by monitoring the so-called "trace files". A very important one being the "master" (or "origin") trace file.
With the recent integration of backports into the main archive, the master trace file of the backports mirrors also changed. Long story short, this change caused backports mirrors to no longer be considered by the mirror redirector as candidates. As long as they were up to date.
After the usual mirror synchronisation delay, more and more mirrors were disabled and subsets of "up to date" candidates re-calculated. This reached a critical point when only one mirror was left in the database. The mirror had not been synchronised for a couple of weeks.
This mirror is located in Costa Rica, and as the only candidate left in the database it was the only one used to serve requests for the backports archive. No matter where the client was located in the world.
The issue was later noticed and the necessary updates to the mirrors master list made. Mirrors started to be re-considered as they were re-checked (with some delay due to the rate limiter) and the subsets re-calculated. In a few hours everything was back to normality.
Correctness and fault-tolerance don't always get together very well...
Part of its job is to detect mirrors that have temporary issues or are entirely gone, down, unavailable. It does so, and many other things, by monitoring the so-called "trace files". A very important one being the "master" (or "origin") trace file.
With the recent integration of backports into the main archive, the master trace file of the backports mirrors also changed. Long story short, this change caused backports mirrors to no longer be considered by the mirror redirector as candidates. As long as they were up to date.
After the usual mirror synchronisation delay, more and more mirrors were disabled and subsets of "up to date" candidates re-calculated. This reached a critical point when only one mirror was left in the database. The mirror had not been synchronised for a couple of weeks.
This mirror is located in Costa Rica, and as the only candidate left in the database it was the only one used to serve requests for the backports archive. No matter where the client was located in the world.
The issue was later noticed and the necessary updates to the mirrors master list made. Mirrors started to be re-considered as they were re-checked (with some delay due to the rate limiter) and the subsets re-calculated. In a few hours everything was back to normality.
Correctness and fault-tolerance don't always get together very well...
Subscribe to:
Posts (Atom)