It's been over a week since I publicly announced the Debian mirrors redirector.
It was covered in several places, including Linux Magazin and mentioned in the latest Debian Project News.
Thanks to all that, and quite a number of early testers of the wheezy image for the Raspberry Pi, http.debian.net has processed over one million requests from APT clients alone. This figure doesn't count users of apt-cacher/apt-cacher-ng/approx, or those who happen to be mirroring Debian with wget. Oh, and this is only the number of requests since last Sunday, when the logs were rotated.
By the way, if you are going to Debconf12 in Nicaragua, why don't you give http.debian.net a try and forget about switching mirrors?
I'm happy to see how it was welcomed and the many emails, blog comments, IRC messages, and other sorts of messages I've received about it. There are surely some rough edges but the service is still under development and all the feedback helps.
In some cases there appear to be some confusion as to how exactly it chooses the mirrors and why sometimes some mirrors are not used. I'll try to cover some of those topics in this blog, hoping the
P.S. If you've sent me an email about http.d.n and I haven't replied, please rest assured I try to take a quick look at them but I can't reply to every single message fast enough.
Thanks to everyone!
June 30, 2012
June 21, 2012
Introducing http.debian.net, Debian's mirrors redirector
You've been there: you are about to install a package, upgrade, or get the source package, and the mirror fails. It is offline, it is out of date, etc. Whatever the reason, you couldn't do it.
That's only one of the issues that http.debian.net attempts to address. In a nutshell: it works as an http-only content distribution network. It uses the network and geographic location to select the best online and up-to-date mirrors.
APT's sources.list one liner:
Use /debian-backports for backports, and /debian-archive for archive.debian.org.
Originally introduced to the Debian mirrors community back in January, the code on the server and client sides has improved since. All is needed is squeeze's APT (or aptitude), but wheezy's and sid's will perform better.
Advantages and more details are discussed at the Debian mirrors redirector's website.
(Thanks to the early testers, everyone who has provided feedback, my friends at PuffinHost, and all the mirrors-related people, obviously including those who sponsor them!)
That's only one of the issues that http.debian.net attempts to address. In a nutshell: it works as an http-only content distribution network. It uses the network and geographic location to select the best online and up-to-date mirrors.
APT's sources.list one liner:
deb http://http.debian.net/debian stable main
Use /debian-backports for backports, and /debian-archive for archive.debian.org.
Originally introduced to the Debian mirrors community back in January, the code on the server and client sides has improved since. All is needed is squeeze's APT (or aptitude), but wheezy's and sid's will perform better.
Advantages and more details are discussed at the Debian mirrors redirector's website.
(Thanks to the early testers, everyone who has provided feedback, my friends at PuffinHost, and all the mirrors-related people, obviously including those who sponsor them!)
June 20, 2012
Not your gsoc project
Thanks to Google, for some years now there has been the so-called Google Summer of Code programme. Quite an interesting project it is.
However, how many of the slots assigned to Debian have resulted in successful projects? how many of them were actually completed by the student and not by the mentor or somebody else? are those projects in fact used or useful at all?
All these questions occurred to me while wondering what would have happened if Joey Hess' git-annex-related kickstarter project had instead been a GSOC project. His kickstart was very successful by the way. Joey: many congratulations!
Not trying to point my finger to the (IMHO many) failed or unsuccessful Debian-related gsoc projects, but I wonder how many successful projects are developed outside the get-paid-in-exchange model. I suspect most, at least in Debian.
What's done for those who actually "deliver" while working on projects as a hobby? is it really worth having GSOC slots for Debian?
However, how many of the slots assigned to Debian have resulted in successful projects? how many of them were actually completed by the student and not by the mentor or somebody else? are those projects in fact used or useful at all?
All these questions occurred to me while wondering what would have happened if Joey Hess' git-annex-related kickstarter project had instead been a GSOC project. His kickstart was very successful by the way. Joey: many congratulations!
Not trying to point my finger to the (IMHO many) failed or unsuccessful Debian-related gsoc projects, but I wonder how many successful projects are developed outside the get-paid-in-exchange model. I suspect most, at least in Debian.
What's done for those who actually "deliver" while working on projects as a hobby? is it really worth having GSOC slots for Debian?
June 19, 2012
Deferring the execution of a command
More than once, I have found myself running a program that calls a given script multiple times, and sometimes, way too many times. In some cases, for instance, it was only necessary to run the script once in one minute, instead of hundreds of times.
Sounds like I want something like dpkg's deferred triggers, right? pretty much so.
With this need, deferred-run was born. Example:
In spite of multiple calls, echo was only called once.
Best part, it only requires sh and flock.
If you are interested in shell trickery, you should take a look at its code.
Sounds like I want something like dpkg's deferred triggers, right? pretty much so.
With this need, deferred-run was born. Example:
$ deferred-run -l lock -- echo hello world $ deferred-run -l lock -- echo hello world $ sleep 3 $ deferred-run -l lock -- echo hello world [some seconds later...] hello world
In spite of multiple calls, echo was only called once.
Best part, it only requires sh and flock.
If you are interested in shell trickery, you should take a look at its code.
May 29, 2012
ldebdiff: what local change did I make to a package?
When fixing or modifying a package I at times change the installed files directly. However, once the changes are okay and it is time to prepare a diff, having to keep track and manually diff'ing the installed files is time consuming.
That's when ldebdiff does its thing: it runs diff against the original files of a given package and the ones installed on a system.
For example:
That's when ldebdiff does its thing: it runs diff against the original files of a given package and the ones installed on a system.
For example:
$ ldebdiff acpi-fakekey --- unpacked/etc/init.d/acpi-fakekey 2012-04-05 05:14:21.000000000 -0500 +++ /etc/init.d/acpi-fakekey 2012-04-12 12:10:55.000000000 -0500 @@ -4,8 +4,8 @@ ### BEGIN INIT INFO # Provides: acpi-fakekey -# Required-Start: $local_fs $remote_fs -# Required-Stop: $local_fs $remote_fs +# Required-Start: $remote_fs +# Required-Stop: $remote_fs # Default-Start: 2 3 4 5 # Default-Stop: # Short-Description: Start acpi_fakekey daemon($remote_fs implies $local_fs)
May 23, 2012
Installed-Size: 0
Redefining the meaning of Installed-Size:
Package: gcipher Version: 1.1-1 Installed-Size: 0
$ dget gcipher=1.1-1 ... $ dpkg -x gcipher_1.1-1_all.deb gcipher $ du -hs gcipher 188K gcipher $ du --apparent-size -hs gcipher 75K gcipher
May 22, 2012
On mirrors and why there are so few
Yes, I said few.
Even if in my blog post about Debian's ever-growing mirrors network it seemed like we have a lot of mirrors, truth be told: they are too few. There are many partial or even complete mirrors out there that are not listed.
If you are the administrator of one of those mirrors, please consider submitting your Debian mirror. It helps keep track of them and expose them to more users.
Granted, there are some that due to policies (on remote connections, bandwidth use, etc.) they are kept private. Hidden behind a LAN, hoping that people inside actually use them. As of this time, there's not much that can be done about them. If they were to be listed somewhere, there's no way for somebody to check them from the outside.
Some other mirror networks require that an IP (or a range) be whitelisted, allowing them to access the mirrors from the outside to perform routine checks. For some reason, I thought Fedora's was one of them but, apparently, they use a different approach: they ask private mirrors to run a tool.
I think that each approach has its pros and cons.
Even if in my blog post about Debian's ever-growing mirrors network it seemed like we have a lot of mirrors, truth be told: they are too few. There are many partial or even complete mirrors out there that are not listed.
If you are the administrator of one of those mirrors, please consider submitting your Debian mirror. It helps keep track of them and expose them to more users.
Granted, there are some that due to policies (on remote connections, bandwidth use, etc.) they are kept private. Hidden behind a LAN, hoping that people inside actually use them. As of this time, there's not much that can be done about them. If they were to be listed somewhere, there's no way for somebody to check them from the outside.
Some other mirror networks require that an IP (or a range) be whitelisted, allowing them to access the mirrors from the outside to perform routine checks. For some reason, I thought Fedora's was one of them but, apparently, they use a different approach: they ask private mirrors to run a tool.
I think that each approach has its pros and cons.
Subscribe to:
Posts (Atom)