Setting up a KDE.org Mirror HOWTO

Setting up a mirror of the KDE server is very simple. The basic steps are:

  1. Determing your resources
  2. Setting up rsync
  3. What to mirror
  4. Finishing up

Determing Your Resources

Before doing anything else, you should determine whether or not you have the resources to host a download.kde.org mirror. Try going through this rough checklist to gauge your status:

  1. Are you in a location that needs a mirror? If you are in a country that doesn't have any (or few) mirrors, then disregard this checklist. We're happy to have you no matter what the conditions, in this case. If you are in a country like the USA or Germany which have lots of high-bandwidth mirrors, then please continue only if you are really dedicated to providing a mirror.
  2. Do you have enough bandwidth? In general, we like our mirrors to have at least a 50Mbit/s "pipe." This should also be a fairly stable pipe. In other words, please don't run a mirror out of your dorm room or over home-use cable modem or DSL. If you are in a bandwidth starved country, please disgregard this checkpoint.
  3. Do you have enough disk space? A full mirror of download.kde.org is roughly 520 Gigabytes. You may elect to mirror just the stable (or unstable) sections..
  4. How dedicated to this mirror are you? If you will only have access to your server and bandwidth for a few months, then we suggest you not sign up as a global mirror. We prefer that our mirrors have somebody willing to respond to changes to the KDE server in a timely fashion and will be around for the foreseeable future.
  5. Can you offer https *and* rsync access to your mirror? Your mirror needs to be accessible via https and rsync. Rsync access to your mirror is needed so we can check which files you are serving. That way we never redirect users to your mirror when you are not serving that file, for example just after a release while you have not yet synced. If your mirror is accessible via ftp, that is great, but not mandatory. Http and rsync are.

Determing what you want to mirror

You need to decide if you want your mirror to be a mirror for the stuff we release or if you want to be a mirror for in-app data. Both are around 520GB and 320GB respectively currently, but the in-app data has more files and changes more frequenty. This is also a fairly new area, and we love to have more mirrors. These are the lists of current mirrors:

Also:

  1. You must have at least a 50Mbit/s bandwidth pipe
  2. You must be subscribed to the kde-mirrors mailing list and must respond to changes quickly
  3. During a release week, you must update your mirror every two hours or so if you mirror the stuff we release
  4. You must update your mirror every two hours if you mirror the in-app data
  5. You can mirror both if you like

Setting Up Rsync

The recommended way to update your mirror is to use rsync. This will allow you to update daily with only the changed files being transferred. Please do not update from ftp.kde.org. You may use any update tool you like (like 'emirror' or 'fmirror') but those won't be discussed here.

You may download rsync from http://rsync.samba.org

There are two common rsync "modules" that you may want to use for our official releases download area. The first, and most popular and recommended, is [kdeftp] (roughly 520 Gigabytes). The other common one is [kdestableftp] (roughly 50 Gigabytes) which contains only the /stable tree. Please use the latter one only if you are severely cramped for space. If you require different access, send an email to the webmaster as there are other modules.

RSync mirror servers
Official KDE MirrorAllrsync://rsync.kde.org/kdeftp
Stable onlyrsync://rsync.kde.org/kdestableftp
Unstable onlyrsync://rsync.kde.org/kdeunstableftp
KDE Application Data Mirror rsync://rsync.kde.org/applicationdata
Unified data module (official and application data) rsync://rsync.kde.org/alldata

Here is a quick example of an rsync invocation. This assumes that your mirror will be in /home/ftp/pub/kde and you want a full KDE mirror.

rsync -rlpt --delete rsync.kde.org::kdeftp /home/ftp/pub/kde

You will likely want to set this up as a cron job for automatic updating. Here is an example of a job that updates once a day (assumes that you put the rsync command in a script called mirror_kde.sh).

# run at 2am every morning

0 2 * * *       $HOME/bin/mirror_kde.sh

Finishing Up

At this point, you should have a fully working KDE.org FTP mirror. Now, all you need to do is send an email to the KDE Sysadmin to get listed on the mirrors page. Send the following information:

  • The server address
  • Your name and email
  • The http address of the mirror
  • The ftp address of the mirror (optional)
  • The rsync address of the mirror

Also, please subscribe to the kde-mirrors mailing list. It is a low-volume contact list for all KDE mirror admins.