HOWTO Use SystemImager with BitTorrent (Ubuntu 6.10 Edgy Eft)

So, now that you've installed SystemImager, and gotten the basics working, you realize that rsync is a very slow and restricting way of imaging those three-hundred workstations? You wish there was a better way of imaging them, huh? BitTorrent to the rescue!

BitTorrent is a file transfer protocol that alleviates the strain on the originating server by enlisting each downloading participant to help out. So instead of everyone downloading 100% of the file from the server, each participant downloads some of it from each other participant, while uploading what they've already gotten to them. The same amount of data transfer happens, but the server is not responsible for uploading 100% of that; as a result the whole thing happens much more efficiently.

This scales up fantastically so that one server can potentially server an unlimited number of clients, which compares very favorably against the rsync method of distributing files which gobbles up the server's memory and does not scale very well at all.

Don't stop the rsyncd server!

It's probably worth noting that we still need SystemImager's rsyncd server running, even after we get BitTorrent running, as it serves a quick purpose during the imaging process by passing the torrent file to the imaging workstations. Also,updates do not use bittorrent, which is why I script updates to run in serial, not at the same time - again, more on scripts in a future posting.

Install BitTorrent dependencies

$ sudo aptitude install python-wxgtk2.6 python-twisted python-crypto python-psyco python-zopeinterface

Install BitTorrent 4.4.0

This took me a little while to figure out because a) the package in the official Ubuntu repositories is version 3, and b) the deb package advertised at the official bittorrent site is version 5. Neither of these are adequate.

Andrea, SystemImager's developer, asked me to try it with bittorrent 4.4.0

The following is written for Ubuntu 6.10 Edgy Eft.

We first need to go in and remove the bittorrent package through Synaptic package manager. This removes the ubuntu-desktop and gnome-btdownload packages as well, but I couldn't find a way to get this working without removing ubuntu-desktop. PLEASE LET ME KNOW IF YOU KNOW A WAY!

Then we need to grab the deb package for version 4.40 and install it. Either download it to your desktop and double-click on it, or you can just go ahead and click on that link and "Open with" in your browser.

Make sure the SystemImager Bittorent Server is installed.

If you installed SystemImager from source as I outlined in my earlier post, then SystemImager is already set up to work with Bittorrent. No worries, there!

If you've installed SystemImager from packages, then you might need to make sure that you have the SystemImager Bittorent Server package installed.

/etc/systemimager/bittorrent.conf

We need to get in and edit this file, so: $ sudo gedit /etc/systemimager/bittorrent.conf

First, if your network interface is different than eth0, make sure you put that under the BT_INTERFACE option.

Now, in order to save time when we can, let's change the BT_UPDATE option to n or else everytime we turn on the BitTorrent server, it'll re-tar each image directory and create a new torrent file; an unnecessary pain.

Now, make sure you list the images you want to distribute via bittorrent under the BT_IMAGES option. For example, I put BT_IMAGES=210,170,1300 which is all of my images.

The USB boot keys, revisited

If you created USB boot keys in our last episode, we'll need to recreate them with an extra append option, check it out:

$ si_mkautoinstalldisk --device /dev/sda --append "IMAGESERVER=$ImageServerIPAddress IMAGENAME=$ImageName BITTORRENT=y SKIP_LOCAL_CFG=y" --yes

The intricacies of the SystemImager Bittorrent Server

So, in order to be able to image the workstation now, we need to have SystemImager's BitTorrent server running, so let me explain how this works (as far as I've figured it out, anyhow).

When you first run the server, it will tar your images and create torrent files for them; this takes a long time, be patient. Also now, when you update the image on the server, before the bittorrent server starts serving out the new image, you'll need to delete the image's tar and torrent file (located at /var/lib/systemimager/tarballs and /var/lib/systemimager/torrents, respectively) and restart the server so it can re-create the tar and torrent with the updated image. I use scripts so that I don't forget to do this - more on that in a future post.

So, how do you start and restart SystemImager's BitTorrent server? Well, let me tell you:

$ sudo /etc/init.d/systemimager-server-bittorrent start

and

$ sudo /etc/init.d/systemimager-server-bittorrent restart

You may want to put the start command into your /etc/rc.local file.

After you first start your BitTorrent server, you might run $ /etc/init.d/systemimager-server-bittorrent status and make sure that both the tracker and seeder components are running.

Imaging

Imaging works functionally identical to before, just faster: boot to your USB key and let 'er rip.

Now instead of rsyncing the entire filesystem, the imaging workstations will download, via torrent, the tarball of the image (and spit out the percentage progress as it goes), then it un-tars the tarball. Otherwise, everything works the same as before.

I think that's it, re:BitTorrent...

...for now, at least.

Powered by ScribeFire.