Moving Week!

So, this past Friday was my last day of work at Bloomington High School North, and I have a week to pack and get moved up North to Fort Wayne, Indiana, where I will begin my new job at Canterbury School May 1st.

It's going to be a hectic week...

You can find the final state of my Image Definition for North (version one point oh) at http://www.north.mccsc.edu/indianaaccess/imagedefinition.html. This describes each and every thing I did to get an image put together for our classrooms from a standard Ubuntu 6.10 Edgy Eft install.

Unfortunately, for some reason (and I don't know if it may be because I installed a few things from source), they don't upgrade smoothly to Feisty Fawn. However, I'll be working with Ubuntu in a very similar environment here at Canterbury, so I'll likely adapt the Image Definition for Feisty.

One of these days, I'll need to figure out how to package things...

I'll no longer be available at sruiz <at> mccsc <dot> edu, so if you want to get a hold of me use my personal e-mail simon.a.ruiz <at> gmail <dot> com.

Powered by ScribeFire.

IU LinuxFest 2007 revisited - with Photos!

I'm back with some photos of the IU LinuxFest!

I don't have anything more to SAY about the LinuxFest, as I've already said it all before. I just wanted to post some photos, cause I like photos ;-)

Click on them to read the Flickr descriptions!

First of all, here is the Indiana Ubuntu LoCo Team Presence:

And one from my presentation:

And that neat airplane computer I mentioned:

Powered by ScribeFire.

Laptops not Autoconfiguring

Frustrating little things seem to pop up right as I feel like I've tied up my loose ends here, like the fact that the laptops, after being imaged or updated were NOT autoconfiguring themselves properly, even though they have the exact same autoconfig script and the only difference in their /etc/rc.local script was what MAC address not to autoconfigure on.

To make it even more perplexing, if I dropped to a command-line and ran $ sudo /etc/rc.local everything went off without a hitch. Since I know /etc/rc.local is run at the end of the boot-up process, I thought MAYBE there's something different on the laptops in the realm of when certain devices are available.

I based this theory on the fact that obviously the scripts were correct, as I could invoke them manually, and the fact that, since the autoconfig scripts starts out with a wget, it would simply error out if the network interface wasn't up yet.

I had to test my theory, so, in order to give the laptops a little more time to get the network adapter up and running, I made the following modification to the laptop's rc.local script:

test `ifconfig | grep -c $GOLDENCLIENTMACADDRESS` == 0 && test `cat /etc/hosts | grep -c nordx--image` != 0 && . /home/administrator/scripts/autoconfig

became

test `ifconfig | grep -c $GOLDENCLIENTMACADDRESS` == 0 && test `cat /etc/hosts | grep -c nordx--image` != 0 && sleep 10 && . /home/administrator/scripts/autoconfig

And it worked. So I'd be inclined to say my theory is correct, or at least useful.

Scripts!

I'm finally done writing out scripts and documenting their use as I look towards my last day of work on Friday.

If you want to check them out, go over to my Image Definition page, the "Scripts" link is at the bottom of the "All Workstations" and "Server" lists.

What kind of scripts?

We use bash scripts to automate a lot of stuff around, which saves us quite a bit of work:

• Automatically reset the default settings on student accounts.
• Automate the setting up of iTALC on teacher workstations.
• Automate common SystemImager tasks such as retrieving images, updating workstations, and setting up USB auto-install disks.
• Automate the configuration of workstation after they've been imaged by SystemImager
• Manage scripts

If any of this sounds useful, go check 'em out.

Powered by ScribeFire.

HOWTO Automatically Configure Clients After Imaging Them with SystemImager (Any Distribution)

You've imaged your 300 workstations, now they're identical

Do you need to run around and manually set up things, making sure that they have unique hostnames? that they they're setup to the right printer? Heck no!

With this handy-dandy system, let them do all that work themselves!

Disclaimer

I've only been working with Linux for about a year and a half, even less than that with SystemImager. There may indeed be a better way to skin this cat. This is what I've figured out and used, and it's worked fine for me. Your Mileage May Vary (YMMV).

Some familiarity with bash scripting will allow you to modify this basic concept to better suit your particular needs. It's just a couple of bash scripts and a text file, but it's saved me a lot of time and work.

The setup

There are a few pieces to this puzzle, so let's describe them and then explain what will happen during the imaging process to save us so much work:

/etc/rc.local

This is a script that is run at the end of the booting-up process, we'll use this in order to set it up so the workstations know when they need to execute the workhorse of this process. We simply add the following lines somewhere in the script (probably the beginning, right after the comments):

# This bit will launch the autoconfiguration routine when it's needed

# i.e. if the MAC address does not match the golden client, and the hostname is nordx--image

GOLDENCLIENTMACADDRESS="00:16:76:B8:A3:1F"

test `ifconfig | grep -c $GOLDENCLIENTMACADDRESS` == 0 && test `cat /etc/hosts | grep -c nordx--image` == ` && . /home/administrator/scripts/autoconfig

Note that you should put your Golden Client's MAC Address there instead of mine ;-)

Also note that we don't have a laptop that is dedicated to be a Golden Client. We use Laptop #1 as a Golden Client, and then we put it out for use. When we need to modify the laptop image, we re-image Laptop #1 back to it's Golden Client state, modify it how we want it, grab the new image, and then we put it back out for use. This means that we cannot rely on the /etc/rc.local script to know when it's time to configure this particular laptop and need to tell it manually. We do that by simply running $ sudo /home/administrator/scripts/autoconfig when we're done using it as a Golden Client.

/var/www/workstations.csv

This comma-delimited text file is stored on the server (nordx-server, here at North) and is available through the server's Apache server (via the wget command in the autoconfig script.). It contains a listing of every workstation with their MAC address, their hostname, what classroom and workstation number they are assigned, and some comments. This information will enable the workhorse of this process to perform the correct operations on every workstation.

A short example of what ours looks like:

MAC Address,Hostname,Classroom,Workstation,Comments

00:16:76:64:6F:A3,nordx--60500,605,0,210L teacher

00:16:76:64:77:FB,nordx--60501,605,1,210L

00:16:76:64:75:E1,nordx--60502,605,2,210L

...

00:13:20:90:7F:16,nordx--60900,609,0,170L teacher

00:13:20:99:D6:BE,nordx--60901,609,1,170L

00:13:20:90:8E:C5,nordx--60902,609,2,170L

...

00:16:76:64:68:E2,nordx--70601,706,0,210L test machine

00:13:20:99:DC:EF,nordx--70602,706,0,170L test machine

00:13:20:D7:E9:E3,,,,170L Replacement 1

00:13:20:99:DD:37,,,,170L Replacement 2

00:13:20:D8:1D:C1,,,,170L Replacement 3

00:15:C5:73:2F:A8 00:16:CF:A4:41:63,nordx--lap01,lap,1,1300

00:15:C5:73:0C:2D 00:16:CF:A4:43:C9,nordx--lap02,lap,2,1300

...

The MAC address is a unique identifier on each computer that will allow us to know, by reading the rest of the line, to know what computer we're at. Note that the laptops have two MAC addresses separated by a space rather than one; this is because they have both an ethernet adapter and a wireless network adapter.

If a computer were to die, we would replace it with one of our replacement computers, and change it's MAC address in this list for the one on the replacement computer, re-image or update the replacement, and it'd be good to go!

/home/administrator/scripts/autoconfig

This is the workhorse of the process. Everything else is in place to make sure this script knows when to run and where it is running.

This script will run all the post-imaging configuration we want to do to the recently-imaged workstations. You can set this up to do whatever you need on that unique workstation based on information gleaned from workstations.csv on the Apache server.

It's pretty long, so I'll just direct you to a copy of the one we're using, if you want to check it out. You'll need a basic understanding of bash to modify it, but bash is pretty human legible, so you can probably get the gist of what's going on by taking a glance.

The important part to note is the very top, here:

# First, let's determine where we are by searching for our MAC address

# in the workstation information file, and pass that information into usable

# variables.



MAC=`ifconfig | grep -m 1 HWaddr | awk -F 'HWaddr ' '{ print $2 }'`

wget http://nordx-server/workstations.csv

CSVLINE=`grep $MAC workstations.csv`

NEWNAME=`echo $CSVLINE | awk -F ',' '{ print $2 }'`

CLASSROOM=`echo $CSVLINE | awk -F ',' '{ print $3 }'`

WORKSTATION=`echo $CSVLINE | awk -F ',' '{ print $4 }'`

rm workstations.csv

We're wgeting the workstations.csv file from the server and then, as the comment says, we're finding out what workstations we're running on by greping for our MAC address. Then we're putting the relevant information into three variables that we can then use to figure out what needs to be done at this workstation: $NEWNAME, $CLASSROOM, and $WORKSTATION. Now, using those variables and a little bit of bash-jitsu, we can configure the workstations however we need.

So, in human language, what's happening?

Every time that a workstation is imaged or updated using SystemImager the workstation is brought back to its pristine proto-form, except for any directories such as the teacher and student home directories, as well as the iTALC private teacher key, that are specified in /etc/systemimager/updateclient.local.exclude.

Then...

• The image wakes up in its new body, it notices that the time has come so it unleashes the autoconfig script.
• The script reconfigures the workstation into its desired form, and then shuts the computer down.

To make a long story short: After you set this up to your liking, your workstations will configure themselves whenever they need to so you won't have to do it for them.

Powered by ScribeFire.

SystemImager Client Update Excludes

The si_updateclient command run on an SystemImager client doesn't just update the client. It searches for any differences between it and the image and deletes, overwrites, or adds any files that don't conform.

What if there are some things you would prefer don't change on the workstation when you update them? You guessed it, exclude to the rescue!

/etc/systemimager/updateclient.local.exclude

This is a configuration file kept on the client that determines what files are left alone during an update. It is delivered with the first image and, as long as /etc/systemimager/updateclient.local.exclude is not actually listed in itself, it will be updated with each update. Keep in mind, however, that it is the file on the client being updated, not the file in the image that matters. Check it out with your favorite super-user editing command:

$ sudo gedit /etc/systemimager/updateclient.local.exclude

What kind of changes might I want to make?

Well, let me talk you through some changes I've made in that file to make things run smoother here.

Under the "Debian stuff" section, I've commented out the /etc/hostname entry. This is because I have an autoconfiguration script that assigns the hostname the first time the workstation boots up after being imaged. By keeping /etc/hostname from remaining the same after an update, I can rename a workstation by modifying the autoconfiguration settings and updating it. If you prefer to have the hostname remain the same, I'd suggest adding /etc/hosts and /etc/samba/smb.conf to the list as well, so those aren't modified either (though this would make updating Samba preferences less than convenient).

...

# Debian stuff

# /etc/hostname

/etc/network/interfaces

...

I've also commented out /home/* in the "generic stuff" section because I want the administrator home and the student account default settings to be updated. I amend this a little later...

...

# generic stuff

/etc/fstab

# /home/*

/mnt/*

...

The following section I added to the end:

# North specific stuff



/home/teacher/*

/home/student/*

/etc/italc/keys/private/*

The first two are to compensate for commenting out /home/* above; I don't want anything that's saved on the users' Desktop to vanish. The last one is to keep from having to set up iTALC's private teacher key after every update of the teacher workstation.

Powered by ScribeFire.

HOWTO Use SystemImager with BitTorrent (Ubuntu 6.10 Edgy Eft)

So, now that you've installed SystemImager, and gotten the basics working, you realize that rsync is a very slow and restricting way of imaging those three-hundred workstations? You wish there was a better way of imaging them, huh? BitTorrent to the rescue!

BitTorrent is a file transfer protocol that alleviates the strain on the originating server by enlisting each downloading participant to help out. So instead of everyone downloading 100% of the file from the server, each participant downloads some of it from each other participant, while uploading what they've already gotten to them. The same amount of data transfer happens, but the server is not responsible for uploading 100% of that; as a result the whole thing happens much more efficiently.

This scales up fantastically so that one server can potentially server an unlimited number of clients, which compares very favorably against the rsync method of distributing files which gobbles up the server's memory and does not scale very well at all.

Don't stop the rsyncd server!

It's probably worth noting that we still need SystemImager's rsyncd server running, even after we get BitTorrent running, as it serves a quick purpose during the imaging process by passing the torrent file to the imaging workstations. Also,updates do not use bittorrent, which is why I script updates to run in serial, not at the same time - again, more on scripts in a future posting.

Install BitTorrent dependencies

$ sudo aptitude install python-wxgtk2.6 python-twisted python-crypto python-psyco python-zopeinterface

Install BitTorrent 4.4.0

This took me a little while to figure out because a) the package in the official Ubuntu repositories is version 3, and b) the deb package advertised at the official bittorrent site is version 5. Neither of these are adequate.

Andrea, SystemImager's developer, asked me to try it with bittorrent 4.4.0

The following is written for Ubuntu 6.10 Edgy Eft.

We first need to go in and remove the bittorrent package through Synaptic package manager. This removes the ubuntu-desktop and gnome-btdownload packages as well, but I couldn't find a way to get this working without removing ubuntu-desktop. PLEASE LET ME KNOW IF YOU KNOW A WAY!

Then we need to grab the deb package for version 4.40 and install it. Either download it to your desktop and double-click on it, or you can just go ahead and click on that link and "Open with" in your browser.

Make sure the SystemImager Bittorent Server is installed.

If you installed SystemImager from source as I outlined in my earlier post, then SystemImager is already set up to work with Bittorrent. No worries, there!

If you've installed SystemImager from packages, then you might need to make sure that you have the SystemImager Bittorent Server package installed.

/etc/systemimager/bittorrent.conf

We need to get in and edit this file, so: $ sudo gedit /etc/systemimager/bittorrent.conf

First, if your network interface is different than eth0, make sure you put that under the BT_INTERFACE option.

Now, in order to save time when we can, let's change the BT_UPDATE option to n or else everytime we turn on the BitTorrent server, it'll re-tar each image directory and create a new torrent file; an unnecessary pain.

Now, make sure you list the images you want to distribute via bittorrent under the BT_IMAGES option. For example, I put BT_IMAGES=210,170,1300 which is all of my images.

The USB boot keys, revisited

If you created USB boot keys in our last episode, we'll need to recreate them with an extra append option, check it out:

$ si_mkautoinstalldisk --device /dev/sda --append "IMAGESERVER=$ImageServerIPAddress IMAGENAME=$ImageName BITTORRENT=y SKIP_LOCAL_CFG=y" --yes

The intricacies of the SystemImager Bittorrent Server

So, in order to be able to image the workstation now, we need to have SystemImager's BitTorrent server running, so let me explain how this works (as far as I've figured it out, anyhow).

When you first run the server, it will tar your images and create torrent files for them; this takes a long time, be patient. Also now, when you update the image on the server, before the bittorrent server starts serving out the new image, you'll need to delete the image's tar and torrent file (located at /var/lib/systemimager/tarballs and /var/lib/systemimager/torrents, respectively) and restart the server so it can re-create the tar and torrent with the updated image. I use scripts so that I don't forget to do this - more on that in a future post.

So, how do you start and restart SystemImager's BitTorrent server? Well, let me tell you:

$ sudo /etc/init.d/systemimager-server-bittorrent start

and

$ sudo /etc/init.d/systemimager-server-bittorrent restart

You may want to put the start command into your /etc/rc.local file.

After you first start your BitTorrent server, you might run $ /etc/init.d/systemimager-server-bittorrent status and make sure that both the tracker and seeder components are running.

Imaging

Imaging works functionally identical to before, just faster: boot to your USB key and let 'er rip.

Now instead of rsyncing the entire filesystem, the imaging workstations will download, via torrent, the tarball of the image (and spit out the percentage progress as it goes), then it un-tars the tarball. Otherwise, everything works the same as before.

I think that's it, re:BitTorrent...

...for now, at least.

Powered by ScribeFire.

CD-ROM wouldn't auto-mount

Ubuntu Edgy on our Dell 210L workstations would not auto-mount CDs...but I could manually mount them just fine.

Show how much I use CDs on them, huh?, if it took me this long to figure that out.

Anyhow, I managed to get them to work properly by running $ sudo mkdir /media/cdrom0 so they had a mount point to use.

Strange that they didn't have that to begin with...

Powered by ScribeFire.

IU LinuxFest

I had a blast!

I had fun manning the booths and representing the Bloomington Linux Users' Group and the Indiana Ubuntu Local Community Team.

There was some really neat stuff including an 8 LCD screen display, and a computer with exposed components built to look like a prop plane.

My presentation, Learning with Linux at every Desk, went very well. I sort of told the story of my experience working with the Indiana ACCESS program here at Bloomington High School North. It was fun, I think, and people seemed interested. I got some really good questions at the end (after a long silence during which I feared I'd bored everyone catatonic). I was even asked to sign a book; I felt like a rock star. ;-)

I did record video of the presentation on that toy Mustek DV-5500 I got at the HECC conference last year. As soon as I figure out how to transcode and compress the 215 MB .ASF file, I'd like to share that with anyone who might want to watch it. Watch this space if you're interested.

HOWTO Use SystemImager Once You Have It Installed (Ubuntu 6.10 Edgy Eft)

Anonymous said...

Can you post a system imager how to? We are a 1:1 school in New Palestine.

Well, since I've been working on this, your wish is my command! :-)

First of all, here is a HOWTO install SystemImager from source on Ubuntu Edgy Eft. The following instructions apply only after you've succesfully installed SystemImager on both the server and at least one client.

Second of all, the number one constraint under which we are working here at North is that we do not control the DHCP server, because of this we miss out on a lot of neat possibilities, and can't rely on the defaults being correct. If you do control your DHCP server, you might have an easier time following the Official Quick-Start instructions. These instructions are designed to work in our environment, and you're encouraged to diverge from them where it makes sense for you to.

Thirdly, I'm writing from the perspective of someone who has installed everything on Ubuntu Edgy Eft. Chances are that most of this applies regardless of your distribution, but it's possible that there are differences between distributions that I don't know about. Forewarned is forearmed.

And finally, these are instructions to get you up and running in the basics, there is a bit more to a smooth SystemImager installation than this, and I'll go more in-depth in the near future about the specific customizations we make here at North to have everything work smoothly, but until you have these basic processes working, those customizations won't do you much good.

Grabbing an image

After you have the golden client set up to your taste, the first thing you want to do is grab its image onto the server.

On the Golden Client, run:

$ sudo si_prepareclient --server $ServerIPorHostname --yes

When that drops you back to a command-line, you're ready, run this on the server:

$ sudo si_getimage --golden-client $GoldenClientIPorHostname --image $ImageName --quiet --exclude '/media/*' --exclude '/lib/klibc/events/*' --exclude '/etc/iftab' --update-script YES --ip-assignment DHCP --post-install reboot

Got that?

I use the golden client's IP address, since they all have the same hostname (nordx--image), and I use the number from their model number as my image name (all the images are the same except from the model they are built for) (170 for the 170Ls, 210 for the 210Ls, 1300 for the Inspiron 1300s). --quite is a personal choice of mine. The first two --excludes I plagiarized from the Official Quick-Start instructions, the third is to keep the imaged workstations from trying to use the wrong network interface information. The rest are to set up the auto-install script for the image so that the workstations reboot when they're done imaging. (Here at North, I have a script that configures the workstation's name, printer options, etc. the first time it is booted after the imaging process and then shuts down the workstation.)

It should be noted that if you control the DHCP server, you're likely running SystemImager on said DHCP server and you should examine what choices you want to make about how you use the auto-install script. You have a lot more flexibility than I do here, and may have better options available to you...

Your server should now be downloading the contents of your golden client via rsyncd and creating an image for you.

Use these same instructions to update your image whenever you modify your golden client (updating packages, installing or uninstalling, or modifying.)

rsync

We'll be using the SystemImager rsyncd server to run all this. This is the most basic way of using SystemImager, the default, and it's functional. It does have drawbacks when you want to image three-hundred workstations, though, so if you have that situation you may want to look at using the multicast and/or bittorrent transport methods after you get the basics set up. We'll discuss bittorrent as a compliment to this system in another post.

We need to make sure the server is running before we distribute images:

$ sudo /etc/init.d/systemimager-server-rsyncd start

If you're going to use the server a lot, it's probably a good idea to add the above command to your /etc/rc.local

Imaging a workstation

So, now that you have an image prepped and ready, how do you blow it out to your waiting workstations??? Well, the way we do it is we set up some USB keys to boot from that take care of everything. You plug a USB key into your server and:

$ sudo umount /dev/sda

$ si_mkautoinstalldisk --device /dev/sda --append "IMAGESERVER=$ImageServerIPAddress IMAGENAME=$ImageName SKIP_LOCAL_CFG=y" --yes

NOTE: Make sure you substitute /dev/sda above for the device path of your USB key. Especially if you have SATA hard disks, or if you have other USB drives mounted, it may not be /dev/sda. An easy way to tell what that is is to plug the USB key in, wait for the icon to appear on the Desktop, then run mount in a terminal and look at the last line, the very first item should be the device path for your USB key.

Now take your shiny auto-install USB key over to a workstation you intend to image and boot from it.

Note, you can pull the USB key out as soon as the kernel and initrd.img are loaded (as soon as the screen text turns grey and starts scrolling really fast) and move on to the next workstation you want to image.

Be aware that using the rsyncd server places certain limits and the more workstation you have imaging, the slower the process will go. rsyncd also takes a lot of memory on the server, so if you don't have much memory, don't do too many workstations. We have 512 MB of memory on the SystemImager server and could do one classroom of 32 workstations overnight ok, though the server slowed to a crawl. When we tried to image two classrooms, 64 workstations, the server started killing off rsyncd processes because it didn't have enough memory to run them all.

Updating a workstation

So, you've imaged the workstation, but a huge critical security patch has been released, or want to install a new application? (Any change you want to make, really)

Well, first update the golden client however you want it. Then grab the image again; there is no difference between grabbing the image the first time and grabbing it for an update.

After you're done with updating the image on the server, go to the workstation you want to update, and run the following:

$ sudo si_updateclient --server $ImageServerIPorHostname --image $ImageName --reboot

It will connect to the server, and download all the new files, (note that any changes made on the workstation will also be overwritten/deleted unless they're in a file or directory specified in /etc/systemimager/updateclient.local.exclude), and reboot.

That's it for the basics

I'll write some more about using bittorent with SystemImager, and such, in the near future.

Powered by ScribeFire.

Indiana Ubuntu Local Community Team - IU LinuxFest

I've totally not mentioned the Indiana Ubuntu Local Community Team on here yet, have I??? Well shame on me!

I've put together, with people from around the state, an Indiana Ubuntu Local Community Team, so if you're in Indiana and use or are interested in using Ubuntu, come check us out! There are plenty of ways of getting a hold of us; we've got a mailing list, an IRC channel, a forum, etc. and we're here to help you.

Incidentally we'll be manning a booth at this year's IU LinuxFest later today, (Eep!). I'll also be giving a presentation at the LinuxFest about the work I've been doing with Linux at North.

I should probably finish up my notes for the presentation and get to bed, huh?

Powered by ScribeFire.

SystemImager Successes

Success!

I've gotten SystemImager to work well and have imaged eight of our nine classrooms to our latest image, and this evening I'm imaging the ninth one. Woohoo!

We had a nasty surprise when we tried to image two classrooms in that the image server simply could not HANDLE 64 rsync connections with 512 MB of memory. Luckily, rather than keel over and die the server started killing off the connections it couldn't handle so instead of having to re-image two whole classrooms, we only had to re-image half of one.

Luckily, I've been able to get SystemImager's bittorrent delivery method up and running so this is a moot point now.

iTALC Memory Limitations

On a side note we're also experiencing memory limitations with iTALC, connecting to 30 or 31 workstations is a bit much for 512 MB of memory, so iTALC periodically dies--luckily it's simply a matter of opening it back up. We're looking into beefing up the memory on our teacher workstations to compensate for this.

Powered by ScribeFire.

Hooray for bug filing!

I just filed a bunch of bugs in Launchpad. Most of them against Samba performance on Ubuntu Edgy (testparm errors by default, which may be the root of the rest of my bugs). One about how our Dell 210Ls randomly (about 3%-10% of the time) do not shut down when told to (they get to the point of powering off and, instead, decide to turn a fan on full blast). And one about how our Dell 170Ls do not show the "thermometer bar" boot-up and shutdown screens.

This is the first time I've filed a bunch of bugs, so let's see how successfully this goes...

A little later

I guess I didn't publish this post when I made it, so I've actually gotten a comment on one of my bugs! I was asked to try some different vga kernel options to see if that would resolve the issue with the 170Ls and their boot/shutdown screens not showing up. It didn't, but it is nice to get suggestions.

Powered by ScribeFire.