Archive for the ‘Computing’ category

Converting Windows VMWare machines to KVM

February 24th, 2010

I had to do some work on a Windows 2003 virtual server running on VMWare this week which is running on our old playground VMWare install (Server 1.0.4!) and it was sooooo slow I decided enough was enough and it was time to move it on to a sensible platform.

It actually turned out to be quite easy to convert from one platform to the other, with a bit of help from my old boss James Lidderdale.

  1. Shutdown the VMWare server and take a full backup
  2. Boot the VMWare server and uninstall VMWare tools
  3. Apply mergeide.reg registry patch to enable windows to boot on KVM virtual hardware. I’ve no idea where this came from. James had it. I’d like to credit the original author though! Rename .txt to .reg and then merge in as normal.
  4. Shutdown the VMWare server
  5. Convert the vmdk disk image to a single pre-allocated monolithic image:
    • vmware-vdiskmanager -r Server.vmdk -t 2 /some/other/folder/Server.vmdk
  6. Copy the resulting Server-flat.vmdk image over to your KVM server
  7. Now optionally convert the disk to a qcow2 file:
    • qemu-img convert Server-flat.vmdk -O qcow2 Server.qcow2
  8. Finally create a suitable virtual machine definition using that file as the main hard drive. If all went well you should see your VMWare machine boot inside KVM.

The first time I tried this it failed miserably. Turned out that the VMWare machine I was working with had snapshots associated with it. In my case, the disk I needed to flatten with vmware-vdiskmanager was actually Server-000001.vmdk. Once I figured that out it worked first time.

Why I unsubscribed Audible

January 28th, 2010

Because they don’t support Android. Other than that I was a pretty happy customer.

Linux Teacher PC Update

December 11th, 2009

Well we’ve spent a little time on this. Seb’s been helping out and has done the majority of the work so far.

We’ve got the following working:

  • “Domain” logins (against the LDAP database)
  • CCTV Software (via Wine)
  • Themed to look like our standard XP workstations (but not to the extent that you can’t tell the difference. We’re still using the standard Gnome menus). This needs further work.
  • Promethean ActivInspire for Linux (full marks Promethean)

lvm2 Snapshot Write Performance

December 11th, 2009

We’re running an OpenFiler based iSCSI SAN at work and have been very pleased with both its speed and reliability, however I was copying some VM images around the other day and it seemed to me that the disk IO was a little laboured.

A quick Google for lvm2 snapshot performance brought me to an article by Dennis van Dok which prompted me to do a little testing of my own.

Here’s the raw results of the tests Dennis proposes on our system:

Without Snapshot
root@virgil:/mnt/vm# sync; time sh -c “dd if=/dev/zero of=/mnt/vm/test bs=1M count=1000; sync”
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 1.79423 s, 584 MB/s

real    0m9.697s
user    0m0.000s
sys     0m2.530s

With Snapshot
root@virgil:/mnt/vm# rm test
root@virgil:/mnt/vm# sync; time sh -c “dd if=/dev/zero of=/mnt/vm/test bs=1M count=1000; sync”
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 1.67537 s, 626 MB/s

real    0m49.165s
user    0m0.000s
sys     0m2.020s

Now I didn’t carry that out in optimal test conditions. The filesystem is mounted over iSCSI and there’s caching and multipathing in the mix too so this is far from scientific. It’s quite possible that some real traffic from one of the VMs has skewed the result somewhat, but none the less that’s a 5x slow down on write with a snapshot compared to without.

So it’s a tricky dilema. Snapshots are so convenient yet appear to impose such a penalty. We’re going away to rethink how we “do” snapshots.

Cold Callers

November 24th, 2009

It’s 14:15, I should be working, but as I’ve been interrupted twice in the last 15 minutes by <Insert Person Name> from <Insert Company I’ve never heard of> who’s been working with a number of local schools, including ones we exclusively support, I thought I’d take another 2 minutes to express my frustration here.

Five top tips for technology companies calling schools:

  1. Don’t call me. It’s invasive and likely to make me think of your company as desperate for business. Most of the companies we deal with regularly are as a result of word of mouth recommendations or me calling those companies. There is a reason for that.
  2. Don’t bend the truth. If you’re working in a local school then tell me which one but PLEASE don’t try and pretend that you’re big around here when you aren’t. You will be found out.
  3. Don’t post me something and then phone to find out if I got it. Royal Mail are very efficient when it comes to delivering your unsolicited junk to the school, and our lovely post-person is very efficient at getting it to my pigeon hole. I’m equally efficient at getting it from the pigeon hole to the recycling box.
  4. If you simply must phone, please have something interesting to say. Asking me for my 3 year requirements is not my idea of a good use of 10 minutes. We have internal reviews and Becta SRF for that thank you.
  5. Finally if I say I’m not interested in your company, and request that you don’t call again, please respect my wishes. Yes I’m looking at you IT Index/Probrand or whatever you’re called right now. Oh, and getting someone else from the same sales team to call doesn’t excuse this.

Rant over. If that saves me ONE cold call it’ll have paid for itself.

Could 2010 be the year of the Linux Desktop?!

November 20th, 2009

With the release of Windows 7 and Windows XP some 8 years old now, it’s getting to the point where it’s time to make some changes on the desktop.

Without signing the school up to a Microsoft Schools Agreement, we’d be looking at approximately £45,000 to upgrade to Office 2007 and Windows 7. We’d also have to start looking at running Samba 4 at the network core to support Windows 7, or, move to Active Directory which would be an additional cost (server licensing + 2008 R2 CALs).

It seems then that this might be an opportune time to consider alternatives. The school has alot of Linux at the network core. File servers, domain controllers (LDAP + Samba + bind + dhcpd), Learning Platform, VoIP, SAN, Virtualisation Solution and more are all Linux-based and we have the expertise in the technical staff to support those systems.

From a BSF point of view I’m hoping this is a smart move too. It will significantly reduce ongoing licensing costs and is pretty hard to beat for long-term sustainability. It also makes the job of any potential managed service provider bidding on our site significantly harder since the expectation will be that the philosophy of the school should not be fundamentally changed by a switch in service provision.

My Christmas holiday project therefore will be to get an Ubuntu 9.10 box to mirror the functionality of a standard teaching workstation as far as possible. We’re pretty sure we know where the sticking points will be, but like most things, you won’t really know until you try.

Main points as I see them now:

  • Support from Leadership
  • Support from Governing Body
  • Training for Staff (and Students?)
  • Support for Windows-only software
  • Support for Legacy files (Microsoft Publisher being the worst offender)
  • MIS software officially requires Microsoft Office to run and runs on .net 2 and 3.5
  • Desktop Lockdown
  • Monitoring
  • Updating software (local apt repo?)

I’ll be posting here progress as I go along in the “Open Source Schools” category.

Secure Offsite Backups

November 16th, 2009

With more and more people having laptops at work, we needed a way to allow them to backup their files from remote locations securely.

We use Backup Professional from Attix5 for our server offsite backups but it’s frighteningly expensive when you have more than a handful of devices to backup. What I wanted was a system that does the same job, but that’s Open Source.

First I looked at duplicity/duplicati which I’ve used before and would do a good job, but they really require you to do a regular full backup and sometimes we don’t see remote devices for months at a time. We could have setup a very long interval between full backups, but the longer the restore chain, the greater the chance of having problems when you need a file restored.

I’d all but resigned myself to that limitation when I stumbled across BoxBackup. It offers continuous incremental-forever backups and you don’t even have to trust the server you’re backing up to. All the files on the remote server are encrypted and access to the server is by SSL certificates.

It’s not the friendliest bit of software to install, especially on Windows but I’ve made it work on all the machines we’ve tried so far, including a couple of Windows machines. The icing on the cake would be VSS support on Windows and a nice cross-platform GUI to wrap the whole install process and make it alot simpler.

The really nice thing is that you don’t notice it backing up. It runs as a service on Windows and as a daemon on Linux in the background and updates files on the central server once every hour (by default). You can configure it to send you email when it finishes each backup run along with details of how much data was sent.

There’s an ftp-client like interface to view backed-up files and to restore them to your machine. You can also restore directly to a USB drive attached to the server if you need to restore alot of files and don’t have the bandwidth to do it.

FileHunter 0.3

November 16th, 2009

I’ve made a two small changes to FileHunter today:

  • Ignore files with zero length
  • The needles directory is now searched recursively so files in subdirectories of needles are considered when searching haystack.



November 5th, 2009

Ever need to clean a bunch of files off a filesystem? It’s easy enough as long as you know what the files are called.

However, there are certain types of files that some users might want to disguise on a server, which shouldn’t be there. FileHunter takes a folder full of files, “needles” and searches (optionally recursively) the directory “haystack” for occurances of those files – regardless of their filenames, extensions etc. It can then optionally remove those files too.

While it didn’t take me very long to write this, I thought it might be generally useful so it’s available here under GPL v3 or any later version.

It requires Python v2.5 or later and is only tested on Linux (Ubuntu 8.04) at the moment. It’s been run on a filesystem with about 160 needles and 2 million files in the haystack without incident. In theory it should work on Windows too but standard disclaimers apply.


There’s a newer version available here.

Off to LugRadio Live and OggCamp

October 22nd, 2009

Wow, it’s been more than a year since the last LRL. How time flies. The memories of Mr Ben in a Raccoon thong are still disturbingly vivid.

I’m off again tomorrow for THE reason to visit Wolverhampton; which has gained a reputation somewhat for hosting truly awesome FLOSS conferences, allowing said conference delegates the run of the town’s pubs and curry houses, and discounts in its hotels.

I’m hoping that this, the final LRL (alledgedly) will be as good as last year’s finalle event – with the added bonus of getting a second event chucked in courtesy of Dan, Fab, and the Ubuntu UK gang literally feet from where I plan to collapse in a heap after the LRL party on Saturday night.

OggCamp sounds an interesting proposition, if only to get an OggCamp mug and check in with the stars of my favourite Linux podcasts. It’s the first barcamp I’ve been to so I’m not 100% sure what to expect. More anon.