Xen on CentOS Host running FreeBSD Guest
Recently i’ve been investigating Xen. In short, Xen is open source virtualisation software that provides you with the ability to split a physical hardware server (host or dom0) into multiple virtual servers (guest or domU).
What makes Xen so special above the rest is that it offers such a wide span of guest operating systems.
I am a big fan of CentOS so I have chosen this as my host, but Xen as a host will pretty much run on any Linux based OS, NetBSD, or even Solaris.
Just as a side note, the system I will be running this on is P4 3.0 CPU with 2GB ram.
To begin with I tried Xen 3.1 (32 bit PAE SMP) on CentOS 4, during this time I came across the following resources…
- Xen on CentOS-4
- How to Install Windows on Xen
- GettingStarted (XenSource Wiki)
- CentOS Xen Wiki Documents
- CentOS 5 Virtualization Guide
- RedHat Virtualization
- Installing Xen On CentOS 5.0 (i386)
I soon discovered that CentOS 5 had far better native support for Xen than CentOS 4.
Therefore, CentOS 5 was to be used, once installed you could simply do “yum groupinstall Virtualization”, and it would install Xen for you ready to use out of the box. More or less.
So this is great right? Xen installs on CentOS 5 via the virtualisation group pretty much hassle free, which means we have the dom0 setup.
The next step is to create a guest or DomU. There are plenty to choose from (Many Linux Distros, NetBSD, FreeBSD, OpenBSD, Solaris, and even Windows XP & 2003 Server), but we want FreeBSD. This is where the fun begins! We find the following resources:
- FreeBSD 5.3 Xen DomU Install HOWTO - Outdated (For Xen 2.2 not Xen 3.0)
- How to install FreeBSD as domU
- Status of FreeBSD xen guest
- FreeBSD as DomU/Guest HOWTO
Okay, fine no problem. Right?
Yes, until you get an error like this:
ERROR: Non PAE-kernel on PAE host.
Which effectively means that the CentOS 5 Dom0 (Host) is running a PAE kernel, yet what we’re trying to install as a domU (Guest) is a non PAE kernel.
So, what is PAE? basically it gives your system the ability to handle 4GB of RAM or more (max 64GB RAM) , if you have less than 4GB, you don’t really need it. It’s primarily a fix for x86 processors, providing they support it, 64-bit processors don’t need it.
So how do we fix this problem? I investigated and found the following resources:
- 32-bit, 64-bit, and PAE, oh my! (Xen Wiki)
- FreeBSD as domU (YUAN Jue’s Group)
- FreeBSD under Xen (xen-users mail list)
- FreeBSD/xen (rink.nu)
In conclusion there are two options:
- Recompile the CentOS 5 dom0 (Host) with a non-PAE kernel with Xen.
- Recompile the FreeBSD domU (Guest) with a PAE kernel.
So what are the problems?
Option 1 would mean moving away from the native kernel, meaning that each time a new update comes up the kernel would have to be rebuilt manually. Although building the kernel without PAE will decrease overheads, building the kernel manually may lose performance. Considering this is the host, it’s vital that it is stable, therefore this is lesser of an option.
Option 2 is apparently really easy to do, practially a flick of a switch, providing you understand what you are doing with FreeBSD and Xen. Unfortunately, my skills in this area are limited.
Where do we go from here?
Well, considering option 1 is the least favourable option, as it seems, the only way to go is to contact the people who know with regards to running FreeBSD as a DomU…
- Kip Macy
- YUAN Jue
- Rink Springer
- Chris Brookes
Alternativly…
- CentOS Forums
- Xen Mailing Lists
- BSD Forums
Any feedback or comments will be greatfully received.
simoncpu said,
July 5, 2007 @ 2:50 pm
I wish i386 and amd64 FreeBSD Xen 3.1 domUs would be here soon. =)
[ simon.cpu ]
Mark said,
July 11, 2007 @ 3:36 pm
I’ve been investigating this as well and have run into the same problem. Have you already contacted the developers you mentioned?
hm2k said,
July 11, 2007 @ 11:38 pm
Ultimately, I gave up on Xen, and used vmware server.
I’ve yet to contact the developers, the general consensus I got from their sites, etc, is that development on this has stopped.
I feel the Xen open source community is very disconnected.
Mark Foster said,
August 12, 2007 @ 3:26 pm
Good writeup, and nice to know CentOS-5 can do xen-dom0 out of the box.
Too bad PAE gets in the way like that. They should make PAE-enabled domU kernel available or the default. VMware server does a pretty good job of running FreeBSD, another option is jail or qemu.
Ian Bonnycastle said,
November 19, 2007 @ 8:35 pm
I’ve been attempting to get this working for a week, except on an Ubuntu host (dom0). From what I can tell, recompiling the Ubuntu host to run a non-PAE hypervisor and kernel crashes the system (I.E. it complains something about running a non-PAE kernel on a PAE hypervisor, even though I made sure that both are non-PAE, etc.) I’ve also tried to recompile the 7.0-BETA2 FreeBSD kernel to have all the XEN and PAE options enabled, but the stock GENERIC kernel doesn’t have the XEN extensions and the Kip kernel source that I’ve downloaded throws hundreds of compiler errors. So, from what I can tell, there is no way to get a PAE XEN kernel under FreeBSD 7.0 (at least BETA2). Also, when I’ve gotten around the actual compiler errors, I ran across a line that said “PAE under XEN not supported.”, and actually stopped the compilation. So, I would think that using VMware server (which is where I went as well, after attempting Qemu, which I thought would be less CPU intensive, would be a better choice… it crashed too often and doesn’t have very good networking support) is the only viable alternative. I really like the Xen methodology, but its pretty useless if its not supported ubiquitously.
Ian
freebsd software said,
November 22, 2007 @ 9:32 pm
Thanks for the post, it’s very usefull !