Xen on CentOS Host running FreeBSD Guest

Recently i’ve been investigating Xen. In short, Xen is open source virtualisation software that provides you with the ability to split a physical hardware server (host or dom0) into multiple virtual servers (guest or domU).

What makes Xen so special above the rest is that it offers such a wide span of guest operating systems.

I am a big fan of CentOS so I have chosen this as my host, but Xen as a host will pretty much run on any Linux based OS, NetBSD, or even Solaris.

Just as a side note, the system I will be running this on is P4 3.0 CPU with 2GB ram.

To begin with I tried Xen 3.1 (32 bit PAE SMP) on CentOS 4, during this time I came across the following resources…

  • Xen on CentOS-4
  • How to Install Windows on Xen
  • GettingStarted (XenSource Wiki)
  • CentOS Xen Wiki Documents
  • CentOS 5 Virtualization Guide
  • RedHat Virtualization
  • Installing Xen On CentOS 5.0 (i386)

I soon discovered that CentOS 5 had far better native support for Xen than CentOS 4.

Therefore, CentOS 5 was to be used, once installed you could simply do “yum groupinstall Virtualization”, and it would install Xen for you ready to use out of the box. More or less.

So this is great right? Xen installs on CentOS 5 via the virtualisation group pretty much hassle free, which means we have the dom0 setup.

The next step is to create a guest or DomU. There are plenty to choose from (Many Linux Distros, NetBSD, FreeBSD, OpenBSD, Solaris, and even Windows XP & 2003 Server), but we want FreeBSD. This is where the fun begins! We find the following resources:

  • FreeBSD 5.3 Xen DomU Install HOWTO – Outdated (For Xen 2.2 not Xen 3.0)
  • How to install FreeBSD as domU
  • Status of FreeBSD xen guest
  • FreeBSD as DomU/Guest HOWTO

Okay, fine no problem. Right?

Yes, until you get an error like this:

ERROR: Non PAE-kernel on PAE host.

Which effectively means that the CentOS 5 Dom0 (Host) is running a PAE kernel, yet what we’re trying to install as a domU (Guest) is a non PAE kernel.

So, what is PAE? basically it gives your system the ability to handle 4GB of RAM or more (max 64GB RAM) , if you have less than 4GB, you don’t really need it. It’s primarily a fix for x86 processors, providing they support it, 64-bit processors don’t need it.

So how do we fix this problem? I investigated and found the following resources:

  • 32-bit, 64-bit, and PAE, oh my! (Xen Wiki)
  • FreeBSD as domU (YUAN Jue’s Group)
  • FreeBSD under Xen (xen-users mail list)
  • FreeBSD/xen (rink.nu)

In conclusion there are two options:

  1. Recompile the CentOS 5 dom0 (Host) with a non-PAE kernel with Xen.
  2. Recompile the FreeBSD domU (Guest) with a PAE kernel.

So what are the problems?

Option 1 would mean moving away from the native kernel, meaning that each time a new update comes up the kernel would have to be rebuilt manually. Although building the kernel without PAE will decrease overheads, building the kernel manually may lose performance. Considering this is the host, it’s vital that it is stable, therefore this is lesser of an option.

Option 2 is apparently really easy to do, practially a flick of a switch, providing you understand what you are doing with FreeBSD and Xen. Unfortunately, my skills in this area are limited.

Where do we go from here?

Well, considering option 1 is the least favourable option, as it seems, the only way to go is to contact the people who know with regards to running FreeBSD as a DomU…

  • Kip Macy
  • YUAN Jue
  • Rink Springer
  • Chris Brookes


  • CentOS Forums
  • Xen Mailing Lists
  • BSD Forums

Any feedback or comments will be greatfully received.

Related posts:

  1. FreeBSD 6.2 “/usr/ports: No such file or directory” I recently setup a new FreeBSD 6.2 server, only to...
  2. Installing IonCube Loader with cPanel on CentOS There seems to be limited details of how this is...
  3. Configuring a FreeBSD IRC Shell Server This is a brief guide created to help configure a...
  4. Setting up a FreeBSD socks proxy server for use with mIRC I’m getting fed up with my current IRC BNC software....
  5. Which is the best operating system? I’ve done my research and i’ve discovered there’s only 6...


  1. simoncpu said,

    July 5, 2007 @ 2:50 pm

    I wish i386 and amd64 FreeBSD Xen 3.1 domUs would be here soon. =)

    [ simon.cpu ]

  2. Mark said,

    July 11, 2007 @ 3:36 pm

    I’ve been investigating this as well and have run into the same problem. Have you already contacted the developers you mentioned?

  3. hm2k said,

    July 11, 2007 @ 11:38 pm

    Ultimately, I gave up on Xen, and used vmware server.

    I’ve yet to contact the developers, the general consensus I got from their sites, etc, is that development on this has stopped.

    I feel the Xen open source community is very disconnected.

  4. Mark Foster said,

    August 12, 2007 @ 3:26 pm

    Good writeup, and nice to know CentOS-5 can do xen-dom0 out of the box.
    Too bad PAE gets in the way like that. They should make PAE-enabled domU kernel available or the default. VMware server does a pretty good job of running FreeBSD, another option is jail or qemu.

  5. Ian Bonnycastle said,

    November 19, 2007 @ 8:35 pm

    I’ve been attempting to get this working for a week, except on an Ubuntu host (dom0). From what I can tell, recompiling the Ubuntu host to run a non-PAE hypervisor and kernel crashes the system (I.E. it complains something about running a non-PAE kernel on a PAE hypervisor, even though I made sure that both are non-PAE, etc.) I’ve also tried to recompile the 7.0-BETA2 FreeBSD kernel to have all the XEN and PAE options enabled, but the stock GENERIC kernel doesn’t have the XEN extensions and the Kip kernel source that I’ve downloaded throws hundreds of compiler errors. So, from what I can tell, there is no way to get a PAE XEN kernel under FreeBSD 7.0 (at least BETA2). Also, when I’ve gotten around the actual compiler errors, I ran across a line that said “PAE under XEN not supported.”, and actually stopped the compilation. So, I would think that using VMware server (which is where I went as well, after attempting Qemu, which I thought would be less CPU intensive, would be a better choice… it crashed too often and doesn’t have very good networking support) is the only viable alternative. I really like the Xen methodology, but its pretty useless if its not supported ubiquitously.


  6. freebsd software said,

    November 22, 2007 @ 9:32 pm

    Thanks for the post, it’s very usefull !

  7. BoeroBoy said,

    April 1, 2009 @ 6:45 pm

    That’s a real nice way to blog a topic like this. I don’t even have to google your points because you’ve already done it and posted the links inside your post. Rock on.

  8. Schade said,

    November 4, 2009 @ 8:14 pm

    Seems FreeBSD now works with Xen; at least as DomU under FreeBSD 8.0…


RSS feed for comments on this post · TrackBack URL

Leave a Comment