Bugzilla Bug 622

I am running Xen 3.0.2-2 with Ubuntu 5.10.  VT hardware is a Pentium D 920
running on an Asus P5WDG2-WS (975X chipset).

I have tried several different versions of FreeBSD as hvm guests, all with a
similar result of BTX halted during the intitial bootloader process.  One of
the versions tested was the FreeBSD 4.11 based m0n0wall, which would serve as a
nice lightweight test case.  GRUB can be used to successfully boot the kernel,
although then drives are not detected instead giving an interrupt error.  If
GRUB loads /boot/loader, a similar BTX halted error as described below occurs.

The error happens every time, in three different situations, so hopefully it is
easily reproducible.  As noted before m0n0wall is an easy ~5 meg download and
should make for a nice test case <http://m0n0.ch/wall/>.

Thanks,
Andy


----

Relevant info is as follows:


- FreeBSD 5.3

An already working FreeBSD 5.3 install (tested standalone without xen) was
attempted as an hvm guest.

-- `freebsdhd` xen config

<standard arch/lib dir stuff>

kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 128
name = "FreeBSD-HD"
acpi=0  # Tried all combinations of acpi off, apic off; apic on, acpi off; ...
apic=0
vif = [ 'type=ioemu, bridge=xenbr0' ]
disk = [ 'phy:/dev/hda,ioemu:hda,w' ]
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
sdl=0
vnc=1
vncviewer=0

-- Boot Output (copied by hand--as seen through vnc--so hopefully no typos)

Plex86/Bochs VGABios current-cvs 13 Apr 2006
This VGA/VBE Bios is released under the GNU LGPL

Please visit :
 . ...

cirrus-compatible VGA is detected

HVMAssist BIOS, 1 cpu, $Revision: 1.138 $ $Date: 2005/05/07 15:55:26 $

ata0 master: QEMU HARDDISK ATA-2 Hard-Disk (-13826 MBytes)
ata0  slave: Unknown Device

Booting from Hard Disk...

F1    FreeBSD

Default: F1


int=0000000d  err=00000000  efl=00010046  eip=000090db
eax=00050033  ebx=00002820  ecx=00000000  edx=0000a000
esi=00009701  edi=000d1f98  ebp=000d7bea  esp=00001800
cs=0008  ds=0000  es=0000    fs=0000  gs=0000  ss=0010
cs:eip=0f 01 15 d0 96 00 00 66-ea e8 90 18 00 b1 20 8e
       d1 8e d9 8e c1 8e e1 8e-e9 48 0f 22 c0 ea fd 90
ss:esp=0a 69 6e 74 3d 30 30 30-30 30 30 30 64 20 20 65
       72 72 3d 30 30 30 30 30-30 30 30 20 20 65 66 6c
BTX halted

<This message, from int= to BTX halted, then repeats indefinitely>

-- Relevant qemu-dm file will be attached



########################


- FreeBSD 4.11 (m0n0wall) -- LVM Backed FS

Standard PC harddrive install of m0n0wall was performed as per
http://m0n0.ch/wall/installation_generic.php on a LVM logical volume.

-- `m0n0wall` xen config

<standard arch/lib dir stuff>

kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 64
name = "m0n0wall"
acpi=0  # Tried all combinations of acpi off, apic off; apic on, acpi off; ...
apic=0
vif = [ 'type=ioemu, bridge=xenbr0' ]
disk = [ 'phy:/dev/xendisks/m0n0wall,ioemu:hda,w' ]
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm.debug'
sdl=0
vnc=1
vncviewer=0

-- Boot Output (copied by hand--as seen through vnc--so hopefully no typos)

Plex86/Bochs VGABios current-cvs 13 Apr 2006
This VGA/VBE Bios is released under the GNU LGPL

Please visit :
 . ...

cirrus-compatible VGA is detected

HVMAssist BIOS, 1 cpu, $Revision: 1.138 $ $Date: 2005/05/07 15:55:26 $

ata0 master: QEMU HARDDISK ATA-2 Hard-Disk (512 MBytes)
ata0  slave: Unknown Device

Booting from Hard Disk...

int=0000000d  err=00000000  efl=00010006  eip=000090ac
eax=00094c00  ebx=00002820  ecx=00000033  edx=0000a000
esi=000096c1  edi=000d1f98  ebp=000d7bea  esp=000017f0
cs=0008  ds=0000  es=0000    fs=0000  gs=0000  ss=0010
cs:eip=ff 35 0c 90 00 00 51 51-51 51 52 b1 07 6a 00 e2
ss:esp=2b 00 00 00 02 02 00 00-00 4c 09 00 33 00 00 00
BTX halted

<Stops and does not repeat>

-- Relevant qemu-dm file will be attached




########################


- FreeBSD 4.11 (m0n0wall) -- CD-ROM iso Image

-- `m0n0walliso` xen config

<standard arch/lib dir stuff>

kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 64
name = "m0n0wall"
acpi=0  # Tried all combinations of acpi off, apic off; apic on, acpi off; ...
apic=0
vif = [ 'type=ioemu, bridge=xenbr0' ]
cdrom='/home/q3aiml/cdrom-1.22.iso'
boot='d'
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm.debug'
sdl=0
vnc=1
vncviewer=0

-- Boot Output (copied by hand--as seen through vnc--so hopefully no typos)

Plex86/Bochs VGABios current-cvs 13 Apr 2006
This VGA/VBE Bios is released under the GNU LGPL

Please visit :
 . ...

cirrus-compatible VGA is detected

HVMAssist BIOS, 1 cpu, $Revision: 1.138 $ $Date: 2005/05/07 15:55:26 $

ata0 master: QEMU CD-ROM ATAPI-4 CD-Rom/DVD-Rom
ata0  slave: Unknown Device

Booting from CD-Rom...
CD Loader 1.2

Building the boot loader arguments
Looking up /BOOT/LOADER... Found
Relocating the loader and the BTX
Starting the BTX loader

int=0000000d  err=00000000  efl=00010006  eip=000090ac
eax=00094c00  ebx=00032820  ecx=00000033  edx=0000a000
esi=002096c1  edi=000d1f98  ebp=000d0000  esp=000017f0
cs=0008  ds=0000  es=0000    fs=0000  gs=0000  ss=0010
cs:eip=ff 35 0c 90 00 00 51 51-51 51 52 b1 07 6a 00 e2
ss:esp=2b 00 00 00 02 02 00 00-00 4c 09 00 33 00 00 00
BTX halted

<Stops and does not repeat>

-- Relevant qemu-dm file will be attached

Created an attachment (id=374) [edit]
qemu-dm for FreeBSD 5.3

Created an attachment (id=375) [edit]
qemu-dm.log for m0n0wall harddisk install

Created an attachment (id=376) [edit]
qemu-dm.log for m0n0wall iso

Actually it appears that using the m0n0wall hard disk install with GRUB does
detect the partitions, unlike stated before.  The FreeBSD 5.3 install still
does not detect them.  I wonder if the drive is too big (hence the negative
space as reported by the ROM BIOS, heh...).  I will continue to test and post
results.

I'm experiencing the same problem.  It seems to be referenced in this freeBSD
mailing list entry. 
http://lists.freebsd.org/pipermail/freebsd-hackers/2006-July/017172.html

The problem seems to occur on both the FreeSBIE live CD from
http://www.freesbie.org/ as well as a 4.11 drive image copied from one of our
working servers.  


#  -*- mode: python; -*-
#============================================================================
# Python configuration setup for 'xm create'.
# This script sets the parameters used when a domain is created using 'xm
create'.
# You use a separate script for each domain you want to create, or
# you can set the parameters for the domain on the xm command line.
#============================================================================

import os, re
arch = os.uname()[4]
if re.search('64', arch):
    arch_libdir = 'lib64'
else:
    arch_libdir = 'lib'

#----------------------------------------------------------------------------
# Kernel image file.
kernel = "/usr/lib/xen/boot/hvmloader"

# The domain build function. HVM domain uses 'hvm'.
builder='hvm'

# Initial memory allocation (in megabytes) for the new domain.
memory = 384

# A name for your domain. All domains must have different names.
name = "NS1"

#-----------------------------------------------------------------------------
# the number of cpus guest platform has, default=1
#vcpus=1

# enable/disable HVM guest PAE, default=0 (disabled)
#pae=0

# enable/disable HVM guest ACPI, default=0 (disabled)
#acpi=0

# enable/disable HVM guest APIC, default=0 (disabled)
#apic=0

# List of which CPUS this domain is allowed to use, default Xen picks
#cpus = ""         # leave to Xen to pick
#cpus = "0"        # all vcpus run on CPU0
#cpus = "0-3,5,^1" # run on cpus 0,2,3,5

# Optionally define mac and/or bridge for the network interfaces.
# Random MACs are assigned if not given.
#vif = [ 'type=ioemu, mac=00:16:3e:00:00:11, bridge=xenbr0' ]
# type=ioemu specify the NIC is an ioemu device not netfront
vif = [ 'type=ioemu, bridge=xenbr0' ]

#----------------------------------------------------------------------------
# Define the disk devices you want the domain to have access to, and
# what you want them accessible as.
# Each disk entry is of the form phy:UNAME,DEV,MODE
# where UNAME is the device, DEV is the device name the domain will see,
# and MODE is r for read-only, w for read-write.

#disk = [ 'phy:hda1,hda1,w' ]
#disk = [ 'phy:hda,ioemu:hda,w' ]
#disk = [ 'file:/stuff/vm1disk,ioemu:hda,w' ]

#----------------------------------------------------------------------------
# Configure the behaviour when a domain exits.  There are three 'reasons'
# for a domain to stop: poweroff, reboot, and crash.  For each of these you
# may specify:
#
#   "destroy",        meaning that the domain is cleaned up as normal;
#   "restart",        meaning that a new domain is started in place of the old
#                     one;
#   "preserve",       meaning that no clean-up is done until the domain is
#                     manually destroyed (using xm destroy, for example); or
#   "rename-restart", meaning that the old domain is not cleaned up, but is
#                     renamed and a new domain started in its place.
#
# The default is
#
#   on_poweroff = 'destroy'
#   on_reboot   = 'restart'
#   on_crash    = 'restart'
#
# For backwards compatibility we also support the deprecated option restart
#
# restart = 'onreboot' means on_poweroff = 'destroy'
#                            on_reboot   = 'restart'
#                            on_crash    = 'destroy'
#
# restart = 'always'   means on_poweroff = 'restart'
#                            on_reboot   = 'restart'
#                            on_crash    = 'restart'
#
# restart = 'never'    means on_poweroff = 'destroy'
#                            on_reboot   = 'destroy'
#                            on_crash    = 'destroy'

#on_poweroff = 'destroy'
#on_reboot   = 'restart'
#on_crash    = 'restart'

#============================================================================

# New stuff
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'

#-----------------------------------------------------------------------------
# Disk image for
#cdrom=
cdrom='/home/cody/FreeSBIE-1.1-i386.iso'

#-----------------------------------------------------------------------------
# boot on floppy (a), hard disk (c) or CD-ROM (d)
#boot=[a|c|d]
boot='d'
#-----------------------------------------------------------------------------
#  write to temporary files instead of disk image files
#snapshot=1

#----------------------------------------------------------------------------
# enable SDL library for graphics, default = 0
sdl=0

#----------------------------------------------------------------------------
# enable VNC library for graphics, default = 1
vnc=1

#----------------------------------------------------------------------------
# enable spawning vncviewer(only valid when vnc=1), default = 1
vncviewer=0

#----------------------------------------------------------------------------
# no graphics, use serial port
nographic=0

#----------------------------------------------------------------------------
# enable stdvga, default = 0 (use cirrus logic device model)
stdvga=0

#-----------------------------------------------------------------------------
#   serial port re-direct to pty deivce, /dev/pts/n
#   then xm console or minicom can connect
serial='pty'

#----------------------------------------------------------------------------
# enable ne2000, default = 0(use pcnet)
ne2000=0


#-----------------------------------------------------------------------------
#   enable audio support
#audio=1


#-----------------------------------------------------------------------------
#    set the real time clock to local time [default=0 i.e. set to utc]
#localtime=1


#-----------------------------------------------------------------------------
#    start in full screen
#full-screen=0

I'm experiencing this same problem with the current tip of xen-unstable.hg
(changeset 11698).  VT hardware is a Xeon 5140, 2.33GHz on a Supermicro X7DBE+
motherboard & 3ware 9550SX-12 SATA RAID controller.

I also encounter the problem for FreeBSD 5.3 and 5.4, with Xen 3.0.3, Debian
Sid packaged, Intel Core2Duo T7200 (VT), kernel 2.6.18-xen-amd64 debian sid
packaged.

The same problem still persists in Xen 3.1.0,
at least with FreeBSD 6.1 and 6.2 (and likely
all other variants using the same BTX code).

Hello, 

I'm having the same problem with stock Xen 3.1.0 on a HVM system (Athlon X2)
and FreeBSD 6.1. Has anyone ever found a workaround of any kind? I'm a little
surprised this issue has been around since at least April 2006 without any
clear progress made. Does anyone have any pointers?

Thanks in advance.

The problem still exists. I've tried somewhat all versions of Xen and all
versions of FreeBSD.

If needed I can supply all logs and exact BTX halted codes.

(In reply to comment #9)
> Has anyone ever found a workaround of any kind? I'm a little
> surprised this issue has been around since at least April 2006 without any
> clear progress made. Does anyone have any pointers?
> 

I assume using GRUB rather than the FreeBSD bootloader still works as
originally stated. As for if the drive detection problem is still around...?

(In reply to comment #8)
> The same problem still persists in Xen 3.1.0,
> at least with FreeBSD 6.1 and 6.2 (and likely
> all other variants using the same BTX code).


attempts @ HVM installs FreeBSD 6.3-RELEASE & 7.0-RC1 exhibit the same problem
under Xen 3.1.2 (Fedora8 Dom0, '3.1.2-1.fc8') on amd64.

tried various combos of OS_Type/Variant (generic/generic, unix/freebsd6) with
no luck ...

I was unable to get the grub chainloader configuration to work under FreeBSD
4.11 which  resulted in "BTX halted" message as well.  Here were the commands
that I ran from a grub boot image under RHEL5.1 containing xen-3.0.3-41.el5 on
a VT enabled Intel processor:
grub> root (hd0,a)
grub> chainloader +1
grub> boot

I've also tried several other combinations for the "root" directive as well:
grub> root (hd0,0)
grub> root (hd0,0,a)

In addition, I've also added the "makeactive" and "rootnoverify" directives as
well using different combinations of the above options with the same results. 
If there's any additional information that you'd like to see, I'd be happy to
provide.

Thanks,
malbert

I'm seeing this with a CentOS-5.1 stock Xen version 3.1.0-53.1.6.el5 and
FreeNAS-i386-liveCD-0.686.1.2744.iso (FreeBSD-6.2 based).

Any workarounds?

(In reply to comment #14)
> I'm seeing this with a CentOS-5.1 stock Xen version 3.1.0-53.1.6.el5 and
> FreeNAS-i386-liveCD-0.686.1.2744.iso (FreeBSD-6.2 based).
> 
> Any workarounds?

I have the same problem with Fedora 8 Xen-kernel  
2.6.21.7-3.fc8xen #1 SMP Thu Mar 20 14:57:53 EDT 2008 i686 i686 i386 GNU/Linux
and xen-3.1.2-2.fc8.src.rpm. When trying to boot the latest FreeBSD
7.0-RELEASE-i386-bootonly.iso, FreeBSD boot is interrupted with register dump
and "BTX halted" error messages that loop forever (you have to pause the DOM-U
to see the content...).

A fix for this would be really nice.

This will work in xen-unstable, and also on 3.2 branch if Xen is built with
build option 'vmxassist=n'. It will not be fixed in older Xen branches.

(In reply to comment #16)
> This will work in xen-unstable, and also on 3.2 branch if Xen is built with
> build option 'vmxassist=n'. It will not be fixed in older Xen branches.
How to set up this option doing Xen 3.2 build on CentOS 5.1 ?
Thanks.
Boris
> 

(In reply to comment #17)

I reran 
# make world vmxassist=n
# make install
Created new ramdisk and rebooted.
Now waiting for FreeBSD 7.0 DVD delivery :-)
SNV87 PVM keeps running fine with new xen 3.2 kernel

> (In reply to comment #16)
> > This will work in xen-unstable, and also on 3.2 branch if Xen is built with
> > build option 'vmxassist=n'. It will not be fixed in older Xen branches.
> How to set up this option doing Xen 3.2 build on CentOS 5.1 ?
> Thanks.
> Boris
> > 
> 

Hi

At this time (June 2009) ..
Is there any way to make FBSD working on xen 3.1 HVM ?

I need to build a xen server on Netbsd Dom0. That means (according to netbsd
wiki) that I HAVE TO build Xen 3.1 HVM on i386, and not Xen 3.3 .. 

For the moment OpenBSD vm's work finely, but FreeBSD is "halted on BTX" ....

Sould I take a look to EsxI ? ;-)

Thks

I'm having this behaviour with a FreeBSD 7.0 based LiveCD (pfSense) using a XEN
3.2-1 HVM.

Is there any fix / workaround for this by now?