> bhyve Zones

To run zones using the bhyve hypervisor, first ensure that the bhyve overlay is installed

  zap install-overlay bhyve

To boot Tribblix (or other illumos distributions) as a bhyve guest, you will need to use a fairly recent version of the distribution. For Tribblix specifically you'll need to use m24.1 or later. The command would be something like:

  zap create-zone -t bhyve -z bhyve1 \
    -x 192.168.0.236  \
    -I /var/tmp/tribblix-0m41.iso \
    -V 8G

Here, the -t flag tells us that this is a bhyve zone, the -z flag gives the name of the zone, the -x flag specifies the zone's permitted IP address, the -I flag tells bhyve where to find the ISO image to boot from, and the -V flag says to create an 8G virtual disk for the guest to use.

You can set the number of vcpus for the guest with the -c flag (the default is a single cpu). Likewise, the guest memory can be set with the -m flag (the default is 1G of memory). As an example, add -c 2 -m 2G.

That will create and boot the zone. To interact with it, use VNC. As root, start up socat like so:

  socat TCP-LISTEN:5905,reuseaddr,fork UNIX-CONNECT:/export/zones/bhyve1/root/tmp/vm.vnc

Note the name of the path to the zone, which you'll need to change depending on the name of the zone. Also, you can choose a different port, just remember that it must be unique, and the VNC display number is offset by 5900.

Then, as yourself, you can connect with

  vncviewer :5

and you should see, and be able to interact with, the system as it boots.

There's a slightly more convenient way to view the graphical console without the manual mucking about with socat and vncviewer described above. If you share your user account with the zone using the -U flag (which in the case of bhyve zones means sharing with bhyve and not with the OS running as the guest):

  zap create-zone -t bhyve -z bhyve1 -U ptribble ...

then the startvnc command can be used to connect to the VNC console as that user, like so:

  startvnc -Z bhyve1

This trick actually moves the socket into a dedicated location, which will be /var/zap/bhyve/bhyve1/vm.vnc in the global zone, but only accessible to the user you shared the zone with. If you want to use socat you'll have to change the filename. If you're running the bhyve zone on a remote server, you can tunnel the VNC socket with ssh, like so:

  ssh -L 5905:/var/zap/bhyve/bhyve1/vm.vnc my.server.name

and then on the desktop machine point your VNC client at :5.

You can run the installer as normal. But before you can use the installed system you need to remove the virtual CD and reboot.

  zap remove-cd -z bhyve1 -r

At the present time, you're unable to run X in Tribblix under bhyve. (Although you could run an X session under VNC.)

Sharing data

In other zone types, you can share directories using the -S and -s flags (for writable and read-only shares respectively).

The same works with bhyve, but the bhyve guest will use virtfs. In a Linux guest you can mount these as follows: 

  mount -t 9p -o trans=virtio,uname=root bhyvefs0 /mnt/foo

The name bhyvefs# is fixed and, if you have multiple shares, will start from 0 and increment, in the order given by the -S and -s flags.

Cloud-init

Often, you'll want the guest to automatically configure itself. You can use cloud-init for this.

Adding the -C flag to the create-zone invocation will enable cloud-init. On its own, this doesn't do a great deal, but should set up the IP address and dns resolution correctly.

Using the -k flag with the name of an SSH public key file will enable cloud-init and configure the guest to allow SSH access to the default account (or root) using the given SSH key. For example, you might use:

  zap create-zone -t bhyve -z bhyve2 \
    -x 192.168.0.237  \
    -I /var/tmp/ubuntu-22.04.1-live-server-amd64.iso \
    -k /export/home/ptribble/.ssh/id_rsa.pub \
    -V 8G

Using raw images

The startup above assumed that you had an installation ISO image. Some projects make raw images, often known as cloud images, available. The process here is slightly different.

First, you need a raw image. If you have a vmdk image, then it can be converted, for example:

  qemu-img convert -O raw omnios-r151058.cloud.vmdk omnios-r151058.img

Then you need to create a zfs volume, making sure it's large enough to take the whole image. You can get the size from ls -lh, which shows the length of the file, not the amount of space it takes up.

  zfs create -V 8G rpool/client1

For most such images, you need to enable cloud-init with an ssh key, otherwise there will be no way to log in. The easiest way to do this is pass the ssh public key you want to use, and zap will fill in the rest. Then create a bhyve zone using the zfs volume you just populated:

  zap create-zone -t bhyve -z client1 \
    -x 192.168.1.236  \
    -v rpool/client1 \
    -k /export/home/ptribble/.ssh/id_rsa.pub

and after a few seconds it should boot up, pick up its name and address, and you can ssh in as root (or whatever the standard username is).

Some notes:

Worked Example: FreeBSD

The following is a quick example of how I set up a bhyve zone running FreeBSD. I downloaded the disc1 iso image, and then started bhyve like so in order to access the installer:

zap create-zone -z freebsd2 -t bhyve \
-I /var/tmp/FreeBSD-15.1-RELEASE-amd64-disc1.iso \
-U ptribble \
-x 192.168.1.215 \
-V 16G -m 4G -c 2

That allocates 4G of RAM and 2 cpus to the guest, and provides a 16G disk which is enough to install a few packages and experiment.

Because I used the -U flag to share my account and give it administration rights, I can directly connect to the zone's console with VNC:

startvnc -Z freebsd2

The installer is reasonably straightforward. I went for the basic packaged install on ZFS, didn't do anything fancy. For networking, set up IPv4 manually with the address specified, and without IPv6 (because the zone is configured for the given IPv4 address and doesn't have any IPv6 enabled).

I give zones the same hostname as the zone name, otherwise it gets a bit confusing. I also created my account with the same userid and username as on my main system, setting a temporary password so I can get in the first time to copy up my ssh authorized_keys file, and when it asks about inviting to a group, answer with "wheel" which will put you into the wheel group, necessary so you can use su.

The first thing once it's up is to set up my account. All I do is copy my ssh key, shell startup, and my personal bin directory which has a set of useful helper scripts. I use the zssh helper from zap which can work out how to ssh to a zone without needing to know the IP address.

tar cf - .ssh/authorized_keys .tcshrc bin \
| zap zssh freebsd2 tar xf -

That will ask you for the temporary password, but after that you'll get passwordless authentication via your ssh key.

Then, in order to have a look around as a desktop (the 4G I allocated is plenty to run a decent desktop and browse the web), I installed some software (after su):

pkg install xfce
pkg install tigervnc-server
pkg install xorg
pkg install firefox

I can then run a VNC session on the FreeBSD system:

vncserver -geometry 1600x1200

This needs a .vnc/xstartup file. I copied the one from /usr/share/tribblix-session/xfce.xstartup on my Tribblix host, changing /usr/bin to /usr/local/bin as applications end up in different places on FreeBSD.

If you set the vnc password in the guest to be the same as the host, you can then connect to it with a simple:

startvnc -z freebsd2:1

Index | |