qjail-intro(8) FreeBSD System Manager’s Manual qjail-intro(8)

NAME

qjail-intro — Introduction to chroot directory tree, jails, and qjail.

DESCRIPTION

Qjail [ q = quick ] is a 4th generation wrapper for the basic chroot jail system that includes security and performance enhancements. Plus a new level of "user friendliness" enhancements dealing with deploying large jail environments, 100’s of jails. Qjail requires no knowledge of the jail command usage.

OVERVIEW

The original FreeBSD developers felt the need for a method to restrict a processes access to the host system resources so if it becomes compromised the host system is protected from also being compromised. They achieved this goal with the "chroot" command which was in the original 4.4BSD system, from which the current FreeBSD RELEASE is a direct descendant. This first generation "chroot" environment, made it look like the named directory was the "root" IE starting point; of a system directory tree. Just like "/" is to FreeBSD. In this basic incarnation, the directory tree would just have the binaries necessary to form a environment for a single application such as apache web server. You could have multiple such "chroot" environments. They all shared the hosts network and disk space. This trait continues into today’s jail systems. As you can imagine, occupants of these basic "chroot’s" influenced users to stay at the RELEASE they were at because of the size of the task to redevelop them under a new RELEASE mixture of binaries. Jail deployments greater than two were uncommon.

The jail utility appeared in FreeBSD 4.0. With this second generation "chroot" enhancement came the renaming of a "chroot" environment to a "jail’, the ability to assign IP address to a jail, auto starting jails at boot time, and a general shift in thought about the occupant of the jail. The customized streamline apache web server jail that had no way too be easily configured, progressed into a complete clone of the operating system with all the customizing options one is familiar with on the host. The major shortcoming of this type jail system is each jail has its own copy of the running system binaries. FreeBSD reserves a limited number of control structures for storing files and directories, called inodes. Creating a few jails consumes many of those valuable inodes, eventually preventing the creation of new jails. Worse yet is each jail loads its own copy of the running binaries into memory, which causes thrashing on the swap device as memory pages are swapped in and out as the limited memory is shared between the host and the jails. Besides consuming resources and creating performance degradation, this also causes a major administration headache when wanting to update the host running system, because the host and the jails have to be running the same version of the binaries. Jail deployments greater that four were uncommon.

Then about RELEASE 5.4 the creative use of the nullfs command added the ability for jails to share a single set of the running binaries between all the jails. This third generation solution solved the performance problems of the second generation, but had its own problems. Setting up a nullfs running binaries environment to support multiple jails was a undocumented manual one. Plus a second type of jail became available called an "image". The image jail introduced the ability to predefine the amount of disk space a jail could consume. This was accomplished with the mdconfig command, which mounts a flat file as a directory tree. Jail deployments greater than 10 were uncommon. The administration of this jail system type became increasing difficult with each newly added jail.

During FreeBSD RELEASE 8.0, "qjail" was introduced which is a wrapper that camouflages the underlying "jail" commands and automates those manual setup steps into a single command. Mounting a directory tree containing the running binaries as read-only files using "nullfs" became the method forming the basic design of the "qjail" jail system. The functions necessary to manage jails were condensed into the following commands, "install" for installing an pristine copy of the RELEASE version of FreeBSD, the "create" command to create both directory tree type jails and sparse image type jails. This includes the ability to assign IP address with their network device name, so aliases are auto created on jail start and auto removed on jail stop. An archived seed jail pre-configured with ports can be used as the template to form new jails. To make the deployment of many jails with the same configuration, jails can be auto duplicated while at the same time incrementing the last octal of the IP address. The archive, restore and delete commands are commonly understood functions. The "update" command for using the portsnap command to populate a complete ports tree, and the ability to copy the host’s running binaries after a host RELEASE upgrade. A "list" command to display the qjail jail status. The "config" command can flag a jail as "norun" to exclude it from being auto started at boot time. The "norun" status can be toggled back and forth on a single jail or all jails at once. Jails can be renamed and their IP address changed.

Qjail deploys two different jail types. The first type is based on a Directory tree. This type has unlimited disk space growth potential,it shares the host’s disk space. The jail will never run out of space until the host does. The second type is based on a sparse image file. A sparse file is one that occupies only the sum size of its contents, not its allocation size. IE; a sparse file allocated size of 5M, but only having 7 files, each 1k in size, only occupies 7k of physical disk space. As content is added, additional physical disk space is occupied up to the 5M allocation ceiling. The sparse file is mounted as a memory disk using the mdconfig command and populated with the directory tree content of a jail. This configuration is called a sparse image jail. It’s major benefits is it provides a way to put a hard limit on the maximum amount of disk space a jail can consume. This provides an addition level of protection to the host from intentional or unintentional run-a-way processes inside of a jail consuming disk space until the host system dies.

But by far "qjail" greatest achievement to the advancement of jailed systems, is the addition of "user-friendliness" that simplifies the management of large deployments of hundreds of jails. This enhancement adds the ability to designate a portion of the jail name as a group prefix so the command being executed will apply to only those jail names matching that prefix. A simple jail naming convention allows the grouping of like function jails together. The other advancement is the ability to create different "zones" consisting of identical jail systems each with their own groups of jails.

Qjail reduces the complexities of large jail deployments to the novice level. Qjail has a fully documented manpage, which is a rarity in the FreeBSD world. Details are given to facilitate the use of qjail’s capabilities to the fullest extent possible.

QJAIL SYSTEM

The qjail system is comprised of two components, qjail and qjail-bootime.

qjail is the main workhorse utility. It’s located at /usr/local/bin/qjail. It can install the qjail environment, create new jails, archive, restore, delete and update jails, open a jail console, and list the status of all the jails. See qjail(8) for complete usage details.

qjail2-bootime script is located at /usr/local/etc/rc.d/. It’s main purpose is to start all jails at boot time and stop all jails when the shutdown command is executed on the host. Adding qjail_enable="YES" to /etc/rc.conf will activate it.

SEE ALSO

qjail(8), qjail-howto(8), qjail-vnet-howto(8), qjail-ipv6-testing(8)
jail(8), mount_nullfs(8), mdconfig(8)

AUTHOR

Joe Barbish

FreeBSD 11.0 February 15, 2017 FreeBSD 11.0