Building Darwin HOWTO

Rob Braun

Darwin Committer
Revision History
Revision 28 October 2001rjh -
Changed title to "Darwin Committer".
Revision 11 May 2001rb -


Darwin's build system builds new packages within a chroot'ed environment consisting entirely of previously built packages. For example, building a new version of xnu installs all the components needed for building xnu, into a chrooted environment, and then starts building xnu in that chrooted environment. Obviously, this leads to a paradox, since you'll need existing packages before you can build anything with this system. Luckily, I have prepared packages for the entire Darwin 1.2 distribution and have made them available at You do not need to install these packages. They are used by the build process in their packaged form.

Table of Contents

Building Individual Packages
Building a Package from a local directory
Building a Package from cvs
Building Everything
The darwin-buildall script
The Manifest File
Installing Packages
Adding a package to the Build System
Adding a GNU Autoconf package
Adding a 3rd Party Package


When building packages, the Darwin build system uses the Tools/buildtools from the Darwin cvs repository. You should check out the latest version and install it before attempting to use the build system. The current system uses a series of perl scripts around the dpkg format, that is why all the packages are .deb. The scripts used are darwin-buildpackage and darwin-buildall. These scripts use the perl modules Dpkg/Package/ and Dpkg/Package/ All these files are included in the buildtools that you should have checked out and installed already.

Building Individual Packages

To build an individual package, you will need buildtools from the cvs repository, and the latest packages needed for building the package you wish to build. That's a bit confusing, so just get the entire directory of packages.

There are two ways to build packages, you can build one from an existing source directory on your local disk, or you can build straight from the cvs repository. I'll cover both of these methods, starting with building from an existing source directory. For these examples it will be assumed that your existing packages are in /src/in and the directory where you wish the new packages to be deposited is /src/out. One important thing to note is that if a required package for building exists in the out directory, it will be used instead of the package in the in directory. This way, you can leave your in directory intact, and use your most recent packages for building.

When building the package, the default build location is /tmp/roots/packagename-version. This is where the required packages will be installed, and the new package will be built. If you wish to change the build location, set the BUILDIT_DIR environment variable to your desired build location. For (t)csh:

setenv BUILDIT_DIR /src/buildit

For Bourne shell:

export BUILDIT_DIR=/src/buildit

Building a Package from a local directory

For this example, it will be assumed that the source directory you're trying to build is /src/package. To build the package, execute the following as root:

darwin-buildpackage --dir /src/package /src/in /src/out

You have to execute the command as root, since the script will be creating a chrooted environment, installing the necessary tools, building the package, installing it into the chrooted environment, and then packaging it all up.

Building a Package from cvs

The first thing you'll have to do for this is to setup your anonymous cvs environment. You'll have to get your anonymous cvs login from Apple, set your CVSROOT environment variable, and login to the cvs server. There are instructions to do that on the Apple anoncvs site,

Once you've got your anonymous cvs account and environment setup, you can start building by running the following command as root:

darwin-buildpackage --cvs package-tag /src/in /src/out

The package-tag is the cvs tag for the project you want to build. For example, xnu-3.1.

Building Everything

You can build an entire distribution, or just a subset of it, all at once. This uses the darwin-buildall script, and uses a Manifest file to describe which packages to build. The buildall script only builds from the cvs repository with the cvs tagged versions specified in the Manifest file. The Manifest file can be checked out from the cvs tree, and is located in Darwin/Manifest. This section will first describe the use of the buildall script, and then discuss the format of the Manifest file.

The darwin-buildall script

Before running the buildall script, you should setup your anoncvs environment, as discussed earlier. Also, your build location (specified in the BUILDIT_DIR environment variable, defaulting to /tmp/roots) should be a UFS filesystem, otherwise some packages will not build correctly (because of HFS+'s case insensitivity). To run the buildall script execute the following as root:

darwin-buildall Manifest /src/in /src/out

This will check out source from the cvs tree, with the cvs tag specified in the Manifest file, and build it in your build location. The buildall script will progress through the Manifest file starting at the top, and continuing to the next item only after successfully checking out and building the current item.

The Manifest File

The Manifest file consistes of one package per line, in the format of:

src     project-version       target

src is where to get the package from. For the Darwin Manifest in the cvs tree, this is always cvs. The only other valid option is dir, and it will use the next item on the line as the directory to use as the source directory.

project-version is the project to build. If the src is cvs, then this is the cvs tag to check out. If the src is dir, then this is the directory to use for the package source.

target is almost always set to all.

The # character is a comment, so you can comment out different projects that won't build with your current packages.

Installing Packages

Although Darwin comes with the dpkg utilities, it has no dpkg database, so it is very difficult to use with Darwin. The easiest way to install .deb's under darwin is to do it by hand. Remember that a .deb is just a bunch of .tar.gz files ar'd together. So to extract the data from a .deb file you can do the following:

ar x file.deb

This will give you several files, but the important one is data.tar.gz. This is the actual data files from the .deb. Then you can cd / and extract the data.tar.gz file. You should use gnutar to extract the file, because of minor behavioral differences between the BSD tar and GNU tar.

Adding a package to the Build System

If you want to add a package to the build system, there are a couple of things needed. If the package is designed to be built with GNU make (most of them), the process is fairly straight forward. For this example, I'll take the example of xinetd. xinetd is setup to use the GNU autoconf configure script. Here's the series of steps I went through to get xinetd building under the Darwin Build System.

Adding a GNU Autoconf package

  • Download the package. The version I'm using is, so the package is xinetd-

  • Unpack the package. After untarring the package, I have the directory xinetd-

  • Create a directory above the package's directory. Create a directory xinetd and move xinetd- into the newly created xinetd directory and rename it xinetd as well:

    mkdir xinetd

    mv xinetd- xinetd/xinetd

  • Create a dpkg control file. Create the dpkg directory xinetd/dpkg, and create a control file. The control file should be similar to the following:

    Package: xinetd
    Maintainer: Synack Communications 
    Vendor: Synack Communications
    Build-Depends: build-base
    Description: xinetd is a replacement for inetd.

    The Build-Depends tag is a space separated list of packages required to build the package. build-base is a predefined tag that consists of the compiler and most libraries and utilities needed to build most packages.

  • Create a high level package Makefile. Create a Makefile, xinetd/Makefile, that looks like:

    Project               = xinetd
    UserType              = Administrator
    ToolType              = Commands
    Extra_Configure_Flags = --with-libwrap

    # It's a GNU Source project
    include $(MAKEFILEPATH)/CoreOS/ReleaseControl/GNUSource.make

    Install_Target = install

    The Extra_Configure_Flags is a list of flags to pass to configure.

  • Build the package. Run the command to build the system:

    sudo darwin-buildpackage --dir xinetd in out

    Now, assuming the package properly supports the use of the --srcdir flag to configure, the package will build correctly. Very odd errors can happen when trying to build a package under the Darwin Build System, because it does not build the package in the source directory. Instead it has a separate temporary directory for the intermediate object files.

Adding a 3rd Party Package

This section describes how to add a package to the Darwin build system that does not use the GNU Autoconf system to build the project. This section will be rather generic, since a "3rd Party Package" can do just about anything to build its self.

  • Get the package. For this example, I'll use mkisofs.

  • Put the package in a nested directory in mkisofs/mkisofs. This will let the wrapper Makefile and control files go in the top directory, and the actual unmodified source go in the lower directory.

  • Create a mkisofs/dpkg directory, and a control file within it. The control file should be similar to the one mentioned in the previous GNU Autoconf example.

  • Create the wrapper Makefile. The Makefile will look similar to this:

    # Project info

    # Project should be the project name, but also the name of the subdirectory
    # where the project actually lives.
    Project               = mkisofs
    UserType              = Administration
    ToolType              = Commands
    # Extra_Environment is a variable you can set with extra environment variables
    # while building the project
    Extra_Environment     =

    # It's a 3rd Party Source project, meaning it doesn't use autoconf or the
    # BSD style project building...
    include $(MAKEFILEPATH)/CoreOS/ReleaseControl/Common.make

    # Dunno, just copied over and it makes things happy.
    lazy_install_source:: shadow_source

    # Every project should have a 'build' target.  This is the target that
    # the build scripts use to compile the whole thing.
    build:: lazy_install_source
          @echo "Building $(Project)..."
          # Fill in necessary commands to build the project
          # Example make command:
          #$(_v) $(MAKE) -C $(BuildDirectory)/$(Project) $(Environment)

    # Every project should also have an 'install' target.  This is the target
    # the build scripts use to install the package, and wrap it up.
    install:: build
          @echo "Installing $(Project)..."
          # Fill in the necissary commands to install the project
          # Example make install command:
          #$(_v) $(MAKE) -C $(BuildDirectory)/$(Project) $(Environment) install

    This Makefile includes the "Common.make" file, which sets up a bunch of the make rules you'll need, but it does not setup the build or install targets. These are the two targets you'll need to implement in your Makefile for the build system to work correctly.

You will run into problems when trying to get the build system to work. There are a couple of things you'll need to remember. First, you cannot modify the source. Second, you're building in a directory separate from where the source lives. The latter will often cause problems because Makefile variables are setup expecting to be built in the source directory. If it is a simple matter of Makefile variables, you can override them with the Extra_Environment variable in the wrapper Makefile.


Copyright © 2001 by Rob Braun.

This material has been released under and is subject to the terms of the Common Documentation License, v.1.0, the terms of which are hereby incorporated by reference. Please obtain a copy of the License at and read it before using this material. Your use of this material signifies your agreement to the terms of the License.