Make Build System

Currently, NuttX supports both CMake and Make build systems. This guide explains the NuttX make-based build system.

Due to requirements, constraints, and the complexity of the build process, NuttX divides this work into multiple files, each handling specific parts of the build process.

As stated in The Inviolable Principles of NuttX, multiple platforms should be supported:

  • Win.mk: handles windows platform support.

  • Unix.mk: handles unix-like platforms support.

NuttX supports multiple build modes. See NuttX Protected Build:

  • FlatLibs.mk: Kernel and user-space built into a single blob.

  • ProtectedLibs.mk: Kernel and user-space built as two separate blobs.

  • KernelLibs.mk: Kernel built into single blob. User apps must be loaded into memory for execution.

NuttX targets multiple libs, or silos, each handling its own compilation:

Note

Gregory Nutt has a nice presentation about NuttX architecture

There the silo concept is explained. Only the silos there are listed below as libs. The build mode influences the needed libs.

$ ls -l staging/
drwxr-xr-x  2 xxx xxx    4096 Oct  6 16:02 .
drwxr-xr-x 27 xxx xxx    4096 Oct  6 16:02 ..
-rw-r--r--  1 xxx xxx  323640 Oct  6 16:02 libapps.a
-rw-r--r--  1 xxx xxx  384352 Oct  6 16:02 libarch.a
-rw-r--r--  1 xxx xxx   62182 Oct  6 16:02 libbinfmt.a
-rw-r--r--  1 xxx xxx    6468 Oct  6 16:01 libboards.a
-rw-r--r--  1 xxx xxx 2820054 Oct  6 16:02 libc.a
-rw-r--r--  1 xxx xxx  161486 Oct  6 16:01 libdrivers.a
-rw-r--r--  1 xxx xxx  981638 Oct  6 16:02 libfs.a
-rw-r--r--  1 xxx xxx  224446 Oct  6 16:02 libmm.a
-rw-r--r--  1 xxx xxx 2435746 Oct  6 16:01 libsched.a
-rw-r--r--  1 xxx xxx   51768 Oct  6 16:02 libxx.a

Verbosity

The V variable can be passed to make to control the build verbosity.

  • Quiet (Default): The build output is minimal.

  • Verbose (`V=1 V=2`): Shows the full compiler commands (enables command echo).

  • Verbose Tools (`V=2`): Enables verbose output for tools and scripts.

# V=1,2: Enable echo of commands
$ make V=1

# V=2:   Enable bug/verbose options in tools and scripts
$ make V=2

Build Process

Note

Keep the configuration step and board folder layout short. Separate docs should be created.

The Make based build process starts with the NuttX tree configuration. This is done by running tools/configure.sh script. The configuration step, prepares the NuttX kernel tree, setting the board specific arch, chip and board files.

The Make build system refers the needed subsystems using generic naming:

  • The current architecture is refered as arch

  • The current chip is refered as chip

  • The current board is refered as board

These generic names are mapped to the actual names by symlinks:

  • The current chip directory gets symlinked to nutt/include/arch/chip.

  • The current board directory gets symlinked to nutt/include/arch/board.

  • The current arch directory gets symlinked to nutt/include/arch.

The board config is stored as defconfig file, which is a minimal config, storing only the configs that differs from default config values. Due to NuttX’s particularity of strict dependance to the app directory, the .config file is not generated by either kconfiglib or kconfig-frontends, but rather by the an in-tree tools/process_config.sh script. This script takes a “base” input file (the boards defconfig file), additional include paths (the most relevant being the apps top directory), and generate an output file (the $(TOPDIR)/.config file).

# part of configure.sh shell script, starting at line 240
#
# src_config=${configpath}/defconfig
# dest_config="${TOPDIR}/.config"
# original_config="${TOPDIR}/.config.orig"
# backup_config="${TOPDIR}/defconfig"

$ ln -sf ${src_makedefs} ${dest_makedefs} || \
  { echo "Failed to symlink ${src_makedefs}" ; exit 8 ; }
$ ${TOPDIR}/tools/process_config.sh -I ${configpath}/../../common/configs \
  -I ${configpath}/../common \
  -I ${configpath} -I ${TOPDIR}/../apps \
  -I ${TOPDIR}/../nuttx-apps \
  -o ${dest_config} ${src_config}

Starting the Build

The root Makefile is the build process entrypoint. Its main job is to check for a .config file and include the appropriate host-specific Makefile. The “actual” first steps of the build process are handled by the host-specific Makefile (either Win.mk or Unix.mk).

Early on, during parsing, both host-specific Makefile will also include

Built-in dependency mechanism

Note

The documentation for mkdeps.c, cnvwindeps.c, mkwindeps.sh, and mknulldeps.sh should be extended. Lots of details are missing. Part of the documentation here should be moved.

NuttX implements a built-in dependency mechanism. See mkdeps.c, cnvwindeps.c, mkwindeps.sh, and mknulldeps.sh. This mechanism uses gcc to generate a *.d like dependency files that can be included by the Makefile to track file dependencies. This mechanism is kick-started by the depend target, and targets specific directories in the NuttX tree based on the build mode. It is required by both pass1dep and pass2dep.

pass1dep: context tools/mkdeps$(HOSTEXEEXT) tools/cnvwindeps$(HOSTEXEEXT)
      $(Q) for dir in $(USERDEPDIRS) ; do \
        $(MAKE) -C $$dir depend || exit; \
      done
pass2dep: context tools/mkdeps$(HOSTEXEEXT) tools/cnvwindeps$(HOSTEXEEXT)
      $(Q) for dir in $(KERNDEPDIRS) ; do \
        $(MAKE) -C $$dir EXTRAFLAGS="$(KDEFINE) $(EXTRAFLAGS)" depend || exit; \
      done

Both pass1dep and pass2dep sets different directories and orders in which the depend target is ran. See Directories.mk.

Building NuttX libs

The host-specific Makefile will not build the required NuttX libs. It will defer work by “recursively” calling make in each of the directories listed in the $(USERLIBS) and $(NUTTXLIBS) variables.

pass1 & pass2

The NuttX binary is always generated by running both pass1 and pass2 targets. The actual dependencies of the mentioned targets may vary depending on the build mode.

Different NuttX build modes will not influence the “execution” of the pass1 and pass2 targets, but rather will influence the dependencies pulled by those targets.

  • pass1 target depends on the $(USERLIBS).

  • pass2 target depends on the $(NUTTXLIBS).

all: pass1 pass2

pass1: $(USERLIBS)
pass2: $(NUTTXLIBS)

The content of the $(USERLIBS) and $(NUTTXLIBS) variables is defined in each build mode makefile. See Build Modes above.

Staging the libs

After compiling libraries defined by the $(USERLIBS) and $(NUTTXLIBS) at the previous step, the make build system will install them at a special staging/ directory, residing at the root of the NuttX tree.

These libraries are passed to the the final target rule ($(BIN)).

# Unix.mk : line 536
$(BIN): pass1 pass2
# ...
      $(Q) $(MAKE) -C $(ARCH_SRC) EXTRA_OBJS="$(EXTRA_OBJS)" LINKLIBS="$(LINKLIBS)" APPDIR="$(APPDIR)" EXTRAFLAGS="$(KDEFINE) $(EXTRAFLAGS)" $(BIN)
# ...

Win.mk

Although targeting different platforms, both Win.mk and Unix.mk aim to produce the same output. The need for independent files is due to the differences in the platform’s approaches.

Forward vs Back slashes

One of the main differences is the use of forward slashes (/) on unix-like platforms versus backslashes (\) on windows

${HOSTEXEEXT} ${HOSTDYNEXT}

These variables are used by the build system to configure the executable suffix required by the used platform. They are defined in Config.mk.

For windows platform:

  • ${HOSTEXEEXT} is set to .exe.

  • ${HOSTDYNEXT} is set to .dll.

Symbolic Linking

For the windows platform, the build system handles symbolic links differently.

Unix.mk

Versioning

The build system will impact versioning if NuttX is cloned as a repo. See Versioning.

config.h, .config, mkconfig

NuttX’s build system defers the config.h generation to a separate tool called mkconfig. See Makefile.host.

tools/mkconfig$(HOSTEXEEXT): prebuild
          $(Q) $(MAKE) -C tools -f Makefile.host mkconfig$(HOSTEXEEXT)

The include/nuttx/config.h recipe calls the mkconfig executable generated by the rule above to create the config.h file from the current .config file.

Dummies

The main reason for the use of dummies is to handle some specific scenarios, such as external code bases, custom chips and boards or to overcome tooling limitations. If any of the features below are not used, the build system will fallback to a dummy.

  • ${EXTERNALDIR}

    Possible values for $(EXTERNALDIR) are external or dummy.

    NuttX code base can be extended by using $(TOPDIR)/external/ directory. The build system searches for a Kconfig file in that directory. If found, the build system defines the EXTERNALDIR variable to external and also appends another lib (libexternal) to the build process.

    # External code support
# If external/ contains a Kconfig, we define the EXTERNALDIR variable to 'external'
# so that main Kconfig can find it. Otherwise, we redirect it to a dummy Kconfig
# This is due to kconfig inability to do conditional inclusion.

EXTERNALDIR := $(shell if [ -r $(TOPDIR)/external/Kconfig ]; then echo 'external'; else echo 'dummy'; fi)

  • dummy/Kconfig

    The dummy/Kconfig is used to handle custom chips and boards.

    If in-tree chip/board is used, the build system will resolve to dummy_kconfig files. - $(CHIP_KCONFIG) is set to $(TOPDIR)$(DELIM)arch$(DELIM)dummy$(DELIM)dummy_kconfig - $(BOARD_KCONFIG) is set to $(TOPDIR)$(DELIM)boards$(DELIM)dummy$(DELIM)dummy_kconfig

    If custom chip/board is used, the build system will resolve to their custom paths.

    # Copy $(CHIP_KCONFIG) to arch/dummy/Kconfig

arch/dummy/Kconfig:
    @echo "CP: $@ to $(CHIP_KCONFIG)"
    $(Q) cp -f $(CHIP_KCONFIG) $@

# Copy $(BOARD_KCONFIG) to boards/dummy/Kconfig

boards/dummy/Kconfig:
    @echo "CP: $@ to $(BOARD_KCONFIG)"
    $(Q) cp -f $(BOARD_KCONFIG) $@

  • boards/dummy.c

    A special boards/dummy.c file is used by the build system to generate a useless object. The purpose of the useless object is to assure that libboards.a/lib is created. Some archivers (ZDS-II, SDCC) require a non-empty library or they will generate errors.

Build Modes

As specified above, NuttX supports multiple build modes. The build mode is selected based on specific Kconfig options.

# Library build selections
#
# NUTTXLIBS is the list of NuttX libraries that is passed to the
#   processor-specific Makefile to build the final NuttX target.
# USERLIBS is the list of libraries used to build the final user-space
#   application
# EXPORTLIBS is the list of libraries that should be exported by
#   'make export' is

ifeq ($(CONFIG_BUILD_PROTECTED),y)
include tools/ProtectedLibs.mk
else ifeq ($(CONFIG_BUILD_KERNEL),y)
include tools/KernelLibs.mk
else
include tools/FlatLibs.mk
endif

The content of each file referenced above is documented in its own section.

Config.mk

Config.mk contains common definitions used by many configuration files.

  • It defines the logic behind the Verbosity

  • It resolves the platform specific particularities, such as

    • build tools e.g. mkdeps, cnvwindeps, link.sh/.bat, unlink.sh/.bat

    • file extensions e.g. .exe, .dll for windows, .o, .a for unix

    • delim based on the host platform. e.g. \ for windows, / for unix

  • Resolve custom bard/chip/arch

  • Defines the rules for the dependency mechanism

    OBJPATH ?= .

%.dds: %.S
    $(Q) $(MKDEP) --obj-path $(OBJPATH) --obj-suffix $(OBJEXT) $(DEPPATH) "$(CC)" -- $(CFLAGS) -- $< > $@

%.ddc: %.c
    $(Q) $(MKDEP) --obj-path $(OBJPATH) --obj-suffix $(OBJEXT) $(DEPPATH) "$(CC)" -- $(CFLAGS) -- $< > $@

%.ddp: %.cpp
    $(Q) $(MKDEP) --obj-path $(OBJPATH) --obj-suffix $(OBJEXT) $(DEPPATH) "$(CXX)" -- $(CXXFLAGS) -- $< > $@

%.ddx: %.cxx
    $(Q) $(MKDEP) --obj-path $(OBJPATH) --obj-suffix $(OBJEXT) $(DEPPATH) "$(CXX)" -- $(CXXFLAGS) -- $< > $@

%.ddh: %.c
    $(Q) $(MKDEP) --obj-path $(OBJPATH) --obj-suffix $(OBJEXT) $(DEPPATH) "$(CC)" -- $(HOSTCFLAGS) -- $< > $@

  • Defines the include flag prefix based on compiler

  • Defines the common functions used to compile, assemble, archive files, etc.

This file (along with <nuttx>/.config) must be included at the top of each configuration-specific Make.defs file like

include $(TOPDIR)/.config
include $(TOPDIR)/tools/Config.mk

Subsequent logic within the configuration-specific Make.defs file may then override these default definitions as necessary.

FlatLibs.mk

This file defines the library sets for the flat build mode. In this mode, the NuttX kernel and all user-space applications are compiled and linked into a single, monolithic binary.

Its primary responsibilities are:

  • Populating ``NUTTXLIBS``: The file systematically appends all required libraries to the NUTTXLIBS variable. This includes core OS libraries (libsched, libmm, libc), architecture and board support libraries, and device drivers.

  • Conditional Library Inclusion: It uses ifeq ($(CONFIG_XXX),y) checks to conditionally include libraries for optional features based on the system’s Kconfig. For example, staging/libnet.a is added only if CONFIG_NET=y, and staging/libcrypto.a is added if CONFIG_CRYPTO=y.

  • Defining ``USERLIBS``: In the flat build, there is no separate user-space binary, so the USERLIBS variable is initialized but remains empty.

  • Setting ``EXPORTLIBS``: It assigns the complete list of NUTTXLIBS to the EXPORTLIBS variable. This ensures that the make export command packages all the compiled libraries required to link the final binary.

ProtectedLibs.mk

This file defines the library sets for the protected build mode. In this mode, the NuttX kernel and user-space applications are built as two distinct binaries, enabling memory protection and a more robust system architecture.

Its primary responsibilities are:

  • Separate Library Lists: Unlike the flat build, this file populates both the NUTTXLIBS variable (for the kernel binary) and the USERLIBS variable (for user-space applications).

  • Kernel vs. User Library Variants: It explicitly differentiates between kernel and user variants of core libraries. For instance:

    • Kernel-specific libraries (e.g., libkc.a, libkmm.a, libkarch.a) are added to NUTTXLIBS.

    • User-space counterparts (e.g., libc.a, libmm.a, libarch.a) are added to USERLIBS.

  • System Call Mechanism: It includes essential components for the system call interface:

    • staging/libstubs.a (system call stubs) is added to NUTTXLIBS.

    • staging/libproxies.a (system call proxies) is added to USERLIBS. These facilitate safe communication between user applications and the kernel.

  • Conditional Library Inclusion: Similar to other build modes, it uses ifeq ($(CONFIG_XXX),y) checks to conditionally include libraries for optional features in both kernel and user-space, based on the Kconfig settings.

  • Exporting User Libraries: A key distinction is that EXPORTLIBS is set to $(USERLIBS). This means that the make export command will primarily package the user-space libraries, which are then used by external build systems or for linking user applications.

KernelLibs.mk

This file defines the library sets for the kernel build mode. In this mode, only the NuttX kernel is compiled and linked into a single binary. User applications are expected to be loaded and executed separately, typically in a protected user-space environment.

Its primary responsibilities are:

  • Kernel-Only ``NUTTXLIBS``: The file populates the NUTTXLIBS variable with all the necessary kernel-space libraries. This includes core OS components (e.g., libsched, libdrivers), kernel variants of common libraries (e.g., libkc.a, libkmm.a, libkarch.a), and board support.

  • Empty ``USERLIBS``: The USERLIBS variable is explicitly initialized but remains empty, as this build mode does not produce a user-space binary.

  • System Call Stubs: It includes staging/libstubs.a in NUTTXLIBS, providing the kernel-side implementation for system calls. User-space system call proxies (libproxies.a) are not included, consistent with the absence of a user-space build.

  • Conditional Library Inclusion: It uses ifeq ($(CONFIG_XXX),y) checks to conditionally include kernel-specific libraries for optional features based on the system’s Kconfig.

  • Exporting User Libraries (Empty): EXPORTLIBS is set to $(USERLIBS). Consequently, in this kernel-only build mode, EXPORTLIBS will be empty, reflecting that no user-space libraries are produced for external packaging.

Directories.mk

This file defines the directories used throughout the NuttX build process. These directory lists are not static but are dynamically constructed based on the active Kconfig configuration.

The file begins by defining a BASEDIRS variable, which includes core directories that are always part of the build. Subsequently, it uses conditional logic (ifeq ($(CONFIG_XXX),y)) to append additional directories to various master lists, such as ALLDIRS and CLEANDIRS, only if their corresponding Kconfig options are enabled.

For example:

  • The net/ directory is added to the build lists only if CONFIG_NET=y.

  • The graphics/ directory is included if CONFIG_GRAPHICS=y.

  • The crypto/ directory is included if CONFIG_CRYPTO=y.

The file defines key variables that hold these dynamically generated directory lists:

  • KERNDEPDIRS: Directories containing kernel files for dependency generation.

  • USERDEPDIRS: Directories containing user-space files for dependency generation.

  • CCLEANDIRS: Directories where the clean_context target will execute.

  • CLEANDIRS: All known directories where the clean target will execute.

  • CONTEXTDIRS: Directories with special pre-build requirements, such as auto-generation of files or symbolic link creation.

LibTargets.mk

The LibTargets.mk file defines all the targets needed to build the NuttX libraries from their respective source directories and then install the resulting library file into the staging/ directory.

For many core libraries (e.g., libc, libm, libmm, libbuiltin, libnx, libarch), LibTargets.mk defines distinct targets for kernel and user variants.

  • Kernel libraries are typically prefixed with libk (e.g., libkc.a for kernel C library, libkmm.a for kernel memory management). These are compiled with specific EXTRAFLAGS that include $(KDEFINE), which sets kernel-specific preprocessor definitions (e.g., -D__KERNEL__). This ensures they are built with the necessary context and APIs for the kernel environment.

  • User libraries retain their standard names (e.g., libc.a, libmm.a). These are compiled without the $(KDEFINE) flag, making them suitable for user-space applications. The build process for user-mode libraries often depends on pass1dep, while kernel-mode libraries depend on pass2dep, reflecting the two-pass build strategy in protected and kernel build modes.

For each variant of a library, two main targets are defined:

  1. A build target: This target recursively calls make within the library’s source directory (e.g., libs/libc/, mm/) to compile the sources and create the library archive (.a) file. It passes the appropriate EXTRAFLAGS depending on whether it’s a kernel-mode or user-mode build. The dependency for these targets is either pass1dep or pass2dep, ensuring dependencies are checked before building.

  2. A staging target: This target depends on the successful creation of the library archive from the previous step. Its recipe calls the INSTALL_LIB macro (defined in Unix.mk or Win.mk) to copy the newly created library to the top-level staging/ directory.

The file contains conditional logic, primarily based on CONFIG_BUILD_FLAT, to adjust dependencies (pass1dep vs. pass2dep) for libraries that can be part of either the user or kernel space, ensuring they are built in the correct pass according to the selected build mode.