The latest development version of this page may be more current than this released 1.14.0 version.

Multiple Repository Management

This page introduces basic concepts related to West and its multiple repository management features, and gives an overview of the associated commands. See History and Motivation and Zephyr issue #6770 for additional discussion, rationale, and motivation.

Note

West’s multi-repo commands are meant to augment Git in minor ways for multi-repo work, not to replace it. For tasks that only operate on one repository, just use plain Git commands.

Introduction

West’s built-in commands allow you to work with projects composed of multiple Git repositories installed under a common parent directory, which we call a west installation. This works similarly to Git Submodules and Google’s repo.

A west installation is the result of running the west init command, which is described in more detail below. This command can either create a new installation, or convert a standalone “mono-repo” zephyr repository into a full west installation. For upstream Zephyr, the installation looks like this:

zephyrproject
├── .west
│   ├── config
│   └── west
├── zephyr
│   ├── west.yml
│   └── [... other files ...]
├── modules
│   └── lib
│       └── tinycbor
├── net-tools
└── [ ... other projects ...]

Above, zephyrproject is the name of the west installation’s root directory. This name is just an example – it could be anything, like z, my-zephyr-installation, etc. The file .west/config is the installation’s local configuration file. The directory .west/west is a clone of the west repository itself; more details on why that is currently needed are given in the next section.

Every west installation contains exactly one manifest repository, which is a Git repository containing a file named west.yml, which is the west manifest. The location of the manifest repository is given by the manifest.path configuration option in the local configuration file. The manifest file, along with west’s configuration files, controls the installation’s behavior. For upstream Zephyr, zephyr is the manifest repository, but you can configure west to use any Git repository in the installation as the manifest repository. The only requirement is that it contains a valid manifest file. See West Manifests for more details on what this means.

Both of the tinycbor and net-tools directories are projects managed by west, and configured in the manifest file. A west installation can contain arbitrarily many projects. As shown above, projects can be located anywhere in the installation. They don’t have to be subdirectories of the manifest directory, and they can be inside of arbitrary subdirectories inside the installation’s root directory. By default, the Zephyr build system uses west to get the locations of all the projects in the installation, so any code they contain can be used by applications. This behavior can be overridden using the ZEPHYR_MODULES CMake variable; see cmake/zephyr_module.cmake for details.

Finally, any repository managed by a west installation can contain extension commands, which are extra west commands provided by that project. This includes the manifest repository and any project repository.

West Structure

West is currently split in two:

  • Bootstrapper: Installed by pip3 install west, which provides the west binary and the west init command.
  • Per-installation clone: this is the west repository cloned into each installation, which provides the built-in commands.

Note

This “bootstrapper” / “everything else” separation is similar to the model used by Google’s repo tool, but unfortunately in retrospect was not a good strategy for west.

In future versions, the west binary and all built-in commands (including init) will be installed by pip3 install west. Besides eliminating complexity, this will also make it possible to use West’s APIs from any Python file, not just extension commands.

Updating west will still be possible manually, e.g. with pip3 install --upgrade west. If necessary, it will also still be possible to use different versions of west on the same computer through Python virtual environments.

Bootstrapper

The bootstrapper module is distributed using PyPI and installed using pip3. A launcher named west is placed by pip3 in the user’s PATH. This the only entry point to west. It implements a single command: west init. This command needs to be run first to use the rest of functionality included in west, by creating a west installation. The command west init does the following:

  • Clones west itself in a .west/west folder in the installation.
  • Clones the manifest repository in the folder specified by the manifest file’s self.path section.
  • Creates an initial local configuration file.

Once west init has been run, the bootstrapper will delegate the handling of any west commands other than init to the cloned west repository. This means that there is a single bootstrapper instance installed at any time (unless you use virtual environments), which can then be used to initialize as many installations as needed, each of which can have a different version of west.

Per-Installation Clone

A west installation, as described above, contains a clone of the west repository in .west/west. This is where the built-in command implementations are currently provided. The rest of West’s APIs are also currently provided to extension commands by this repository. So that west can update itself, the built-in west update and west selfupdate commands fetch and update the .west/west repository.

The manifest-rev branch

West creates a branch named manifest-rev in each project, pointing to the commit the project’s revision resolves to. The branch is updated whenever project data is fetched by west update. Other multi-repo commands also use manifest-rev as a reference for the upstream revision as of the most recent update. See Multi-Repo Commands, below, for more information.

manifest-rev is a normal Git branch, but if you delete or otherwise modify it, west will recreate and/or reset it as if with git reset --hard on the next update (though git update-ref is used internally). For this reason, it is normally a bad idea to modify it yourself. manifest-rev was added to allow SHAs as project revisions in the manifest, and to give a consistent reference for the current upstream revision regardless of how the manifest changes over time.

Note

West does not create a manifest-rev branch in the manifest repository, since west does not manage the manifest repository’s branches or revisions.

Multi-Repo Commands

This section gives a quick overview of the multi-repo commands, split up by functionality. Some commands loosely mimic the corresponding Git command, but in a multi-repo context (e.g. west diff shows local changes on all repositories).

Project arguments can be the names of projects in the manifest, or their paths within the installation. Passing no project arguments to commands that accept a list of projects usually means to use all projects in the manifest.

Note

For detailed help, see each command’s --help output (e.g. west diff --help).

Main Commands

The west init and west update multi-repo commands are the most important to understand.

  • west init [-l] [-m URL] [--mr REVISION] [PATH]: create a west installation in directory PATH (i.e. .west etc. will be created there). If the PATH argument is not given, the current working directory is used. This command does not clone any of the projects in the manifest; that is done the next time west update is run.

    This command can be invoked in two ways:

    1. If you already have a local clone of the zephyr repository and want to create a west installation around it, you can use the -l switch to pass its path to west, as in: west init -l path/to/zephyr.
    2. Otherwise, omit -l to create a new installation from a remote manifest repository. You can give the manifest URL using the -m switch, and its revision using --mr. For example, invoking west with: west init -m https://github.com/zephyrproject-rtos/zephyr --mr v1.15.0 would clone the upstream official zephyr repository at the tagged release v1.15.0 (-m defaults to https://github.com/zephyrproject-rtos/zephyr, and --mr defaults to master).

  • west update [--rebase] [--keep-descendants] [--exclude-west] [PROJECT ...]: clone and update the specified projects (default: all projects) based on the current west manifest.

    This command parses the manifest, clones any project repositories that are not already present locally, and checks out the project revisions specified in the manifest file, updating manifest-rev branches along the way.

    For safety, west update uses git checkout --detach to check out a detached HEAD at the manifest revision for each updated project, leaving behind any branches which were already checked out. This is typically a safe operation that will not modify any of your local branches. See the help for the --rebase / -r and --keep-descendants / -k options for ways to influence this.

    By default, west update also updates the west repository in the installation. To prevent this, use --exclude-west.

Miscellaneous Commands

West has a few more commands for managing the multi-repo, which are briefly discussed here.

  • west list: Lists project information from the manifest (URL, revision, path, etc.), along with other manifest-related information.

  • west manifest --freeze [-o outfile]: Save a “frozen” representation of the current manifest; all revision fields are converted to SHAs based on the current manifest-rev branches.

  • west diff [PROJECT ...]: Runs a multi-repo git diff for the specified projects (default: all cloned projects).

  • west status [PROJECT ...]: Like west diff, for running git status.

  • west forall -c COMMAND [PROJECT ...]: Runs the shell command COMMAND within the top-level repository directory of each of the specified projects (default: all cloned projects). If COMMAND consists of more than one word, it must be quoted to prevent it from being split up by the shell.

    To run an arbitrary Git command in each project, use something like west forall -c 'git <command> --options'. Note that west forall can be used to run any command, though, not just Git commands.

  • west selfupdate: Updates the west repository in the installation.