Using development branch SaltStack python modules in the stable release

SaltStack has a lot of development work occurring and sometimes you want early access to features. Thankfully, for a number of Salt’s features it’s possible to very easily bring them into the stable release without needing to maintain a fork or a non-standard version.

Salt is built to be very modular, so that you can write/run your own custom modules. This same mechanism works for overriding the core modules as well. I’ll explain how this works in a masterless setup (since that’s what I use), but the process is pretty similar for master/minion setups as well.

Salt’s minion configuration has an option module_dirs that lets you define extra modules. Salt will look inside of this directory for grains, module, pillar, states and utils directories. Inside of each of those directories are python files that are associated with each type of module. utils is a bit of a special case. In Salt 2015.5 (Lithium) a new module abstraction layer was added to be able to override common utility code that other modules may use as common code. Let’s backport boto_kms as an example.

First, let’s add a modules directory, in /etc/salt/minion:

module_dirs:
  - /srv/salt/extra_modules

Now, we’ll copy some modules from the develop branch of salt:

$ cd git/salt/salt

$ mkdir /srv/salt/extra_modules/modules
$ cp modules/boto_kms.py /srv/salt/extra_modules/modules

$ mkdir /srv/salt/extra_modules/states
$ cp states/boto_kms.py /srv/salt/extra_modules/states

$ mkdir /srv/salt/extra_modules/utils
$ cp utils/boto.py /srv/salt/extra_modules/utils

Now we can run boto_kms:

$ salt-call boto_kms.describe_key 'alias/my-master-key'
local:
    ----------
    key_metadata:
        ----------
        AWSAccountId:
            12345
        Arn:
            arn:aws:kms:us-east-1:12345:key/76g7gg7g-778g-8h9h-hh8h-n99n9n9n9n
        CreationDate:
            1423687326.34
        Description:
            my-master-key
        Enabled:
            True
        KeyId:
            76g7gg7g-778g-8h9h-hh8h-n99n9n9n9n
        KeyUsage:
            ENCRYPT_DECRYPT

In general we try to keep the boto modules in a relatively stable state in the develop branch. Lyft’s development model for the boto modules is to upstream changes to the develop branch first, then override the modules in our own repos after they’ve been merged into develop. This also means that we try our best to ensure the boto modules are backwards compatible with the stable release at all times.