Truly ordered execution using SaltStack

SaltStack’s documentation implies that by default, since the Hydrogen (2014.1.x) release, execution of states is ordered as defined. In practice, however, this isn’t true. SaltStack supports a feature called requisites, which provide features like require, watch, onchange, etc.. Some requisites, like watch, are basically impossible to live without. For instance, if you want to conditionally restart a service when a configuration file changes you need watch. If you use requisites you can’t ensure the state run will execute in order.

I opened an issue for this, with a suggestion of how to fix the problem. Minutes later I was having a discussion with the SaltStack Inc. folks about the solution in IRC. That night Thomas Hatch pushed in a fix and let me know about it on Twitter. SaltStack’s responsiveness is truly awesome and is a bar I’d be ecstatic setting in any of my own Open Source projects.

The new feature, listen/listen_in, is something that acts just like the watch requisite, but does not modify state ordering. All state mod_watch actions triggered by listen will occur at the end of the state run. I say mod_watch, since some states, like service, have normal execution and a mod_watch action. For instance, service.running will ensure a service is running, but it’s mod_watch action is to restart a service.

Here’s an example of using listen_in:

Ensure apache2 is installed:
  pkg.installed:
    - name: apache2

Ensure apache2 is running:
  service.running:
    - name: apache2

Ensure apache2 is configured:
  file.managed:
    - name: /etc/apache2/apache2.conf
    - source: salt://apache2/apache2.conf
    # This will trigger an apache2 restart at the end of the run
    - listen_in:
      - service: apache2

Sometimes it’s necessary to immediately run a command, though. In that case you should still use watch. The major avoidance with watch is that you may define a state somewhere and use watch_in in multiple places, which makes it impossible to know the order in which your states will run. However, when you need to immediately run a state based off the action of a preceding
state, you know (and want) the order defined. For example:

Ensure myuser mongodb user exists:
  mongodb_user.present:
    - name: myuser
    - database: mydatabase
    - host: 127.0.0.1
    - port: 27017

# Mongo databases are created by adding something to the database.
Ensure the mydatabase mongodb database is populated:
  cmd.wait:
    - name: /srv/mycode/populatedb.py
    - watch:
      - mongodb_user: myuser