Announcing "saltcheck"!  A new way to test the logic of salt states and highstates dynamically

Check yourself before (and after) you Salt yourself

salt_check on github 

Background

After ending my project 'Auto Scaling and On-Demand Deployment of Custom Applications using SaltStack' with a presentation at SaltConf16 this year, the need to test salt states was the natural next step to improve and fortify the hosptalityPulse infrastructure.

Several presentations at SaltConf16 covered testing salt states. Salt has built-in functionality to aid in testing states and logic. But, they only cover testing what will happen before a state runs.

The best approach I found required using ServerSpec or TestInfra.  Unfortunately, this just didn't feel like the right answer.

At some point it dawned on me that this is an open source project and I could contribute in this area, gracefully supported by hospitalityPulse. 

Goal
The main objective of salt_check is to have an easy, and fast (run in parallel) testing solution for the logic of salt states, and highstates - a vague cousin of unit tests, dedicated to salt states. The solution should make writing tests as easy as running salt execution modules. And, programming knowledge will not be required to use this tool.

Here's how it works:

1. Create a state as a directory (e.g.  /srv/salt/apache/init.sls)
2. Create a sub-directory of the state directory and name it 'salt-check-tests'
3. Put one or more test files in the 'salt-check-tests' directory, each with a file name ending with .tst .

YAML Syntax for one test (replace text in caps):

Quick example of a saltcheck test



How do I run a saltcheck test?  Easily.



What tests are available:
All salt execution module functionality is available.  If there is functionality you need that is not in the 407+ salt execution modules just create a new one.
Salt Modules

Pros of saltcheck:

• supports salt renderers (e.g. yaml + jinja = dynamic tests)

• runs in parallel across servers

• makes use of salt event bus - very fast

• supports all salt cli targeting

• supports testing over salt-ssh

• more dynamic than serverspec/testinfra

• no required additional infrastructure to use

Cons of saltcheck:

• to test salt states that consist of single *.sls files they must be converted to directories in order to have a sub-directory beneath them

How to get started:

1. Clone the github repository locally
2. Copy the saltcheck.py file to your custom modules directory on your salt master in the "file_roots" location (typically in /srv/salt/_modules)
3. Sync the modules to your minions (salt '*' saltutil.sync_modules)
4. Check that saltcheck is available  (salt '*' saltcheck -d)
5. Write your first saltcheck test suite

** There is an example of how to test apache on ubuntu in the "examples" directory of the github repo.