Building custom saltstack modules

Building custom saltstack modules 

This blog post was inspired by Joseph Hall's (Senior Engineer at Salt Stack, Inc.)  presentation about writing custom SaltStack modules.
Thank you Joseph!  http://www.youtube.com/watch?feature=player_detailpage&v=YP73LM8mzL0

SaltStack comes with a large number of extremely useful modules.  

http://docs.saltstack.com/ref/modules/all/

One very handy module is the cmdmod module.
http://docs.saltstack.com/ref/modules/all/salt.modules.cmdmod.html#module-salt.modules.cmdmod

The cmdmod module gives the ability to easily run shell commands against a set of minions.
This provides an ssh like ability across one or more servers in parallel, and is great for ad-hoc commands.

So, why would we need to create a custom SaltStack module?  Eventually, a need arises for new functionality which is not already included in the SaltStack project and does not fit neatly into an ad-hoc shell command.

My example:  Gather metadata about an AWS EC2 instance using a custom SaltStack module.

Background:

AWS metadata is very useful.  A running server configured for a purpose might need to be accessed directly, or identified for use by another service.

Example 1:  To ssh directly to an EC2 instance we need a DNS address or IP address
Example 2:  To add an EC2 instance to a load balancer, we need its AWS instance id.

Amazon recognized this need and created a way for an EC2 instance to retrieve data about itself.  By issuing a specific HTTP GET request from an EC2 instance, the HTTP response returned contains metadata specific to that instance.

AWS reference on metadata:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html

SaltStack to the rescue!  We can run a command on an EC2 instance without knowing its DNS name or IP address, and gather metadata!  This means we do not need to log into the AWS console to get information about an EC2 instance.  And furthermore, we can automate the consumption of such information.

Some metadata in which we might be interested:

• public-hostname (public dns address)
• public-ipv4 (accessible outside aws)
• instance-id (the unique id of an instance)
• placement (availability zone)
• security-groups (names of security groups applied to the instance)

Building the custom module

First:  Choose a module name that does not overlap with the modules supplied by SaltStack.  A name collision would result in our custom module replacing a SaltStack module.  My recommendation is to consistently prepend any module name with either a company name, abbreviation, developer name, initials or etc.  e.g.  wc_ec2_metadata.py

SaltStack reference on custom modules:

http://docs.saltstack.com/ref/modules/

Pre-requisites:

• An AWS account is set up and enabled for EC2
• A salt-minion is installed and running on the EC2 instance

Building our custom module on the EC2 instance:

• Create a directory for building custom saltstack modules (e.g.  # mkdir salt_modules)
• Start writing a module

(e.g. salt-call -m /path/to/my/modules module.fn )


Create a skeleton module and test it works

Test that our module and function will work as a salt module:

• salt-call -m ~/salt_modules  ec2_metadata.test

Expected Output:
local:
    True

Now let's add a useful function with some logging and error handling:
Test that our module and new function will work as a salt module:
salt-call -m ~/salt_modules  ec2_metadata.get_public_dns

Expected Output - a valid DNS name, not exactly what is below:
local:
    ec2-54-221-126-106.compute-1.amazonaws.com

All that is left to do is implement the remaining functions; (public-ipv4, instance-id, placement, security-groups).

Completed module:

There are a number of ways to distribute your new custom SaltStack module to the minions.

On a salt-master you first place your module in the appropriate _modules subdirectory under the "file_roots" location (typically /srv/salt/_modules) and then run one of the following:

• salt \* state.highstate 
• salt \* saltutil.sync_all
• salt \* saltutil.sync_modules  (only updates modules)

Viewing your module and function docstrings is easy using the included sys.doc module.

• salt-call sys.doc my-module  (returns all docstrings in a module)
• salt-call sys.doc my-module.some-func (returning a docstring for just one function in a module)
• Note: these work using 'salt' as well  e.g.  salt myserver sys.doc my-module

My public github repository:

git@github.com:wcannon/saltstack-related.git

https://github.com/wcannon/saltstack-related.git

https://github.com/wcannon/saltstack-related

Community contributed efforts:

Additionally, community contributed efforts (modules, states, grains and more) can be found here.

http://saltstarters.org/