Deploying a typical LAMP application using SaltStack - my experience

Using SaltStack to deploy a typical LAMP application

This article describes my approach to a recent software deployment project.  The goal of the project is to create a secure fast deployment method for multiple LAMP stack applications.  

Previously I have used the python fabric library to deploy software applications.  Having recently set up "instant" software infrastructures using SaltStack I was eager to take advantage of its remote execution capabilities, and node introspection ala grains.

SaltStack Introduction for Deployment

SaltStack is designed to allow parallel remote execution over encrypted channels, and also provides configuration management.  Essentially, SaltStack provides the basis to make things happen quickly, repeatedly, and exactly as desired.    It is also a python based open source project with an active developer community.

Overall Deployment Design

I chose to conceptualize deployment to consist of two distinct pieces.  

Piece # 1:  The assembly or build of the application.  For the three applications I deployed this consisted of combining php files with configuration files.

Details related to Piece # 1:

Each application has a specific github repository.  And, each branch for the repository is deployable if an environment name matches the branch name (convention over configuration).  At build time, the least amount of effort to update the code locally is used (e.g. if a project has already been cloned, and its branch checked out, we simply perform a "git pull")

Piece # 2:  The actions necessary to put the new application into place and be served by the webserver.  This consists of executing a custom python module.

Details related to Piece # 2:

A SaltStack custom module is simply a standard python module that has been synchronized onto the minion.  (e.g.  salt \* saltutil.sync_modules ).  A function in a module can call a function in another module.  This is very handy and is called "cross-calling."  

Note:  If a function in the module is prepended with underscore characters it will not be callable outside the module (iow a 'hidden' function).  

** A third piece exists:  A wrapper that takes user input, and causes piece 1 and piece 2 to happen.  This piece uses docopt for command line argument processing, the salt filesystem, and a salt runner to signal the targeted servers to run their custom python module.

Details related to Piece # 3:

A targeted server is really given a message via the queuing system on the salt-master.  The targeted server identifies it should do something, and then does it, reporting its result back to another queue on the salt-master.  I have used the "grains" system in order to have a server know what roles it has.  (e.g.  is_wepapp1_server: true)

Code Snippets:

DocOpt Sample:

Salt Runner Sample:

SaltStack Custom Module Sample:

Results:

Very fast deployments for a 5 mb application -- (averaged 5.5 seconds)

An application deployment consists of these steps:

1. log onto salt master
2. run the deployment script:   # ./deploy.py  APPNAME ENVIRONMENT                               e.g.  ./deploy.py  webapp1  production
3. observe output of deployment script

Using the same name as above we could call the custom module directly:

# salt -C  'G@environment:production and G@hosting_webapp1:true'  deployment_webapp1.deploy  my-build-file.tar.gz

Positive elements:

• no list of servers to maintain
• no ssh key management 
• scalable, works against lots (read 100's of servers in parallel)
• can use the grains on the servers for other remote ad-hoc execution
• very simple command line usage (dynamically building the module name to publish)
• trivial to add more deployment modules

Negative elements:

• none specific to SaltStack

Items for improvement / enhancement:

• capture deployment history
• rollback capability / specific revision deployment 
• use of a custom returner (rather than stdout on salt-master) -- would prove helpful for queries, or dashboards
• remove older previous deployments (e.g. only retain the last 3)

Choices that worked out well:

• docopt = simply a great library (sorry argparse - my previous favorite)
• keeping a copy of repos / branches on the deployment server = major deployment time reducer
• use of grains on servers (roles & etc) = great way to encourage better management practices, very handy for general usage 
• use of a salt runner rather than calling specific deployment modules via the salt command = allowed me to simplify the usage of the deploy script
• cross-calling salt modules = great way to reuse code, and helps shrink the size of a deployment module

Helpful Links:

docopt:  http://docopt.org/

saltstack:  http://docs.saltstack.com/

saltstack installation:  http://docs.saltstack.com/topics/installation/index.html

saltstack targeting:  http://docs.saltstack.com/topics/targeting/grains.html

saltstack custom module:  http://docs.saltstack.com/ref/modules/index.html
saltstack runner: http://docs.saltstack.com/ref/clients/index.html#runnerclient

saltstack module cross calling:  http://docs.saltstack.com/ref/modules/index.html?highlight=cross#cross-calling-modules