SaltStack
Salt is a remote execution and configuration management system written in Python (Private). It's designed to be heavily modular, and can be extended by simply dropping Python modules into the appropriate directories of the Salt state tree.
Design
Salt is typically deployed in a client-server scenario, where the Salt Master hosts the configuration and distributes it to a number of Minions. In larger deployments, Salt Syndic offers an intermediary between the Minion and Server allowing larger deployments by shielding the Master from the load of all Minions below it.
Other deployment models include:
salt-masterless
, where the Minion operates without a Master (file_client: local
).salt-ssh
, allowing an agent-less configuration where the Master performs actions over SSH.
Salt is built on two messaging channels used for all master-minion communication:
- Publish (
pub
) is used by the Master to communicate a job payload to a minion. - Returner (
req
) is used by the Minions to fetch files and send back job returns.
Transports
The messaging channels can be configured to use alternative transports. The default is ZeroMQ, using ports 4505/TCP and 4506/TCP for the pub
and req
channels.
Fileserver
State and pillar data is requested by the Minions from the Master, which behaves as a fileserver. By default, the fileserver uses the roots
module to serve files from directories local to the Master, but others allow serving from remote repositories and storage shares.
PKI
Since there's a risk of secret exposure in both the files shipped to Minions and the responses shipped back, Salt encrypts traffic in both directions. Salt Masters have public and private keys, and they'll need a copy of the public keys for all accepted minions too. Salt Minions have public and private keys, and must have the public key of the Master stored.
Key acceptance can be automated using the Reactor, by pre-seeding keys or through salt-key
on the Master.
Salt Cloud
Salt Cloud extends Salts management to Cloud computing providers like AWS and Azure.
Salt Thin
salt-thin
is a transportless version of Salt that allows it to execute entirely standalone, and forms the basis of the platform's container orchestration tooling. It works by packaging the minion into a tarball which can be shipped to the destination system for execution.
Package management
Salt provides a Windows-specific package management system called winrepo
that allows automated, unattended software installations similar to Linux systems.
Operating system support
Operating system | Master supported? | Minion supported? |
---|---|---|
Linux | ✅ | ✅ |
macOS | ❌ | ✅ |
Windows | ❌ | ✅ |
Installation
Repositories for most major Linux distributions are made available. Salt Bootstrap simplifies installation by determining the most appropriate packages for the system given your version preference.
Upgrading
The Salt Master is designed to be backwards-compatible with older versions of the Minion, except where doing so isn't possible due to breaking security fixes. Always update the Salt Master prior to updating the Minions.
As Minions require a restart in order to apply an update, be sure to use scheduled restarts with service.restart
.
Deployment over salt-ssh
This installation method is easier for onboarding large numbers of minions where no image-based deployment option is available. On the master, configure /etc/salt/roster
with an entry for each host:
foo:
host: foo.example.com
user: admin
passwd: secret
sudo: True
Test connectivity and credentials:
salt-ssh foo test.ping
If the fingerprint looks good, accept the host key:
salt-ssh -i foo test.ping
Now go ahead with installation (assuming the state already exists):
salt-ssh foo state.apply salt.minion
State tree
A state tree contains the definitions of individual service or file states. Conventionally, a state tree named base
exists alongside any number of application-specific trees. These should all be stored in /srv/salt
.
top.sls
defines the relationship between minions and the state tree, each entry selecting the appropriate nodes and listing states to apply to them. Each entry can match as many nodes as desired and as many states as desired, and a node can match multiple entries.
State files can exist directly in the root of the state tree, or nested in a directory below. When using directories, init.sls
will serve as the default state within that directory, and will run if an entry named after the directory appears in the top file. You can specify individual state files within the directory by adding entries in the form directory.state
.
The files are written in YAML, and are passed through the Jinja template engine (Private), allowing basic loops and execution of Salt modules. A few key variables defined in the templates:
salt
allows accessing execution modules (like__salt__
in Python).pillar
provides dictionary access into the Salt pillar. Usesalt['pillar.get']()
for more complex accesses, including setting a default value.
As the matrix of supported operating systems increases, variables should be extracted to a map.jinja
file alongside the state files, and imported via Jinja, e.g. to abstract package names and versions.
Pillar tree
Pillar trees contain sensitive data. The structure of the /srv/pillar
directory and the format of files within matches that of the state directory.
Targeting
Salt determines which configuration to apply to and which actions to perform on which Minions based on targeting data. This includes:
- Minion ID allows targeting by the Salt Minion's
id
, usually derived from its hostname. This is the default behaviour if no targeting switch is specified when usingsalt-call
, and allows specifying globs, e.g. wildcards (*
) to target, multiple Minions. - Regular expressions against the Minion ID can be specified with
-E
. - Comma-delimited lists of Minion IDs can be specified with
-L
. - Grains, automatically- or administrator-assigned key-value pairs describing properties of a system, can be used with
-G os:Debian
. Common ones include:os
, containing the operating system, e.g.Debian
orRedHat
.oscodename
, e.g.buster
.osrelease
, e.g.10
.
- Node groups can be specified with
-N
. - Compound matches allow combining multiple different conditions, along with basic negation, with
-C
. For example,G@grain:value and N@nodegroup
.
Node groups are configured in the Master's nodegroups
option, comprising a name and a compound target specification or a YAML list. Node groups can be nested.
Mine
The mine allows targeting a set of minions to obtain data for storage on the master, which can then be queried via the mine
execution module. This allows states to query information about the environment and template the state based on it.
The data to be collected is specified under the mine_functions
key of the state, which must be targeted via the top file:
mine_functions:
network.ip_addrs: []
# Note the use of the same mine function twice, but with an alias.
internal_ip_addrs:
mine_function: network.ip_addrs
interface: eth1
Mine functions are executed every mine_interval
(on the minion) minutes, but can be updated immediately using mine.update
.
Orchestration
Orchestrations are stored in /srv/salt/orch
and allow coordinating execution of Salt modules. For instance, to handle applying states to a newly provisioned backend application server, update the mine data to include the new server, and update the frontend configuration:
backend.apply-state:
salt.function:
name: state.apply
tgt: {{ pillar['backend'] }}
backend.update-mine:
salt.function:
name: mine.update
tgt: {{ pillar['backend'] }}
require:
- backend.apply-state
frontend.apply-state:
salt.function:
name: state.apply
tgt: {{ pillar['frontend'] }}
require:
- backend.update-mine
We can run from the master:
salt-run state.orch orch.sls pillar='{"backend":"backend42","frontend":"role:frontend"}'
CLI
Call module.function
on Minions matching target
with positional argument value1
and keyword argument arg2
set to value2
:
salt target module.function value1 arg2=value2
Execute the same function on the local Minion:
salt-call module.function value1 arg2=arg2
Execute the same function on a master, without requiring that the Minion be installed:
salt-run module.function value1 arg2=arg2
Not ideal for larger files, and contents is logged:
salt-cp target sourcefile destfile
Modules
Virtual modules abstract away operating system-specific implementation details, allowing Salt to dynamically load the most appropriate implementation for a given platform.
After adding a new module to the state tree, ensure you call the appropriate saltutil.sync_*
function to distribute it to the minions, or saltutil.sync_all
to sync everything.
Execution modules
Execution modules (_modules/*.py
) expose as many functions as desired, each of which performs an action and emits output.
Grains modules
Grains modules (_grains/*.py
) extend Salt's targeting capabilities. All public functions exposed by modules in this directory will be merged with the grains dictionary.
Returner modules
Returners (_returners/*.py
in the state tree) contain a single returner(ret)
function which can process and/or store Salt activity and result data. They might be used to raise tickets, store results in a database for later analysis or to send alerting emails to administrators when a critical state fails to apply.
Runner modules
Runner modules (_runners/*.py
) are Master-only execution modules, typically used for processing results or managing remote command execution.
State modules
State modules (_states/*.py
) expose functions for stateful resource management, typically used in the state tree. They should generally wrap an execution module, first checking whether any action is necessary to remediate drift from the desired state, then format a return value indicating what differences were fixed.
Utility modules
Utility modules (_utils/*.py
) are similar to execution modules, but designed for behaviours which are needed across other module types for reuse rather than being publicly exposed.
Backlinks