A co-worker wanted a third-party Python modules installed onto the Ansible Tower servers that I maintain. I don’t like installing any and all packages that people ask for since this is a shared system, but I had to bring myself up to speed on how he could get his module installed to use by his playbooks.
-
The machine that the Ansible playbook is executed from only needs to
have Ansible and a set of pre-requisite Python modules installed on it.
For this document, we’ll refer to that system as the “Ansible Server”
or just “the server”.
-
The machine(s) that the Ansible playbook make work on and perform
changes to need ssh and a small sub-set of Python modules (usually the
core Python packages). We’ll refer to those systems as “the clients”.
To make things more confusing there are two types of “modules” that will be referenced:
-
An “Ansible module” is a package that Ansible uses on the server to
execute steps on the clients. These are usually written in Python and
the “core” Ansible modules are included and maintained by the Ansible
developers.
-
There are some Ansible modules that may be included with Ansible but
they are maintained by the community. These are usually specialized
modules specific to a hardware or software vendor and are usually
maintained by the vendor or others interested in automating
that vendors tools.
-
The other “module” referenced in this document are add-ons to Python and are called “Python modules”.
-
A Python module may perform some low-level task (e.g. network connection, DNS lookup, etc) and are NOT Ansible specific.
Documentation for the Ansible modules are located here:
https://docs.ansible.com/ansible/latest/modules/modules_by_category.html
The request mentioned the need for the “GitHub3.py” Python module so the “github_release” Ansible module would work. The
documentation for the “github_release” module has a requirements
section and it notes that also. The documentation page also notes that
this is a preview module (“This module is not guaranteed to have a
backwards compatible interface.”) and it is also
maintained outside of the Ansible core develoers (“This module is
maintained by the Ansible Community.”).
So, how do we add this module? I’m glad you asked!
The first thing to understand is that all the requirements for this
module have to be installed on the clients, not on the Ansible servers.
While this sounds like more work, it really isn’t and it keeps the
Ansible servers free from conflicts that different
users might have requiring different Python module versions. The key
to all this is the use of Python “Virtual Environments” (or “venvs”).
These virtual environments are walled-off areas that have their own
Python executable and associated modules; it’s
even possible to have different versions Python installed in different
venvs for testing.
In the playbook that needs to use an Ansible module that has special
Python module dependencies, there are a few steps to take that we’ll go
over in detail below:
-
Ensure pip is installed
-
Install Python virtual environment packages into the venv
-
Setup the virtual environment
-
Install base Tower packages into venv
-
Install the Python module specifically needed into venv
-
Use the new venv to execute the Ansible module
Step 1 – Ensure pip is installed
This is a basic step and will vary by OS but the “pip” package is needed for the “pip:” modules later.
- name: "Ensure pip is installed"
yum:
name: python2-pip
state: installed
Step 2 – Install Python virtual environment packages
This is also OS dependent, but it installs the “python-virtualenv” package so Python can build venvs.
- name: "Install Python virtual environment packages"
yum:
name:
- python-virtualenv
state: installed
Step 3 – Setup the virtual environment
This step does the initial work to build the virtual environment.
The venv is just a directory structure (in this case “/tmp/test01.venv”)
that contains helper files and some wrapper scripts and configuration
defaults.
- name: "Setup the initial virtual environment"
pip:
name:
- setuptools
extra_args: --upgrade
virtualenv: "/tmp/test01.venv"
Step 4 – Install base Tower packages into venv
Strictly speaking, if you’re not using this venv with Ansible Tower
it is not necessary, but it will make the playbook usable in more
places.
- name: "Install base Tower packages"
pip:
name:
- python-memcached
- psutil
umask: "0022"
state: present
virtualenv: "/tmp/test01.venv"
virtualenv_site_packages: yes
extra_args: --ignore-installed
Step 5 – Install the Python module specifically needed into venv
Finally we’re at the step where we’re installing the Python module we
need. This Python module (like the others earlier) are only installed
into the venv directory structure.
- name: "Install github3.py module"
pip:
name:
- github3.py
virtualenv: "/tmp/test01.venv"
Step 6 – Put the new venv with the Python module to work
The key at this step is the “vars:” section that tells the Ansible
execution environment to use the “python” binary found in the “venv” on
the remote system, “/tmp/test01.venv/bin/python” in this case.
- name: "Download latest relase of ISOwithKS"
# https://github.com/dglinder/ISOwithKS.git
vars:
ansible_python_interpreter: "/tmp/test01.venv/bin/python"
github_release:
user: "dglinder"
repo: "ISOwithKS"
action: latest_release
PLEASE NOTE: The “github_release:” example above does NOT work due to something un-related to the venv created.
How does this work?
When the playbook runs it connects to all of the clients and makes
sure the “python2-pip” and “python-virtualevnv” packages are installed,
it then builds the bare virtual environment into “/tmp/test01.venv/” and
populates that venv with additional modules,
then installs the Python modules necessary to execute the Ansible
module. The Ansible module is executed using the “python” executable in
the newly built venv.
Note that ALL of these steps are preformed on the Ansible clients, no
changes are made to the Ansible server. In testing, the initial
execution of these steps took about 40-50 seconds to get to the final
step – most of that time was due to downloading packages
from the Pip repository (on the Internet). Subsequent runs that were
able to re-use the venv directory took 20-25 seconds to get to the same
location.
Caveats
One big shortcoming of this process is the necessity of the Ansible
clients to have access to download the packages from an Internet
location. If the clients are shielded from the Internet, it may be
necessary to setup a proxy server they can use (if permitted).
It might be necessary to perform the venv build on a single server
with Internet access, then replicate that venv directory structure to
each of the clients. (These workarounds have not been validated, so
test and report back any success or failures.)
