A co-worker wanted a third-party Python modules installed onto the Ansible Tower servers that I maintain. I don’t like installing any and all packages that people ask for since this is a shared system, but I had to bring myself up to speed on how he could get his module installed to use by his playbooks.
- The machine that the Ansible playbook is executed from only needs to have Ansible and a set of pre-requisite Python modules installed on it. For this document, we’ll refer to that system as the “Ansible Server” or just “the server”.
- The machine(s) that the Ansible playbook make work on and perform changes to need ssh and a small sub-set of Python modules (usually the core Python packages). We’ll refer to those systems as “the clients”.
To make things more confusing there are two types of “modules” that will be referenced:
-
An “Ansible module” is a package that Ansible uses on the server to
execute steps on the clients. These are usually written in Python and
the “core” Ansible modules are included and maintained by the Ansible
developers.
- There are some Ansible modules that may be included with Ansible but they are maintained by the community. These are usually specialized modules specific to a hardware or software vendor and are usually maintained by the vendor or others interested in automating that vendors tools.
-
The other “module” referenced in this document are add-ons to Python and are called “Python modules”.
- A Python module may perform some low-level task (e.g. network connection, DNS lookup, etc) and are NOT Ansible specific.
Documentation for the Ansible modules are located here: https://docs.ansible.com/ansible/latest/modules/modules_by_category.html
The request mentioned the need for the “GitHub3.py” Python module so the “github_release” Ansible module would work. The documentation for the “github_release” module has a requirements section and it notes that also. The documentation page also notes that this is a preview module (“This module is not guaranteed to have a backwards compatible interface.”) and it is also maintained outside of the Ansible core develoers (“This module is maintained by the Ansible Community.”).
So, how do we add this module? I’m glad you asked!
The first thing to understand is that all the requirements for this module have to be installed on the clients, not on the Ansible servers. While this sounds like more work, it really isn’t and it keeps the Ansible servers free from conflicts that different users might have requiring different Python module versions. The key to all this is the use of Python “Virtual Environments” (or “venvs”). These virtual environments are walled-off areas that have their own Python executable and associated modules; it’s even possible to have different versions Python installed in different venvs for testing.
In the playbook that needs to use an Ansible module that has special Python module dependencies, there are a few steps to take that we’ll go over in detail below:
- Ensure pip is installed
- Install Python virtual environment packages into the venv
- Setup the virtual environment
- Install base Tower packages into venv
- Install the Python module specifically needed into venv
- Use the new venv to execute the Ansible module
Step 1 – Ensure pip is installed
This is a basic step and will vary by OS but the “pip” package is needed for the “pip:” modules later.
- name: "Ensure pip is installed"
yum:
name: python2-pip
state: installed
Step 2 – Install Python virtual environment packages
This is also OS dependent, but it installs the “python-virtualenv” package so Python can build venvs.
- name: "Install Python virtual environment packages"
yum:
name:
- python-virtualenv
state: installed
Step 3 – Setup the virtual environment
This step does the initial work to build the virtual environment. The venv is just a directory structure (in this case “/tmp/test01.venv”) that contains helper files and some wrapper scripts and configuration defaults.
- name: "Setup the initial virtual environment"
pip:
name:
- setuptools
extra_args: --upgrade
virtualenv: "/tmp/test01.venv"
Step 4 – Install base Tower packages into venv
Strictly speaking, if you’re not using this venv with Ansible Tower it is not necessary, but it will make the playbook usable in more places.
- name: "Install base Tower packages"
pip:
name:
- python-memcached
- psutil
umask: "0022"
state: present
virtualenv: "/tmp/test01.venv"
virtualenv_site_packages: yes
extra_args: --ignore-installed
Step 5 – Install the Python module specifically needed into venv
Finally we’re at the step where we’re installing the Python module we need. This Python module (like the others earlier) are only installed into the venv directory structure.
- name: "Install github3.py module"
pip:
name:
- github3.py
virtualenv: "/tmp/test01.venv"
Step 6 – Put the new venv with the Python module to work
The key at this step is the “vars:” section that tells the Ansible execution environment to use the “python” binary found in the “venv” on the remote system, “/tmp/test01.venv/bin/python” in this case.
- name: "Download latest relase of ISOwithKS"
# https://github.com/dglinder/ISOwithKS.git
vars:
ansible_python_interpreter: "/tmp/test01.venv/bin/python"
github_release:
user: "dglinder"
repo: "ISOwithKS"
action: latest_release
PLEASE NOTE: The “github_release:” example above does NOT work due to something un-related to the venv created.
How does this work?
When the playbook runs it connects to all of the clients and makes sure the “python2-pip” and “python-virtualevnv” packages are installed, it then builds the bare virtual environment into “/tmp/test01.venv/” and populates that venv with additional modules, then installs the Python modules necessary to execute the Ansible module. The Ansible module is executed using the “python” executable in the newly built venv.
Note that ALL of these steps are preformed on the Ansible clients, no changes are made to the Ansible server. In testing, the initial execution of these steps took about 40-50 seconds to get to the final step – most of that time was due to downloading packages from the Pip repository (on the Internet). Subsequent runs that were able to re-use the venv directory took 20-25 seconds to get to the same location.
Caveats
One big shortcoming of this process is the necessity of the Ansible clients to have access to download the packages from an Internet location. If the clients are shielded from the Internet, it may be necessary to setup a proxy server they can use (if permitted).
It might be necessary to perform the venv build on a single server with Internet access, then replicate that venv directory structure to each of the clients. (These workarounds have not been validated, so test and report back any success or failures.)