Coalescing Jenkins and Ansible

aka Jensible

Every technology that exists in this world has two phases just like a coin i.e. every tool and technology that’s having some brilliance highlighted features also contains some effortful challenges. Ansible is meant for configuration management or workflow automation, it’s not a great CI/CD tool while Jenkins is a great CI/CD tool but not good at configuration and provisioning. So here’s the power of both is put together to create a simple completely automated workflow.

Ansible is a general-purpose automation tool that may be used for configuration management or workflow automation. Configuration management is an “infrastructure as code” practice that codifies things, e.g. what packages and versions should be installed on a system, or what daemons should be running. Workflow automation may be anything from provisioning cloud infrastructure to deploying software.

Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software.

Here Ansible is used as a provisioning and configuration tool and Jenkins as a CI/CD tool. First of all, ansible will provision the AWS resources like Instance, Keypairs, Security Groups, EKS Cluster and etc. After provisioning of the instance, its IP is dynamically updated in the ansible inventory file. Then on that instance, Jenkins will be installed and configured using ansible. After installing Jenkins, ansible will create and build a seed job which will generate other jobs and a build pipeline. This pipeline consists of four jobs and on triggered, this pipeline will test and deploy the website on EKS Cluster infrastructure after downloading the code from the developer’s GitHub Repository. One of the jobs of the pipeline will again use Ansible Playbook to provision EKS Infrastructure and deploy the website on it. Ansible is used in two scenarios, first to install and configure Jenkins then Jenkins job uses it to deploy the website. After this, when the developer pushes updated website code to Github Repository that pipeline will be again triggered using Github webhooks and automatically website running in the production environment will be updated.

Here is the directory structure of the project. ‘web_cdcd.yml’ is the main playbook which includes the roles.

• Role aws will run on localhost and connect with AWS as a client for provisioning the infrastructure. It’ll create key-pairs, security groups, launch instances and create EKS cluster. This role is also responsible for updating the Public IP address of newly launched instance in ansible inventory file under the specified host group. Here our host group name is ‘jensible’.
• Role jenkins will install and setup the Jenkins on the hosts under jensible group which is dynamically updated by aws role with IP of launched instance.
• Role jenjobs will create and trigger the seed job in Jenkins which will generate other jobs and pipeline.

Main Playbook

Ansible Configuration — This is the ansible configuration file which includes the path to inventory, path to the private key file, privilege escalation block and some other configurations.

Ansible Configuration File

Inventory before the playbook runs doesn’t contain any ‘jensible’ group because that will be dynamically updated after launching the instances.

Inventory Before

AWS Status Before — Before running the playbook here is the status of EC2 dashboard and EKS. There is no running instances, no keypairs, only one default security group and no EKS cluster.

AWS Pre Status

Running Playbook — Now Its time to run the playbook and after this ansible will do everything steps by steps form provisioning the infrastructure to creating and triggering the jobs in Jenkins.

Running Playbook

EC2 Status After — After some time of running the playbook, here is the status of EC2 which shows that two key pairs, four security groups and one instance is created.

EC2 Dashboard After

Here is the instance which is launched and now its IP will be updated in Inventory and then Jenkins will be installed and configured in it.

Jenkins Instance

Inventory After launching the instance is dynamically updated with ‘jensible’ group and instance IP Address.

Also after creating the key-pairs, their private key pem files are stored locally and they will be used by Ansible to connect with instance and configure it.

Key PEM and Updated Inventory

EKS Status — One EKS Cluster is also deployed along with one Node Group. This EKS Cluster will be used to deploy the website afterwards.

EKS Cluster

Mailed Jenkins URL — After installing and configuring Jenkins in the instance which is launched, ansible will send a mail to specified mail address containing the Jenkins URL and the initial Admin Password.

Here Seed Job will generate four Jobs -

1. Git Pull — This Job will download the WebApp Code from GitHub. This Job is Triggered by GitHub Webhooks i.e. when any developer pushes new code to GitHub this job will be triggered.
2. Check and Deploy — First of all, this Job will check which language is used in the code, which is downloaded and then runs the environment accordingly to deploy the website. If the required environment is already running then this job will only copy the new code.
3. Test App — This Job will test the website whether it is working well or not and if the website is not working well then this job will be successful and triggers Job4 to notify the developer. If the site is working well then this Job will fail intentionally, So that next Job will not be triggered which is for notifying the site developer.
4. Send Mail — This Job will only be triggered if Job3 will get Success and this will only happen if the site is down due to some issue. On trigger, this Job will send mail to the developer asking to solve the issue in the website and again push updated code on GitHub.

Jenkins Jobs

Pipeline — Here’s the pipeline which is generated by a seed job. This pipeline is a workflow of those four generated jobs.

Jenkins Pipeline

Job2 using Ansible — In the pipeline, Job2 is to deploy the website. This job2 will again use ansible to deploy website over AWS EKS Infrastructure. All the Kubernetes resources like PVC, Deployment and Services are created on EKS Cluster using Ansible.

Jenkins Job2 using Ansible

Website URL Mail — Ansible playbook in Job2 of Jenkins will send a mail on specified Address after deployment of the website. This mail contains the URL of the website.

Now as soon as developer pushes updated website code to GitHub, the Jenkins pipeline will be triggered automatically using GitHub Webhooks and again all jobs will run and finally the updated website will be deployed in the production environment.