Home

Welcome to the jenkins-docker-vagrant-ansible documentation!

To get started view the Introduction page, otherwise, select the specific topic that you require.

Jump right in!

github Repository

wiki Edit On Guthub

Useful Links:

Introduction

This repository was created to provide a new team with a pre configured Jenkins master and slave environment. The Jenkins master is configured with all the essential tools needed to run Jenkins right out of the box. The Jenkins slave machine is pre-configured with docker installed, to support the provisioning of docker on demand slaves.

Vagrant and Ansible is being used to provision these environments, on either Amazon Web Services or by using a local virtual environment such as Virtual Box. With the tools provided, this will enable any team to provision these environments with little effort and in a repeatable fashion.


Credits

This repository was inspired and designed around ansible-jenkins:

https://github.com/ICTO/ansible-jenkins

Docker Build Pipeline

Currently, the ansible scripts provision a generic Docker build pipeline, with the following layout.

  1. Phase 1
    • GetLatestFromSCM
  2. Phase 2
    • IntegrationTesting
    • UnitTesting
  3. Phase 3
    • PushToDockerHub
  4. Phase 4
    • FunctionalTesting
  5. Phase 5
    • PushToDockerHub-QA
  6. Phase 6
    • DeployToQA

Key Jenkins plugin used:

NB. All plugins and dependencies listed above are all installed using Ansible by default. Also, most of the tools needed to run builds are installed on the docker slaves.

To provision the Jenkins environment with your pipeline of choice, the target_jenkins_env variable should be overridden.

To override, locate the desired override variable file in the provisioners/ansible/roles/shared/vars folder and place the following within the yaml file.

Example

target_jenkins_env: docker # creates a docker pipeline

Quick Start Guide

Purpose

This guide will give you an understanding of the elements that needs to be updated in order get up and running with the Jenkins environment provision by the ansible scripts.

The Jenkins environment that is provided by the ansible script is configured for general usage. What this mean is that, there isn't a “one configuration fits all scenario”. The continuous integration environment has to be configure on a case by case basis. In order to accomplish this, you’re required to have some familiarity with the following elements before proceeding:

There is a wealth of documentation here. Its highly recommended that you review this document.

Below there are links pointing to specific elements that can help as well.

Each step is listed below with the required items identified.

All the Job in the pipeline may require important build variables to be propagated throughout the pipeline an example of this can be found here

GetLatestFromSCM

IntegrationTesting

UnitTesting

PushToDockerHub

FunctionalTesting

PushToDockerHub-QA

DeployToQA

Sample Build Variables

These are some sample build variables that can be used in your build pipeline, to make the pipeline configurations cleaner.

This can be used with parameterized-trigger that already provision on Jenkins, once this script was used to provision it. Documentation on this plugin can be found here.

DB_NAME - Test database name
CI_DB_DOMAIN - Address for the CI test database
CI_DB_PORT - Port for the CI test database
QA_DB_URL - Address for the QA test database
QA_DB_PORT - Port for the CI test database
APP_VERSION_TAG - Unique tag for the docker container
APP_SERVER - Address for the application server
FLEETCTL_TUNNEL - IP address for the CoreOS fleet
CI_HTTP_PORT - IP address per build when doing functional test

Requirements

Two vagrant plugins are required. To install these plugins, please run:

vagrant plugin install vagrant-vbguest
vagrant plugin install vagrant-aws

Features

Whats already inside

Major Apt Packages

jenkins, docker, jmeter, git, jre-7, jdk-7, groovy

To see a full list of apt packages installed, see the following file:

Shared Vars

Note: Most of the tools that will be need to run builds are expected to be installed on the Docker containers, which will be used as Jenkins slaves in this setup.

CheckUrl Groovy Script

What is CheckUrl?

CheckUrl is a script created to ping a website to check if it is giving a 200 response in a favorable response time.

Use Case

The perfect use case for this script is to check until a site comes online after a deploy. If it doesn't get a 200 response after a specified interval, then the deploy can be considered a failure. Works great for deploying a test or production site.

Where is it?

The script is transferred to the /var/lib/jenkins/shared_ansible location on each machine provisioned. This ensures there is a consistent location on Jenkins Master and all slaves to enable seamless builds across the environment. The script does not run indefinitely; it runs for a specified amount of tries and will exit with a failure or success depending on the Final Response.

How to Use

groovy and curl must be installed to run the script. Luckily these are already installed when provisioning with Ansible.

To get help with figuring out the arguments to pass when running the script, you can use the -h argument to see instructions.

$ groovy checkurl.groovy -h

usage: checkurl.groovy -[hmst] [url]
 -h,--help                 Show usage information
 -m,--max-time <maxTime>   max time in seconds before killing a curl
                           command, Default: 20
 -s,--sleep <sleep>        time in seconds before next retry, Default: 4
 -t,--tries <tries>        number of tries before exit, Default: 3

Example

The example below pings a site every minute and attempts 12 tries before exiting.

$ groovy /var/lib/jenkins/shared_ansible/checkurl.groovy -m 30 -s 60 -t 12 http://ec2-54-221-51-114.compute-1.amazonaws.com

Checking 'http://ec2-54-221-51-114.compute-1.amazonaws.com' with: 
  12 tries 
  30 seconds timeout per try 
  60 seconds interval before next try 

LETS ROCK!
Response is 404: not satisfactory ... executing retry (11 left)
elapsed time: 0.359 seconds 

Response is 404: not satisfactory ... executing retry (10 left)
elapsed time: 1 minutes, 0.480 seconds 

Response is 404: not satisfactory ... executing retry (9 left)
elapsed time: 2 minutes, 0.512 seconds 


(0) Final Response is 200: site is ready!
elapsed time: 3 minutes, 4.586 seconds

The (0) in this line (0) Final Response is 200: site is ready! is the exit code.

SauceLabs Python Script

The SauceLabs.py script, discovered in a blog, is used to run robotframework tests on SauceLabs while creating the outputs on the machine that executed the command.

This Requires the following pip packages:

Where is it?

The script is copied to the /usr/local/bin directory where it can be referenced by python when a script is run that imports the SauceLabs library

How to Setup

There are several steps that are required and some may be optional that will get you going with this repository. The following Pages will give you detailed information on these setup steps.

Install Plugins

To edit the list of plugins to install when provisioning, look for the following file and edit the plugins section as necessary provisioners/ansible/roles/shared/vars/main.yml

plugins: # Jenkins Plugins
  - 'ldap'
  - 'translation'
[...]

Create Jobs

To have a job be created by the Ansible Provisioning then a job xml should be placed in the following directory: provisioners/ansible/files/jenkins/jobs

The job created will match the name of the file

eg.

The file: provisioners/ansible/files/jenkins/jobs/ExampleJob.xml

Will create a job called: ExampleJob

Share Files with the Image/Box

If there are extra files needed to be shared from Ansible to Jenkins, place them in the following folder: provisioners/ansible/files/jenkins/shared. This will create and copy these files to a folder in Jenkins home: {jenkins_home}/shared_ansible. The default user will be the owner of these files once copied over but will be accessible to any user of the machine.

Update Override Variables

Where to find these?

These variables are located in different files in the provisioners/ansible/extra_vars folder. The file to change will depend on the environment you are provisioning.

Check/Update Waiting time for jenkins restarts

To override the time it waits (in seconds) for Jenkins to start please edit the respective file in the provisioners/ansible/extra_vars folder

startup_delay_s: 50

Set Git Credentials

Update the git name and email to the credentials specific to your Jenkins setup

Settings:

eg.

git:
  enable_configure: true
  email: 'noreply@gmail.com'
  name: 'Jenkins CI'

Set Rally Variables to Preconfigure Plugin

Settings:

eg.

rally:
  enable_configure: true
  server: "rally1.rallydev.com"
  email: ""
  jenkins_machine: "localhost:8080"

Please note that this step does not fully configure the rally plugin. You will have to navigate to configure system when Jenkins goes live and enter the password for the rally user/email

Edit GitHub Security Settings

To setup Jenkins security please edit the respective file in the provisioners/ansible/extra_vars folder with the necessary variables

Settings:

To get the information from github:

eg.

security:
  enable_security: true,
  jenkins_admins: "admin1,admin2", #comma delimited list eg. "admin1,admin2"
  github_orgNames: "medullan", #comma delimited list eg. "medullan,google"
  github_clientId: "532534253fw3245",
  github_clientSecret: "32refwdfs324rewf343q4rwqr32qr"

Things to Note:

If there are raw xml config files that you want to be copied to Jenkins. Then simply adding them to the provisioners/ansible/files/jenkins/config directory will get them to Jenkins for pre-configuration.

Configure Jenkins Memory

Configure the heap size Jenkins will be assigned on startup.

eg.

jenkins_opts:
  enable_configure: true,
  maxPermSize: 512, # cannot be less than 512
  memory: 1024 # cannot be less than 256

Install global npm packages

eg.

npm:   # bower, grunt-cli and istanbul are installed by default
  global_packages: "doxx npm-check-updates" # global_packages is a space delimited list eg. 'bower grunt-cli'

Create Jenkins Environment with Vagrant

There are currently two options for creating a Jenkins environment with Vagrant:

Local provisioning with VirtualBox

By default, the Vagrantfile is setup to provision using VirtualBox and a clean ubuntu box from VagrantCloud

To provision:

Provision with AWS (Amazon Web Service)

Step 1 - Change the vm box to be used

To provision with AWS, the AWS box built with packer or a referenced AWS vagrant box from some archive must be used. If the box was built update the base box in the Vagrantfile

for eg.

config.vm.box = "dummy.box"

Step 2 - Change Ansible Override Variables

from:

ansible.extra_vars = "provisioners/ansible/extra_vars/jenkins-master-playbook-vars.yml"

to:

ansible.extra_vars = "provisioners/ansible/extra_vars/jenkins-master-aws-playbook-vars.yml"

Step 3 - Fill in required AWS info

Inorder to provision environments within AWS, it is a requirement to provide sensitive AWS information such as the access_key_id and secret_access_key. Since this code is being committed to a github repository where other persons can view the code base, environment variables are being used to set this information in a local terminal, for e.g. export AWS_SECRET_ACCESS_KEY=SOMEKEYHASH.

  def aws_provider_configs(config)
    config.vm.provider :aws do |aws, override|
        override.ssh.username = ENV["AWS_SSH_USER"]
        override.ssh.private_key_path = ENV["AWS_KEY_LOCATION"]

        aws.keypair_name = ENV["AWS_KEYPAIR_NAME"]
        aws.access_key_id = ENV["AWS_ACCESS_KEY_ID"]
        aws.secret_access_key = ENV["AWS_SECRET_ACCESS_KEY"]
        aws.ami = ENV["AMI"]
        aws.security_groups = [ENV['AWS_SECURITY_GROUP']]

        aws.region = "us-east-1"
        aws.tags = {
          'Name' => 'jenkins-docker-master',
          'Provisioner' => 'Medullan',
          'OS_Version' => 'Ubuntu',
          'Release' => 'Latest'
        }
    end
  end

Step 4 - Uncomment Rsync Folder sharing

This step is important. The rsync line lists files that should be ignore when syncing files on the local machine with the AWS machine. If these are not ignore then the process will attempt to transfer very huge files and you may wait a very long time before seeing any progress.

config.vm.synced_folder ".", "/vagrant", type: "rsync", :rsync_excludes => ['packer_cache/', 'http/', ... ]

then run:

$ vagrant up --provider=aws

This will create and run an AWS instance in your account.

For more information on provisioning with AWS please view the following repository:

https://github.com/mitchellh/vagrant-aws

Extras

There are some Ansible roles that are shared when provisioning the base image and the Jenkins environments. One such role would be setup. If you wat to ignore this role when provisioning the Jenkins environments, Then uncomment the following line in the Vagrantfile:

ansible.skip_tags = ['setup']

Caveats

When provisioning the Jenkins environment with AWS for the first time, the provisioning will fail for SSH reasons. To see how to resolve, please see the Known Issues section for this topic

Documentation

About

This documentation is hosted for editing on Github wiki and parsed into HTML for the gh-pages. These pages are parsed in the order of how they appear and will be displayed on the website in the same order. The Home Page is an exception to this rule however, it always appears first in the generated documentation. This is so because of the globbling pattern used in the gruntfile:

var markdown = [
      'jenkins-docker-vagrant-ansible.wiki/Home.md',
      'jenkins-docker-vagrant-ansible.wiki/*.md',
      '!jenkins-docker-vagrant-ansible.wiki/_Footer.md'
    ];

This pattern includes the Home page first, includes all other files and then ignores the Footer page used in the wiki.


Editing the Docs

You can edit the documentation by visiting the Github wiki of this repository. The wiki is parsed and used to generate the documentation for the website.


Generate Documentation Website

To get started with generating the documentation, you must have already cloned the git repository and be inside the root directory with your console.

The tools needed to get you started are all powered by:

Therefore, you must have Nodejs and npm installed with grunt installed globally as a npm package

Assuming all the dependencies above are installed and ready to use, the following steps will show you how to generate documentation for the repository.

Step 1

run

$ npm install

This will install all npm modules/dependencies needed within the project to generate the documentation.

Step 2

run

$ grunt docs

This will then generate the documentation locally to the docs folder.


Deploy Docs to GH-Pages

When the documentation is generated and parsed properly then you can deploy to the website

NB. Please review generated docs locally before deploying

You can deploy by running:

$ grunt deploy

Development

Rules

Known Issues

Issue: Invalid machine state when provisioning

With VirtualBox v4.3.14, when doing vagrant up, an error (or similar error) sometimes occurs saying:

Exception

The guest machine entered an invalid state while waiting for it to boot. Valid states are 'starting, running'. The machine is in the 'poweroff' state. Please verify everything is configured properly and try again. If the provider you're using has a GUI that comes with it, it is often helpful to open that and watch the machine, since the GUI often has more helpful error messages than Vagrant can retrieve. For example, if you're using VirtualBox, run vagrant up while the VirtualBox GUI is open.

Resolution:

To resolve the issue, downgrading to version VirtualBox v4.3.12 worked


Issue: ansible.host turned into executable file

Sometimes pulling the repository down will make ansible.host an executable file and will produce the following error:

Exception

ERROR: The file provisioners/ansible/ansible.host is marked as executable, but failed to execute correctly. If this is not supposed to be an executable script, correct this with chmod -x provisioners/ansible/ansible.host.

Resolution

To resolve the issue, run chmod -x provisioners/ansible/ansible.host


Issue: Ansible Provisioning fails when using Jenkins CLI

Sometimes this will happen because Jenkins is not fully online after a restart during provisioning. This is either because the sleep time isn't long enough or Jenkins takes a little longer to start.

Exception

failed: [54.69.156.112] => {"changed": true, "cmd": "java -jar /opt/jenkins/jenkins-cli.jar -s http://localhost:8080 list-jobs All", "delta": "0:00:06.720559", "end": "2014-09-30 14:49:18.306571", "rc": 1, "start": "2014-09-30 14:49:11.586012"}
stderr: Exception in thread "main" java.io.IOException: No X-Jenkins-CLI2-Port among [X-Jenkins, null, X-Hudson, X-Hudson-Theme, Content-Length, Expires, X-Jenkins-Session, Set-Cookie, Content-Type, Server, Cache-Control]
    at hudson.cli.CLI.getCliTcpPort(CLI.java:283)
    at hudson.cli.CLI.<init>(CLI.java:126)
    at hudson.cli.CLIConnectionFactory.connect(CLIConnectionFactory.java:72)
    at hudson.cli.CLI._main(CLI.java:466)
    at hudson.cli.CLI.main(CLI.java:382)
    Suppressed: java.io.IOException: Server returned HTTP response code: 503 for URL: http://localhost:8080/cli
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
        at hudson.cli.FullDuplexHttpStream.<init>(FullDuplexHttpStream.java:78)
        at hudson.cli.CLI.connectViaHttp(CLI.java:156)
        at hudson.cli.CLI.<init>(CLI.java:130)
        ... 3 more

FATAL: all hosts have already failed -- aborting

Resolution

To resolve this just run vagrant provision again and it will should resolve the issue


Issue: Ansible Provisioning with AWS fails with SSH Exception

When provisioning the Jenkins environment with AWS for the first time, the provisioning will fail for SSH reasons. This can be resolved by getting the Public IP Address of the created instance in your EC2 Console and replacing the Public IP Address within the /provisioners/ansible/ansible.host file.

Exception

GATHERING FACTS ***************************************************************
fatal: [127.0.0.1] => SSH encountered an unknown error during the connection.
We recommend you re-run the command using -vvvv, which will enable SSH
debugging output to help diagnose the issue

Resolution

Current File ( /provisioners/ansible/ansible.host )

[jenkins]
127.0.0.1 ansible_ssh_port=2222

Eg. of what to be updated to:

[jenkins]
54.69.58.64

then run:

$ vagrant provision

This will now provision the machine properly (given the IP address you put is correct).

NB. Note that the machine is already running and we do not need to run vagrant up again. Thus running vagrant provision will work just fine

License

The MIT License

Copyright (c) 2014. https://github.com/medullan/jenkins-docker-vagrant-ansible

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.