CloudBees Jenkins

Viktor Farcic


Viktor Farcic


  • I speak fast
  • I do things even faster
  • Stop me if you get lost
  • It's OK to ask questions at any time
  • You will not learn how to do something
  • ... but you will learn what can be done
  • Everything is online
  • I will make a mistake. Don't be judgemental!


  • Continuous Integration
  • Continuous Delivery
  • Continuous Deployment
  • Orchestrator
  • Awesome


Can I set up a fully functioning production cluster with two Jenkins services registered in CJOC?

... and create a continuous deployment flow executing on each commit?

... and explain the process?

... and be over with it in 45 minutes?


  • Create a Swarm cluster
  • Create production services
  • Create CJOC service
  • Create two Jenkins master services
  • Register Jenkins masters in CJOC
  • Install plugins across all masters
  • Create Jenkins agents

45 minutes?


A central place to manage all masters

... because there might be too many

Hands-On Time

Setting Up CJOC and masters

Creating a Cluster

git clone



eval $(docker-machine env swarm-1)

docker node ls

docker service ls

curl -i $(docker-machine ip swarm-1)/demo/hello

Creating CJOC Service

docker network create --driver overlay jenkins

mkdir -p docker/jenkins-oc

docker service create --name jenkins-oc \
  --network jenkins -p 8090:8080 -p 50000:50000 \
  --constraint 'node.labels.env == prod' --reserve-memory 300m \
  --mount "type=bind,source=$PWD/docker/jenkins-oc,target=/var/jenkins_home" \

docker service ps jenkins-oc

Creating Jenkins Services

for i in 1 2; do
  mkdir -p docker/jenkins-$i

  docker service create --name jenkins-$i \
    --network jenkins -p 808$i:8080 -p 5000$i:50000 \
    --constraint 'node.labels.env == prod' --reserve-memory 300m \
    --mount "type=bind,source=$PWD/docker/jenkins-$i,target=/var/jenkins_home" \

docker service ps jenkins-1

docker service ps jenkins-2

Setting Up CJOC

open http://$(docker-machine ip swarm-1):8090

cat docker/jenkins-oc/secrets/initialAdminPassword # CJOC password

cat license-key.secret

cat license-cert.secret

# Use admin/admin as user/pass

# Choose "Install Suggested Plugins"

Setting Up CJOC

# Create a new "Client Master" job "jenkins-1"

echo http://$(docker-machine ip swarm-1):8081 # Master address

cat docker/jenkins-1/secrets/initialAdminPassword # Master password

# Install suggested plugins

# Create a new "Client Master" job "jenkins-2"

echo http://$(docker-machine ip swarm-1):8082

cat docker/jenkins-2/secrets/initialAdminPassword # Master password

# Install suggested plugins

installing Plugins

  • Create new Cluster Operations job
  • Add Operation > Masters
  • Check From Operations Center Root
  • Add Step > Install Plugin
  • Plugin ID = swarm
  • Version = 2.2
  • Save > Run

Creating Agents

docker-machine ssh swarm-4

sudo mkdir -p /workspace && sudo chmod 777 /workspace && exit

docker-machine ssh swarm-5

sudo mkdir -p /workspace && sudo chmod 777 /workspace && exit

export USER=admin

export PASSWORD=admin

Creating Agents

docker service create --name jenkins-agent \
  --mode global --constraint 'node.labels.env == test' \
  --network jenkins \
  -e COMMAND_OPTIONS="-master http://jenkins-1:8080 -username $USER -password $PASSWORD -labels 'docker' -executors 5" \
  --mount "type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock" \
  --mount "type=bind,source=$HOME/.docker/machine/machines,target=/machines" \
  --mount "type=bind,source=/workspace,target=/workspace" \

open http://$(docker-machine ip swarm-1):8081/computer/

docker service ps jenkins-agent


open http://$(docker-machine ip swarm-1):8081/configure

docker-machine ip swarm-1 # Env. var PROD_ID

# PROD_NAME = swarm-1

Production Services

docker service ps go-demo

Jenkins Pipeline

open http://$(docker-machine ip swarm-1):8081/newJob

# Name it "go-demo"

# Select "Pipeline" as job type

Jenkins Pipeline

node("docker") {

  git ""

  ]) {

    stage "Unit"
    sh "docker-compose run --rm unit"
    sh "docker build -t go-demo ."

    stage "Staging"
    try {
      sh "docker-compose up -d staging-dep"
      sh 'HOST_IP=localhost docker-compose run --rm staging'
    } catch(e) {
      error "Staging failed"
    } finally {
      sh "docker-compose down"

    stage "Publish"
    sh "docker tag go-demo localhost:5000/go-demo:2.${env.BUILD_NUMBER}"
    sh "docker push localhost:5000/go-demo:2.${env.BUILD_NUMBER}"

    stage "Production"
    ]) {
      sh "docker service update --image localhost:5000/go-demo:2.${env.BUILD_NUMBER} go-demo"
	for (i = 0; i <5; i++) {
      sh "HOST_IP=${env.PROD_IP} docker-compose run --rm production"


Production Updated

docker service ps go-demo


Decentralized job definitions

... in code repository

... where they belong

Hands-On Time

Defining jobs outside Jenkins

Creating a Jenkinsfile job



open http://$(docker-machine ip swarm-1):8081/newJob

# Name it "go-demo-mb"

# Select "Multibranch Pipeline" as job type

# Add Source > GitHub

# Owner = vfarcic

# Repository = go-demo

Global Library

To not repeat yourself...

Hands-On Time

Defining global functions

Global Functions

open http://$(docker-machine ip swarm-1):8081/configure

# Global Pipeline Libraries > Add
# Name = my-lib
# Default Version = master
# Legacy SCM > Git
# Repository URL =

open http://$(docker-machine ip swarm-1):8081/job/go-demo-mb/job/global-lib/


In case you need to store stuff...

Hands-On Time

Storing artifacts in Artifactory

Artifactory Service

docker service create --name artifactory \
  --constraint 'node.labels.env == prod' -p 8091:8081 \

docker service ps artifactory

open http://$(docker-machine ip swarm-1):8091/artifactory/webapp/#/admin/repository/local/new

# Use admin/password as user/pass

# Add "Generic" repository called "go-demo"

Artifactory in Jenkins

open http://$(docker-machine ip swarm-1):8081/pluginManager/available

# Install "Artifactory Plugin"

open http://$(docker-machine ip swarm-1):8081/configure

# Artifactory servers > Add

# Server ID = artifactory-1

# Username/Password = admin/password

echo http://$(docker-machine ip swarm-1):8091/artifactory # URL



open http://$(docker-machine ip swarm-1):8081/job/go-demo-mb/job/artifactory/

open http://$(docker-machine ip swarm-1):8091/artifactory/webapp/#/artifacts/browse/tree/General/go-demo


When you have trust issues...

Hands-On Time

Securing Jenkins with RBAC

RBAC Setup

open http://$(docker-machine ip swarm-1):8081/pluginManager/available

# Install the "Mock Security Realm" plugin

open http://$(docker-machine ip swarm-1):8081/configureSecurity

# Select "Enable Security"

# Select "Mock Security Realm"

# Specify "god admin", "admin admin" and "developer devs" as users and groups

# Select "Role-based matrix authorization strategy"

RBAC Setup

open http://$(docker-machine ip swarm-1):8081/plugin/nectar-rbac/manage/
# Add "administrator" role and select all permissions
# Add "developer" role and select "Overall Read", "Job Build & Read" permissions
open http://$(docker-machine ip swarm-1):8081/groups/newGroup
# Type "admins" as "Group name"
# Select "Granted" to "administrator"
# Click "Add user/group" and type "admin"
# Click "Back to Groups" > "New Group"
# Type "developers" as "Group name"
# Select "Granted" to "developer"
# Click "Add user/group" and type "dev"
open http://$(docker-machine ip swarm-1):8081/plugin/nectar-rbac/manage/
# Deselect all permissions to "authenticated"
# Explore logging in as "dev" or "admin"


Things should be organized...

Hands-On Time

Organizing Jenkins jobs through folders

Folders Setup

open http://$(docker-machine ip swarm-1):8081

# Click "New Item"

# Name if "my-folder" and select "Folder" as type

open http://$(docker-machine ip swarm-1):8081/job/go-demo/

# Click "Move/Copy/Promote", write "/my-folder" as "Destination", click "Move"


Because you have trust issues...

Hands-On Time

Waiting for a confirmation

Input step in a pipeline



To stop you from doing something else...

Hands-On Time

Sending email on buil failure

SMTP Service

docker service create --name postfix --network jenkins -p 25:25 \
    -e -e smtp_user=admin:admin \

open http://$(docker-machine ip swarm-1):8081/configure

# SMTP server = postfix
# User Name / Password = admin/admin
# SMTP Port = 25


open http://$(docker-machine ip swarm-1):8081/job/go-demo-mb/job/notif/

Jenkins Best Practices

  • Always secure Jenkins
  • Don't build on the master
  • Backup Jenkins Home regularly
  • Setup Jenkins on a fast hard disk
  • Cleanup (remove) old builds
  • Create a single Pipeline job for a project
  • Fail fast
  • Fixing a broken build has the highest priority
  • DRY, define pipeline shared libraries
  • Do not pull SCM, use web hooks
  • Use Jenkinsfile

Pipeline Best Practices

  • Treat pipelines as code
  • Always work inside stages
  • Do all material work inside nodes
  • Run steps in parallel
  • Define nodes inside parallel steps
  • Don't use input inside a node
  • Wrap inputs with timeouts
  • Use withEnv instead global env. vars
  • Use stash/unstash to share files between nodes

Viktor Farcic





docker-machine rm -f swarm-1 swarm-2 swarm-3 swarm-4 swarm-5

rm -rf docker go-demo