2. Jenkins

../_images/devops-continuous.png

Fig. 2.3. Continuous Integration -> Continuous Delivery -> Continuous Deployment

2.1. Jenkins in Devtools Ecosystem

../_images/ecosystem-big-picture-01.png

Fig. 2.4. Ecosystem Big Picture

2.2. Architecture

  • Local executors (default: 2)
  • Remote workers via SSH and labels
  • Docker build
  • New UI (Blue Ocean) currently accessible as a plugin, but soon to be default
  • Jenkins uses Groovy scripts in Jenkinsfile in repository main directory

2.3. Administration

../_images/geek-and-poke-jenkins-down-and-up.jpg

Fig. 2.5. Jenkins down… and up again.

2.3.1. User Management

  • Always use LDAP (OpenLDAP or Active Directory)
  • name groups as jenkins-users or jenkins-administrators
  • local administrator jenkins-administrator only for fixing bugs with LDAP
  • use jenkins@example.com (for easy email fiterling)
  • use jenkins.example.com as domain name with Firewall blocking external access

2.3.2. Plugin installation

  • Dependencies hell
  • Plugin support (especially those free ones)
  • Open Source plugins
  • Plugin and upgrades
  • Once given out, cannot be easily taken away

2.3.3. Build Triggers

../_images/geek-and-poke-development-driven-tests.jpg

Fig. 2.6. Development Driven Tests

Code Listing 2.1. build trigger via Jenkins API
curl -X POST http://localhost:8080/job/JOB_NAME/build \
    --user USER:TOKEN \
    --data-urlencode json='{
        "parameter": [
            {"name":"id", "value":"123"},
            {"name":"verbosity", "value":"high"}
        ]}'

2.4. Notifications

  • Email
  • Slack / HipChat
  • IRC

2.5. SonarScanner

If your code is in other version:

Code Listing 2.2. Minimal Sonar Project Properties
# Required metadata
sonar.projectKey=MyProject
sonar.projectName=MyProject
sonar.projectVersion=1.0

sonar.sources=src/main/java
sonar.java.binaries=target/classes

# java version used by source files:
sonar.java.source=7
Code Listing 2.3. Extra Sonar Project Properties
sonar.host.url=https://sonarcloud.io
sonar.organization=astromatt
sonar.login=...

sonar.language=py
sonar.sourceEncoding=UTF-8
sonar.verbose=true

sonar.projectKey=habitatOS
sonar.projectName=habitatOS
sonar.projectDescription=Operating System for analog extraterrestrial habitats.
sonar.links.homepage=https://github.com/AstroMatt/habitatOS
sonar.links.scm=https://github.com/AstroMatt/habitatOS
sonar.links.issue=https://github.com/AstroMatt/habitatOS/issues
sonar.links.ci=https://www.travis-ci.org/AstroMatt/HabitatOS

sonar.projectBaseDir=habitat
sonar.sources=.
sonar.exclusions=**/migrations/**

# Pylint
sonar.python.pylint=/usr/local/bin/pylint
sonar.python.pylint_config=.pylintrc
sonar.python.pylint.reportPath=pylint-report.txt

# Unit tests
sonar.python.xunit.reportPath=test-reports/*.xml
sonar.python.coverage.reportPath=coverage.xml

# Integration tests
sonar.python.coverage.itReportPath=it-coverage.xml

# Turn off these rules
sonar.issue.ignore.multicriteria=e1,e2
# python:s100: "Method names should comply with a naming convention" gives many false positives when overriding
# TestCase methods (such as setUp and tearDown) in test files.
sonar.issue.ignore.multicriteria.e1.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e1.resourceKey=**/tests.py
sonar.issue.ignore.multicriteria.e2.ruleKey=python:S100
sonar.issue.ignore.multicriteria.e2.resourceKey=**/tests.py

2.6. Large repos

  • is a sign of git missuse, and should be tackled with GIT LFS
  • Use command line git rather than jGit
  • command line git handles memory better
  • Use reference repository (bare)
  • Shallow clone (GIT from 1.9+ can push from shallow clones)
  • Don’t fetch tags
  • Narrow refspec - only clone specific branches (honor refspec on initial clone)
  • Pipeline stash / unstash (sparse checkout on master node, and then stash checkout, and unstash on remotes)
  • Sparse checkout (Subset of working tree - single directory [exclude or include on per file basis])

2.7. Blue Ocean

  • New UI
  • Interoperable with old UI
  • Accessible at /blue/ in the URL after “Blue Ocean” plugin installation.
  • Pipeline editor

2.8. Environment Variables

Variable Description
BUILD_NUMBER The current build number, such as ‘153’
BUILD_ID The current build id, such as 2005-08-22_23-59-59 (YYYY-MM-DD_hh-mm-ss, defunct since version 1.597)
BUILD_URL The URL where the results of this build can be found (e.g. http://localhost:8080/job/MyJobName/1337/)
NODE_NAME The name of the node the current build is running on. Equals master for master node.
JOB_NAME Name of the project of this build. This is the name you gave your job when you first set it up. It’s the third column of the Jenkins Dashboard main page.
BUILD_TAG String of jenkins-${JOB_NAME}-${BUILD_NUMBER}. Convenient to put into a resource file, a jar file, etc for easier identification.
JENKINS_URL Set to the URL of the Jenkins master that’s running the build. This value is used by Jenkins CLI for example
EXECUTOR_NUMBER The unique number that identifies the current executor (among executors of the same machine) that’s carrying out this build. This is the number you see in the ‘build executor status’, except that the number starts from 0, not 1.
JAVA_HOME If your job is configured to use a specific JDK, this variable is set to the JAVA_HOME of the specified JDK. When this variable is set, PATH is also updated to have $JAVA_HOME/bin.
WORKSPACE The absolute path of the workspace.
SVN_REVISION For Subversion-based projects, this variable contains the revision number of the module. If you have more than one module specified, this won’t be set.
CVS_BRANCH For CVS-based projects, this variable contains the branch of the module. If CVS is configured to check out the trunk, this environment variable will not be set.
GIT_COMMIT For Git-based projects, this variable contains the Git hash of the commit checked out for the build (like ce9a3c1404e8c91be604088670e93434c4253f03) (all the GIT_* variables require git plugin)
GIT_URL For Git-based projects, this variable contains the Git url (like git@github.com:user/repo.git or https://github.com/user/repo.git)
GIT_BRANCH For Git-based projects, this variable contains the Git branch that was checked out for the build (normally origin/master)

2.9. Groovy syntax

Code Listing 2.4. Variable
// Variable declaration
String x
def o


// You can also use implicit declaration
x = 1
println x

x = new java.util.Date()
println x

x = -3.1499392
println x

x = false
println x

x = "Hi"
println x

def (a, b, c) = [10, 20, 'foo']

def nums = [1, 3, 5]
def a, b, c
(a, b, c) = nums


// String literals
def username = 'Jenkins'
echo 'Hello Mr. ${username}'
echo "I said, Hello Mr. ${username}"


// Multi line strings
def viewspec = '''
    //depot/Tools/build/... //jryan_car/Tools/build/...
    //depot/commonlibraries/utils/... //jryan_car/commonlibraries/utils/...
    //depot/helloworld/... //jryan_car/helloworld/...
'''
Code Listing 2.5. Conditional
def x = false
def y = false

if ( !x ) {
    x = true
}

assert x == true

if ( x ) {
    x = false
} else {
    y = true
}

assert x == y
Code Listing 2.6. Control structure
def x = 1.23
def result = ""

switch ( x ) {
    case "foo":
        result = "found foo"
        // lets fall through

    case "bar":
        result += "bar"

    case [4, 5, 6, 'inList']:
        result = "list"
        break

    case 12..30:
        result = "range"
        break

    case Integer:
        result = "integer"
        break

    case Number:
        result = "number"
        break

    case ~/fo*/: // toString() representation of x matches the pattern?
        result = "foo regex"
        break

    case { it < 0 }: // or { x < 0 }
        result = "negative"
        break

    default:
        result = "default"
}
Code Listing 2.7. Function
// Optional ``return``
def jobName = 'example'

job(jobName) {

}



// Variable function parameter length

def concat1 = { String... args -> args.join('') }
assert concat1('abc','def') == 'abcdef'

def concat2 = { String[] args -> args.join('') }
assert concat2('abc', 'def') == 'abcdef'

def multiConcat = { int n, String... args ->
    args.join('')*n
}
assert multiConcat(2, 'abc','def') == 'abcdefabcdef'
Code Listing 2.8. Class
// Simple class declaration
class Person {
    String name
    int age
    def fetchAge = { age }
}

def p = new Person(name:'Jessica', age:42)



class Person {
    String name
    String toString() { name }
}
def sam = new Person(name:'Sam')

// Create a GString with lazy evaluation of "sam"
def gs = "Name: ${-> sam}"
Code Listing 2.9. Loop
String message = ''

for (int i = 0; i < 5; i++) {
    message += 'Hi '
}

assert message == 'Hi Hi Hi Hi Hi '
Code Listing 2.10. Import
package utilities

class MyUtilities {
    static void addMyFeature(def job) {
        job.with {
            description('Arbitrary feature')
        }
    }
}



import utilities.MyUtilities

def myJob = job('example')
MyUtilities.addMyFeature(myJob)
Code Listing 2.11. Exception
try {
    'moo'.toLong()   // this will generate an exception
    assert false     // asserting that this point should never be reached
} catch ( e ) {
    assert e in NumberFormatException
}
Code Listing 2.12. Rest API HTTP queries
def project = 'Netflix/asgard'
def branchApi = new URL("https://api.github.com/repos/${project}/branches")
def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader())

branches.each {
    def branchName = it.name
    def jobName = "${project}-${branchName}".replaceAll('/','-')

    job(jobName) {
        scm {
            git("https://github.com/${project}.git", branchName)
        }
    }
}

2.10. Jenkinsfile - Pipeline model definition

../_images/ecosystem-jenkins-pipeline.png

Fig. 2.7. Pipeline model definition plugin

Sample Jenkinsfile:

Code Listing 2.13. Simple
pipeline {
    agent any

    stages {
        stage("simple") {
            steps {
                // Put your code here
            }
        }
    }
}
Code Listing 2.14. Example
pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                echo 'Building..'
            }
        }

        stage('Test') {
            steps {
                echo 'Testing..'
            }
        }

        stage('Deploy') {
            steps {
                echo 'Deploying....'
            }
        }
    }
}
Code Listing 2.15. Test
pipeline {
    agent any

    stages {
        stage('Test') {
            steps {
                /* `make check` returns non-zero on test failures,
                * using `true` to allow the Pipeline to continue nonetheless
                */
                sh 'make check || true'
                junit '**/target/*.xml'
            }
        }
    }
}
Code Listing 2.16. Build
pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'make'
                archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
            }
        }
    }
}
Code Listing 2.17. Deploy
pipeline {
    agent any

    stages {
        stage('Deploy') {
            when { currentBuild.result == 'SUCCESS' }
            steps {
                sh 'make publish'
            }
        }
    }
}
Code Listing 2.18. Environment
pipeline {
    agent any

    stages {
        stage('Example') {
            steps {
                echo "Running ${env.BUILD_ID} on ${env.JENKINS_URL}"
            }
        }
    }
}


// Environment variables can be overriden and declared locally
pipeline {
    agent any

    environment {
        FOO = "BAZ"
        AWS_ACCESS_KEY_ID     = credentials('AWS_ACCESS_KEY_ID')
        AWS_SECRET_ACCESS_KEY = credentials('AWS_SECRET_ACCESS_KEY')
    }

    stages {
        stage("baz") {
            steps {
                sh 'echo "FOO is $FOO"'
            }
        }

        stage("bar") {
            environment {
                FOO = "BAR"
            }

            steps {
                sh 'echo "FOO is $FOO"'
            }
        }
    }
}
Code Listing 2.19. Parameters
pipeline {
    agent any

    parameters {
        booleanParam(defaultValue: true, description: '', name: 'flag')

        // soon to be changed to stringParam
        string(name: 'Greeting', defaultValue: 'Hello', description: 'How should I greet the world?')
        string(name: 'PERSON', defaultValue: 'Mr Jenkins', description: 'Who should I say hello to?')
    }

    stages {
        stage('Example') {
            steps {
                echo "${Greeting} World!"
                echo "Hello ${params.PERSON}"
            }
        }
    }
}
Code Listing 2.20. Agent
/* It is required whithin the ``pipeline {...}``
 * At the begining of pipeline directive:
 * - ``agent any``
 * - ``agent none``
 * - ``agent label:'some-label'``
 * - ``agent docker:"python:3.6.3", dockerArgs:"-v /tmp:/tmp -p 80:80"``
 * - ``agent dockerfile:true, dockerArgs:"-v /tmp:/tmp -p 80:80"`` ## Dockerfile in root of your repo
 * - ``agent dockerfile:"SomeOtherDockerfile", dockerArgs:"-v /tmp:/tmp -p 80:80"``
 */

pipeline {
    agent none
    stages {
        stage('Build') {
            agent any
            steps {
                checkout scm
                sh 'make'
                stash includes: '**/target/*.jar', name: 'app'
            }
        }

        stage('Test on Linux') {
            agent {
                label 'linux'
            }
            steps {
                unstash 'app'
                sh 'make check'
            }
            post {
                always {
                    junit '**/target/*.xml'
                }
            }
        }

        stage('Test on Windows') {
            agent {
                label 'windows'
            }
            steps {
                unstash 'app'
                bat 'make check'
            }
            post {
                always {
                    junit '**/target/*.xml'
                }
            }
        }
    }
}
Code Listing 2.21. Parallel
pipeline {
    agent any

    stage('Test') {
        parallel(
            windows: {
                echo "Windows branch"
            },

            linux: {
                echo "Linux branch"
            },

            macOS: {
                echo "macOS branch"
            }
        )
    }
}


// If you use parallel inside your steps block, you cannot have anything besides that
pipeline {
    agent any
    stages {
        stage('Non-Parallel Stage') {
            steps {
                echo 'This stage will be executed first.'
            }
        }

        stage('Parallel Stage') {
            failFast true

            when {
                branch 'master'
            }

            parallel {
                stage('Branch A') {
                    agent {
                        label "for-branch-a"
                    }

                    steps {
                        echo "On Branch A"
                    }
                }

                stage('Branch B') {
                    agent {
                        label "for-branch-b"
                    }

                    steps {
                        echo "On Branch B"
                    }
                }
            }
        }
    }
}


// Jobs In Parallel
// in this array we'll place the jobs that we wish to run
def branches = [:]

//running the job 4 times concurrently
//the dummy parameter is for preventing mutation of the parameter before the execution of the closure.
//we have to assign it outside the closure or it will run the job multiple times with the same parameter "4"
//and jenkins will unite them into a single run of the job

for (int i = 0; i < 4; i++) {
  def index = i //if we tried to use i below, it would equal 4 in each job execution.
  branches["branch${i}"] = {
//Parameters:
//param1 : an example string parameter for the triggered job.
//dummy: a parameter used to prevent triggering the job with the same parameters value.
//       this parameter has to accept a different value each time the job is triggered.
    build job: 'freestyle', parameters: [
      string(name: 'param1', value:'test_param'),
      string(name:'dummy', value: "${index}")]
  }
}
parallel branches



// Parallel Multiple Nodes
def labels = ['precise', 'trusty'] // labels for Jenkins node types we will build on
def builders = [:]

for (x in labels) {
    def label = x // Need to bind the label variable before the closure - can't do 'for (label in labels)'

    // Create a map to pass in to the 'parallel' step so we can fire all the builds at once
    builders[label] = {
      node(label) {
        // build steps that should happen on all nodes go here
      }
    }
}

parallel builders
Code Listing 2.22. Option
pipeline {
    agent any

    options {
        timeout(time: 1, unit: 'HOURS')
        // Prepend all console output generated by the Pipeline run with the time at which the line was emitted
        timestamps()

        /*
         * Parameters for logRotator
         * - daysToKeepStr: history is only kept up to this days.
         * - numToKeepStr: only this number of build logs are kept.
         * - artifactDaysToKeepStr: artifacts are only kept up to this days.
         * - artifactNumToKeepStr: only this number of builds have their artifacts kept.
         */
        buildDiscarder(logRotator(numToKeepStr: '30', artifactNumToKeepStr: '30'))
    }

    stages {
        stage('Example') {
            steps {
                echo 'Hello World'
            }
        }
    }
}
Code Listing 2.23. Tool
/*
* Requirements
* Do not work with docker
* If you put invalid, it will list valid
* Automaticly installs requirements
*/

pipeline {
    agent any

    tools {
        maven "apache-maven-3.1.0"
        jdk "default"
    }

    stages {
        stage('Test') {

        }
    }
}
Code Listing 2.24. Timeout
pipeline {
    agent any
    stages {
        stage('Deploy') {
            steps {
                retry(3) {
                    sh './flakey-deploy.sh'
                }

                timeout(time: 3, unit: 'MINUTES') {
                    sh './health-check.sh'
                }
            }
        }
    }
}
Code Listing 2.25. Input
pipeline {
    agent {
        docker {
            image 'node:6-alpine'
            args '-p 3000:3000 -p 5000:5000'
        }
    }
    environment {
        CI = 'true'
    }
    stages {
        stage('Build') {
            steps {
                sh 'npm install'
            }
        }
        stage('Test') {
            steps {
                sh './jenkins/scripts/test.sh'
            }
        }
        stage('Deliver for development') {
            when {
                branch 'development'
            }
            steps {
                sh './jenkins/scripts/deliver-for-development.sh'
                input message: 'Finished using the web site? (Click "Proceed" to continue)'
                sh './jenkins/scripts/kill.sh'
            }
        }
        stage('Deploy for production') {
            when {
                branch 'production'
            }
            steps {
                sh './jenkins/scripts/deploy-for-production.sh'
                input message: 'Finished using the web site? (Click "Proceed" to continue)'
                sh './jenkins/scripts/kill.sh'
            }
        }
    }
}
Code Listing 2.26. Artifact
// This shows a simple example of how to archive the build output artifacts.
node {
    stage "Create build output"

    // Make the output directory.
    sh "mkdir -p output"

    // Write an useful file, which is needed to be archived.
    writeFile file: "output/usefulfile.txt", text: "This file is useful, need to archive it."

    // Write an useless file, which is not needed to be archived.
    writeFile file: "output/uselessfile.md", text: "This file is useless, no need to archive it."

    stage "Archive build output"

    // Archive the build output artifacts.
    archiveArtifacts artifacts: 'output/*.txt', excludes: 'output/*.md'
}

2.10.1. Post Actions

At the end of pipeline directive:

always:Run the steps in the post section regardless of the completion status of the Pipeline’s or stage’s run.
changed:Only run the steps in post if the current Pipeline’s or stage’s run has a different completion status from its previous run.
failure:Only run the steps in post if the current Pipeline’s or stage’s run has a “failed” status, typically denoted by red in the web UI.
success:Only run the steps in post if the current Pipeline’s or stage’s run has a “success” status, typically denoted by blue or green in the web UI.
unstable:Only run the steps in post if the current Pipeline’s or stage’s run has an “unstable” status, usually caused by test failures, code violations, etc. This is typically denoted by yellow in the web UI.
aborted:Only run the steps in post if the current Pipeline’s or stage’s run has an “aborted” status, usually due to the Pipeline being manually aborted. This is typically denoted by gray in the web UI
Code Listing 2.27. Post
pipeline {
    agent any

    stages {
        stage('Test') {
            steps {
                sh 'make check'
            }
        }
    }

    post {

        // evaluated first
        always {
            echo "Done."

            // Lets assume the step was ``sh './gradlew build'``
            archive 'build/libs/**/*.jar'
            junit 'build/reports/**/*.xml'
            deleteDir() /* clean up our workspace */
        }

        sucess {
            echo "Sucess. Will now deploy."
            slackSend channel: '#ops-room',
                      color: 'good',
                      message: "The pipeline ${currentBuild.fullDisplayName} completed successfully."
        }

        failure {
            echo "Failure. Will cleanup."
            mail to: 'team@example.com',
                 subject: "Failed Pipeline: ${currentBuild.fullDisplayName}",
                 body: "Something is wrong with ${env.BUILD_URL}"
        }

        unstable {
            echo 'I am unstable :/'
            hipchatSend message: "Attention @here ${env.JOB_NAME} #${env.BUILD_NUMBER} has failed.",
                        color: 'RED'
        }

        changed {
            echo 'Things were different before...'
        }
    }
}

2.10.2. Triggers

cron:Accepts a cron-style string to define a regular interval at which the Pipeline should be re-triggered, for example: triggers { cron('H */4 * * 1-5') }
pollSCM:Accepts a cron-style string to define a regular interval at which Jenkins should check for new source changes. If new changes exist, the Pipeline will be re-triggered. For example: triggers { pollSCM('H */4 * * 1-5') } Available since Jenkins 2.22
upstream:Accepts a comma separated string of jobs and a threshold. When any job in the string finishes with the minimum threshold, the Pipeline will be re-triggered. For example: triggers { upstream(upstreamProjects: 'job1,job2', threshold: hudson.model.Result.SUCCESS) }
Code Listing 2.28. Trigger
pipeline {
    agent any

    triggers {
        cron('@daily')
    }

    stages {
        stage("name") {

        }
    }
}

2.10.3. When

branch:Execute the stage when the branch being built matches the branch pattern given, for example: when { branch 'master' }. Note that this only works on a multibranch Pipeline.
environment:Execute the stage when the specified environment variable is set to the given value, for example: when { environment name: 'DEPLOY_TO', value: 'production' }
expression:Execute the stage when the specified Groovy expression evaluates to true, for example: when { expression { return params.DEBUG_BUILD } }
not:Execute the stage when the nested condition is false. Must contain one condition. For example: when { not { branch 'master' } }
allOf:Execute the stage when all of the nested conditions are true. Must contain at least one condition. For example: when { allOf { branch 'master'; environment name: 'DEPLOY_TO', value: 'production' } }
anyOf:Execute the stage when at least one of the nested conditions is true. Must contain at least one condition. For example: when { anyOf { branch 'master'; branch 'staging' } }
Code Listing 2.29. When
pipeline {
    agent any
    stages {

        stage('Example Build') {
            when {
                echo 'Should I run?'
                return true
            }

            steps {
                echo 'Hello World'
            }
        }

        stage('Example Deploy') {
            when {
                expression { BRANCH_NAME ==~ /(production|staging)/ }

                anyOf {
                    environment name: 'DEPLOY_TO', value: 'production'
                    environment name: 'DEPLOY_TO', value: 'staging'
                }
            }

            steps {
                echo 'Deploying'
            }
        }
    }
}

2.10.4. Docker

$ docker pull openjdk:7-jdk
$ docker pull openjdk:8-jdk
$ docker pull maven:3-jdk-7
$ docker pull maven:3-jdk-8
$ docker pull golang:1.7
$ docker pull ruby:2.3
$ docker pull python:2
$ docker pull python:3
Code Listing 2.30. Docker
// Docker container declaration
pipeline {
    agent { docker 'python:3.6.3' }

    stages {
        stage('build') {
            steps {
                sh 'python --version'
            }
        }
    }
}

// Verbose declaration
pipeline {
    agent {
        docker {
            image 'maven:3-alpine'
            label 'my-defined-label'
            args  '-v /tmp:/tmp'
        }
    }

    stages {
        stage('build') {
            steps {
                sh 'mvn --version'
            }
        }
    }
}

// Declaring docker container per build
pipeline {
    agent none

    stages {

        stage('Example Build') {
            agent { docker 'maven:3-alpine' }
            steps {
                echo 'Hello, Maven'
                sh 'mvn --version'
            }
        }

        stage('Example Test') {
            agent { docker 'openjdk:8-jre' }
            steps {
                echo 'Hello, JDK'
                sh 'java -version'
            }
        }
    }
}

2.10.5. Maven

Code Listing 2.31. Maven
pipeline {
    agent any

    stages {
        stage("simple") {
            steps {

                // withMaven will discover the generated Maven artifacts, JUnit Surefire & FailSafe & FindBugs reports...
                // Maven installation declared in the Jenkins "Global Tool Configuration"
                withMaven(maven: 'mvn_3.0.4') {
                      sh "mvn clean install"
                 }
            }
        }
    }
}

2.11. Dobre praktyki

  • Skrypt releasowy trzymany w konfiguracji narzędzia
  • Instalacja nadmiarowych pluginów
  • Korzystanie z pluginów zamiast z linii poleceń
  • Przygotowanie środowiska + provisioning
  • Spawnowanie agentów w cloud i czas setupu nowego środowiska
  • Długość buildów
  • Ignorowanie testów ?!
  • Skipowanie testów (verbose)
  • Budowanie Pull Requestów
  • Jak długo trzymać branche?
  • Jak automatycznie czyścić branche?
  • Budowanie na różnych środowiskach
  • Spockframework: https://www.youtube.com/watch?v=64jZVsScbU8
Code Listing 2.32. Spock Framework
// source: http://thejavatar.com/testing-with-spock/

def "should return false if user does not have role required for viewing page"() {
   given:
      // context
      pageRequiresRole Role.ADMIN
      userHasRole Role.USER
   when:
      // some action is performed
      boolean authorized = authorizationService.isUserAuthorizedForPage(user, page)
   then:
      // expect specific result
      authorized == false
}

2.11.1. Extra

Code Listing 2.33. Color
// This shows a simple build wrapper example, using the AnsiColor plugin.
node {

    // This displays colors using the 'xterm' ansi color map.
    ansiColor('xterm') {

        // Just some echoes to show the ANSI color.
        stage "\u001B[31mI'm Red\u001B[0m Now not"

    }
}
Code Listing 2.34. Artifactory
 node {
    git url: 'https://github.com/jfrogdev/project-examples.git'

    // Get Artifactory server instance, defined in the Artifactory Plugin administration page.
    def server = Artifactory.server "SERVER_ID"

    // Read the upload spec and upload files to Artifactory.
    def downloadSpec =
            '''{
            "files": [
                {
                    "pattern": "libs-snapshot-local/*.zip",
                    "target": "dependencies/",
                    "props": "p1=v1;p2=v2"
                }
            ]
        }'''

    def buildInfo1 = server.download spec: downloadSpec

    // Read the upload spec which was downloaded from github.
    def uploadSpec =
            '''{
            "files": [
                {
                    "pattern": "resources/Kermit.*",
                    "target": "libs-snapshot-local",
                    "props": "p1=v1;p2=v2"
                },
                {
                    "pattern": "resources/Frogger.*",
                    "target": "libs-snapshot-local"
                }
            ]
        }'''

    // Upload to Artifactory.
    def buildInfo2 = server.upload spec: uploadSpec

    // Merge the upload and download build-info objects.
    buildInfo1.append buildInfo2

    // Publish the build to Artifactory
    server.publishBuildInfo buildInfo1
}
Code Listing 2.35. Commit Hash from git shell
// These should all be performed at the point where you've
// checked out your sources on the agent. A 'git' executable
// must be available.
// Most typical, if you're not cloning into a sub directory
shortCommit = sh(returnStdout: true, script: "git log -n 1 --pretty=format:'%h'").trim()
  • Jenkins odpalający git bisect i testy dla każdego commita z próby, tak długo aż nie znajdzie problemu

2.12. Build Strategy

../_images/build-strategy.jpg

Fig. 2.8. Build Strategy

../_images/git-flow-whiteboard.jpg

Fig. 2.9. GIT Flow

2.13. Ćwiczenia

2.13.1. Instalacja Jenkinsa i konfuguracja buildów

  • Zainstaluj Jenkins za pomocą paczek DEB przez apt-get

  • Alternatywnie możesz użyć Docker albo manifestów Puppeta

  • Czy wcześniej zainstalowałeś Bitbucket Server?

  • Zacznij budować różne projekty sonar-examples/projects/languages/java:

    • ut - unit tests
    • it - integration tests
  • Ustaw joby przez Jenkinsa

Tip

Bitubcket plugin do Jenkinsa

Pokaż rozwiązanie za pomocą ``apt-get`` na *Ubuntu*

Pokaż rozwiązanie za pomocą ``docker`` na *Ubuntu*

Warning

Sprawdź, czy w swoim pliku Vagrantfile masz skonfigurowany forwardnig portów dla guest:8080 -> host:80

2.13.2. Budowanie Pull Requestów

  • Skonfiguruj ręcznie plan by budował gałęzie GIT Flow

    • Pull Requests
    • feature
    • bugfix
    • master
../_images/git-pull-request-09.jpg

Fig. 2.10. Pull Requests

Pokaż konfigurację dla Bitbucket Server

Pokaż rozwiązanie dla Pull Requestów

Pokaż rozwiązanie dla brancha ``master``

Pokaż rozwiązanie dla brancha ``feature``

Pokaż rozwiązanie dla brancha ``bugfix``

Pokaż plugin, który to zrobi za Ciebie

2.13.3. Budowanie Checkstyle, PMD, JaCoCo, Findbugs i PITest

  • Dla repozytorium sonar-examples
  • Zacznij budować różne projekty sonar-examples/projects/languages/java
  • Wyniki upublicznij w SonarQube
  • Do instalacji możesz wykorzystać puppet module install maestrodev/sonarqube
  • Dodaj w pom.xml zależność pitest i przetestuj projekt wykorzystując domyślne mutatory

2.13.4. Job DSL

  • Przepisz całą konfigurację wykorzustując plik Job DSL

2.13.5. Jenkins Docker Plugin

  • Skonfiguruj zadanie aby uruchamiało kontener
  • Zadanie ma provisionować konfigurację wewnątrz kontenera
  • Zadanie ma uruchamiać build wewnątrz kontenera
  • Zadanie ma niszczyć kontener po buildze

2.13.6. Jenkins i testy wydajnościowe JMeter

  • Przeprowadź test wydajnościowy głównej strony aplikacji uruchomionej na Twoim komputerze (np. SonarQube jeżeli wykonałeś poprzednie ćwiczenie)
  • Test wydajnościowy powinien zapisany w xml oraz uruchamiany bez wykorzystania GUI

Pokaż rozwiązanie JMeter

2.13.7. Selenium

  1. Nagraj test w selenium
  2. Uruchom test przy każdym commicie do brancha feature i bugfix