Jenkins Best Practices

Jenkins pipelines have undergone significant evolution to meet the demands of modern software development. Initially relying on freestyle projects, Jenkins introduced pipelines to enable more complex and flexible workflows. Over time, two primary flavors of pipelines have emerged: declarative and scripted. Let's explore their differences and use cases.

Introduction

Jenkins Pipeline is a suite of plugins that supports implementing and integrating continuous delivery pipelines into Jenkins.

A continuous delivery (CD) pipeline is an automated expression of your process for getting software from version control to your users and customers. Every change to your software (committed in source control) goes through a complex process on its way to being released. This process involves building the software in a reliable and repeatable manner, as well as progressing the built software (called a "build") through multiple stages of testing and deployment.
Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines "as code" via the Pipeline domain-specific language (DSL) syntax.
The definition of a Jenkins Pipeline is written into a text file (called a Jenkinsfile) which in turn can be committed to a project’s source control repository. This is the foundation of "Pipeline-as-code"; treating the CD pipeline as a part of the application to be versioned and reviewed like any other code.
While the syntax for defining a Pipeline, either in the web UI or with a Jenkinsfile is the same, it is generally considered best practice to determine the Pipeline in a Jenkinsfile and check that into source control.

Creating a Jenkinsfile and committing it to source control provides several immediate benefits:

• Automatically creates a Pipeline build process for all branches and pull requests.
• Code review/iteration on the Pipeline (along with the remaining source code).
• Audit trail for the Pipeline.
• Single source of truth for the Pipeline, which can be viewed and edited by multiple project members.
• Code: Pipelines are implemented in code and typically checked into source control, giving teams the ability to edit, review, and iterate upon their delivery pipeline.
• Durable: Pipelines can survive both planned and unplanned restarts of the Jenkins controller.
• Pausable: Pipelines can optionally stop and wait for human input or approval before continuing the Pipeline run.
• Versatile: Pipelines support complex real-world CD requirements, including the ability to fork/join, loop, and perform work in parallel.
• Extensible: The Pipeline plugin supports custom extensions to its DSL and multiple options for integration with other plugins.

Pipeline concepts

• Pipeline: A Pipeline is a user-defined model of a CD pipeline. A Pipeline’s code defines your entire build process, which typically includes stages for building an application, testing it, and then delivering it. Also, a pipeline block is a key part of the Declarative Pipeline syntax.
• Agent: Agent is Declarative Pipeline-specific syntax that instructs Jenkins to allocate an executor (on a node) and workspace for the entire Pipeline.
• Node: A node is a machine that is part of the Jenkins environment and is capable of executing a Pipeline. Also, a node block is a key part of Scripted Pipeline syntax.
• Stage: A stage block defines a conceptually distinct subset of tasks performed through the entire Pipeline (e.g. "Build", "Test" and "Deploy" stages), which is used by many plugins to visualize or present Jenkins Pipeline status/progress.
• Step: A single task. Fundamentally, a step tells Jenkins what to do at a particular point in time (or "step" in the process). For example, to execute the shell command make, use the sh step: sh 'make'. When a plugin extends the Pipeline DSL, that typically means the plugin has implemented a new step.
• Sh: sh is a Pipeline step (provided by the Pipeline: Nodes and Processes plugin) that executes the given shell command.
• Junit: Junit is another Pipeline step (provided by the JUnit plugin) for aggregating test reports.

Declarative Pipeline

In Declarative Pipeline syntax, the pipeline block defines all the work done throughout your entire Pipeline.
Declarative pipelines in Jenkins provide a simplified and opinionated syntax for defining pipelines. The emphasis is on readability and ease of use, making them accessible to developers and non-developers alike. Declarative pipelines aim to abstract away the intricacies of the underlying structure, providing a high-level view of the pipeline flow. They are designed for simplicity and well-suited for projects with straightforward build and deployment requirements.

1. Declarative Pipelines:

   • Structured and Opinionated: Declarative pipelines provide a structured and opinionated syntax that promotes best practices and readability. They are suitable for most pipeline scenarios.
   • Simpler Syntax: Declarative pipelines are easy to write and understand. The main block contains the entire pipeline declaration, including sections for defining the agent, stages, and steps.
   • Directives: Declarative pipelines use directives to include additional logic, such as setting triggers, environment variables, and parameters.
   • Example:

    pipeline {
        agent any
        stages {
            stage('Hello World') {
                steps {
                    sh 'echo Hello World'
                }
            }
        }
    }
   

• Benefits: Faster entry for beginners and convenient integration with scripted pipelines using the script directive.

Example:

Jenkinsfile (Declarative Pipeline)
pipeline { 
    agent any 
    options {
        skipStagesAfterUnstable()
    }
    stages {
        stage('Build') { 
            steps { 
                sh 'make' 
            }
        }
        stage('Test'){
            steps {
                sh 'make check'
                junit 'reports/*.xml' 
            }
        }
        stage('Deploy') {
            steps {
                sh 'make publish' 
            }
        }
    }
}

Scripted pipelines

Scripted pipelines offer a more flexible and powerful approach, leveraging the Groovy scripting language. This gives users extensive control over the pipeline logic and the ability to implement complex build and deployment processes. Unlike declarative pipelines, scripted pipelines are more expressive and allow for fine-grained control over every aspect of the pipeline. They are ideal for projects with intricate workflows and advanced automation requirements.

1. Scripted Pipelines:

   • Full Flexibility: Scripted pipelines, built with Groovy, offer full flexibility and extensibility. They allow complex logic and customizations.
   • Groovy Knowledge Required: Scripted pipelines require basic knowledge of Groovy.
   • Basic Blocks: Scripted pipelines have only two basic blocks: node (specifies the execution machine) and stage (groups steps).
   • Example:

    node {
        stage('Hello Jenkins') {
            echo 'Hello Jenkins'
        }
    }
   

• Benefits: Ideal for experienced users who need advanced features and customizations.

Advantages of Jenkins Declarative Pipeline

• Readability and conciseness: Declarative pipelines boast a concise and easily readable syntax, enhancing accessibility for users with varying levels of expertise. The clear structure promotes collaboration within the team, fostering a shared understanding of the pipeline.
• Opinionated structure: Following a predefined structure, declarative pipelines enforce best practices. This opinionated approach simplifies pipeline creation by guiding users to organize workflows in a standardized manner, ensuring consistency across projects.
• Ease of adoption: Declarative pipelines are especially well-suited for teams new to Jenkins or CI/CD. Their straightforward syntax and abstraction of complexity make them an excellent starting point, allowing teams to get acquainted with pipeline concepts gradually.
• Limited scripting complexity: By design, declarative pipelines restrict the use of complex scripting. This limitation encourages a straightforward, structured approach to pipeline development, ideal for projects requiring less intricate automation.
• Built-in step catalog: Declarative pipelines come with a built-in step catalog, providing a convenient way to incorporate standard functionality without extensive scripting. This simplifies the addition of common steps like source code checkout and artifact archiving.
• Enhanced security: Declarative pipelines support a more controlled execution environment. With restricted scripting capabilities, they provide an additional layer of security by minimizing the risk of arbitrary code execution, making them suitable for projects with stringent security requirements.

Disadvantages of Jenkins declarative pipeline

• Less flexibility: The opinionated nature of declarative pipelines may limit flexibility for projects with unique or complex requirements. Advanced customization may be challenging to achieve within the confines of the declarative syntax, potentially leading to workarounds.
• Learning curve for advanced features: While declarative pipelines are easy, mastering advanced features may require transitioning to scripted pipelines. Teams with evolving requirements might need more support as they outgrow the simplicity of the declarative syntax.
• Limited expressiveness: While clear and concise, the declarative syntax might need more expressive power for highly customized or intricate CI/CD workflows. Teams with complex automation needs may find themselves constrained.

Advantages of Jenkins scripted pipeline

• Unmatched flexibility: Scripted pipelines offer unparalleled flexibility, providing developers with complete programmatic control. This enables the implementation of complex logic, conditional statements, and advanced automation workflows tailored to specific project needs.
• Reusability of code: Scripted pipelines support the creation of reusable code snippets and functions. This modularity enhances code organization and allows for the sharing of standard pipeline components across different projects, promoting a more efficient development process.
• Advanced logic and customization: Scripted pipelines shine for projects with intricate workflows or specific automation requirements. They allow the implementation of advanced programming constructs, custom functions, and conditional execution, providing a high level of customization.
• Parallel execution and optimization: Scripted pipelines excel in parallel execution of tasks, optimizing the overall pipeline runtime. This feature benefits projects with extensive testing suites or complex build processes, significantly reducing overall turnaround time.
• Dynamic parameterization: Scripted pipelines support dynamic parameterization, enabling the adjustment of pipeline behavior based on runtime conditions. This flexibility is advantageous for projects requiring adaptive and dynamic automation workflows.

Disadvantages of Jenkins scripted pipeline

• Steeper learning curve: Scripted pipelines, leveraging Groovy scripting, come with a steeper learning curve than declarative pipelines. Team members with limited scripting experience may need help to grasp and contribute, necessitating additional training.
• Increased complexity and maintenance: The flexibility of scripted pipelines comes at the cost of increased complexity. Managing and maintaining intricate scripts may require higher expertise, and readability could be compromised without proper documentation.
• Potential for code duplication: Scripted pipelines may duplicate code across projects without careful organization and modular design. This can impact maintainability and make enforcing consistent practices with diligent code management easier.
• Dependency on external libraries: Scripted pipelines might rely on external libraries or plugins, introducing dependencies that must be managed. This can complicate versioning and updates, potentially leading to compatibility issues.

In summary, declarative pipelines provide simplicity and opinionated syntax, while scripted pipelines offer more power and flexibility. Leveraging declarative pipelines with script directive

Ref: https://www.jenkins.io/doc/book/pipeline/syntax/