Jenkins

Jenkins

What is Jenkins?
  • Jenkins is an open-source automation server used to automate software development processes.
  • It facilitates continuous integration and deployment.
  • Jenkins is highly extensible with hundreds of plugins available.
Features Of Jenkins
  1. Easy Installation & Configuration:
      • Jenkins is platform-agnostic and can be installed on popular operating systems like Windows, Unix, and Mac OS.
      • It offers both a normal installer and a .war file for installation, with configuration done through its web interface.
  1. Open-Source:
      • Jenkins is freely available as an open-source tool, backed by a strong community.
      • Users can leverage community support for extensibility, documentation, and other Jenkins-related features.
  1. Thriving Plugin Ecosystem:
      • Over 1500 plugins are available in the Update Center, contributed and tested by the Jenkins community.
      • These plugins enhance Jenkins' functionality, making it adaptable to various development environments and workflows.
  1. Easy Distribution:
      • Jenkins facilitates the distribution of work across multiple machines and platforms.
      • This enables faster build, testing, and deployment processes, enhancing overall efficiency.
 
Architecture
  • Jenkins integrates with source code repositories like GitHub.
  • Developers check code into the repository, triggering a build in Jenkins.
  • Jenkins pulls code changes, builds, tests, and optionally deploys changes.
  • Jenkins publishes results to the repository.
Installation and Configuration:
  • Jenkins can be installed via various methods like WAR file, Docker, or executables.
  • Verify Java installation (versions 8 or 11).
  • Download Jenkins WAR file.
  • Start Jenkins by running java -jar jenkins.war.
  • Access Jenkins through localhost:8080.
  • Set up Jenkins using initial admin password.
  • Go to your .jenkins/secrets/initialAdminPassword for password.
  • Install suggested plugins.
Projects
  1. Freestyle Project:
      • Suitable for simple software development workflows with fewer steps.
      • Ideal for projects or code bases where complexity is low.
  1. Pipeline Jobs:
      • Designed for complex software development workflows involving build, test, and deployment steps.
      • Allows storing workflow steps as code in a Jenkins file for better management and repeatability.
      • Covered in detail in a dedicated tutorial video within the series.
  1. Multi-Configuration Projects:
      • Intended for pipelines targeting multiple platforms or baselines.
      • Useful for scenarios like testing mobile apps on different platforms (e.g., Android and iOS).
  1. Folders:
      • Serve as containers for organizing other items, such as jobs, into a hierarchical structure.
      • Allows grouping of jobs for better management and organization.
  1. GitHub Organization:
      • Utilizes the GitHub Branch Source plugin to scan GitHub repositories and automatically create jobs for repositories containing Jenkins files.
      • Can create dedicated jobs for branches containing Jenkins files, enabling automated job creation based on repository structure.
  1. Multi-Branch Pipeline Job:
      • Similar to GitHub Organization but not limited to GitHub repositories.
      • Can scan branches of a specified code base, creating Jenkins jobs for branches containing Jenkins files.
      • Offers flexibility for defining the code base from various source code management systems.
Triggers
Manual Build Trigger:
  • Build Now: You can manually trigger a build at any time by clicking the “Build Now” or “Build” button on the Jenkins dashboard for a specific job. This is useful for ad-hoc or on-demand builds.
Scheduled Build Trigger:
  • Build Periodically: You can schedule builds to run at specific intervals using the “Build periodically” option in the job configuration. You can use cron syntax to define the schedule. For example, to run a build every night at 2:00 AM, you can use the cron expression 0 2 * * *.
SCM (Source Code Management) Trigger:
  • Poll SCM: If you’re using a version control system (e.g., Git, Subversion), you can configure the job to poll the repository for changes. When changes are detected, Jenkins will automatically trigger a build. This is known as the “Poll SCM” build trigger.
Webhooks Trigger:
  • Webhooks: Many version control systems and external services support webhooks, which allow them to notify Jenkins when code changes occur. Jenkins can listen for these webhook notifications and trigger builds in response.
Dependency Build Trigger:
  • Build after other projects are built: You can configure a job to be triggered after one or more upstream jobs have been built successfully. This is useful for creating build pipelines or ensuring that certain prerequisites are met before starting a build.
Parameterized Build Trigger:
  • Trigger builds with parameters: You can set up parameterized builds where a build is triggered with specific parameter values. This is useful for customizing builds based on input parameters.
Pipeline Trigger:
  • Pipeline Trigger: If you’re using Jenkins Pipelines (defined in a Jenkinsfile), you can use various triggers within the pipeline script itself. For example, you can set up a webhook trigger, a schedule trigger, or a manual input trigger as part of your pipeline script.
Remote API Trigger:
  • Remote API: You can trigger Jenkins builds remotely using its REST API. This allows you to programmatically initiate builds from external scripts or applications.
Plugin-Based Triggers:
  • Jenkins has numerous plugins available that can provide additional trigger mechanisms. For example, the “GitHub Webhook” plugin allows Jenkins to listen to GitHub events and trigger builds accordingly.
Manage Jenkins Page
notion image
  1. System Configuration:
      • Accessible through "Configure System."
      • Allows updating Jenkins configuration and plugin configurations.
      • Notable configurations include:
        • Number of executors: Determines the maximum number of concurrent builds.
        • Jenkins home directory: Contains all data related to the Jenkins instance.
        • Jenkins URL and plugin-specific configurations (e.g., GitHub plugin).
  1. Global Tool Configuration:
      • Enables specification of tool installations required for building and testing code.
      • Examples include specifying JDK versions and Git installations.
      • Supports adding multiple versions of tools like JDK for flexibility.
  1. Managing Plugins:
      • Accessible from the "Manage Plugins" page.
      • Allows updating, installing, and searching for plugins to extend Jenkins functionality.
      • Hundreds of plugins available for various purposes, enhancing Jenkins capabilities.
User Management In Jenkins
To manage users in Jenkins, you should navigate to Manage Jenkins ? Configure Global Security. The ideal option is to let Jenkins’ own user database. You can create a user Anonymous that has only read access. Create entries for users you intend to add in the next step.
  1. In the Authorization strategy, select ‘Matrix-based security’.
notion image
Troubleshooting
Troubleshooting Jenkins
  • I would start by examining the pipeline console output to identify the specific error or failure message.
  • Then, I would review the pipeline configuration, including the build steps, environment variables, and plugin versions, to determine the root cause.
  • If necessary, I would consult logs from the application, Docker containers, or any external services involved in the deployment process for further insights.
Updating a Jenkins plugin
  • I would first check the release notes and documentation of the updated plugin to identify any breaking changes or compatibility issues. If possible,
  • I would roll back the plugin update to temporarily restore functionality while investigating alternative solutions. Alternatively,
  • I might seek community support or open a ticket with the plugin maintainers to address the compatibility issue or request guidance on mitigating the impact.
Merge Conflicts
  • I would resolve the merge conflict in the feature branch by manually addressing the conflicts or rebasing onto the latest changes from the main branch.
  • Once the conflict is resolved, I would push the changes to the remote repository and trigger the Jenkins pipeline again to verify successful execution.
  • Additionally, I would consider implementing automated tests and pre-merge checks to detect and prevent merge conflicts in the future.
Docker image build stage due to a dependency error
  • I would review the Dockerfile and the associated build context to identify any missing or incorrect dependencies.
  • Then, I would update the Dockerfile or build configuration to include the necessary dependencies or resolve any version conflicts.
  • If the issue persists, I would explore alternative dependency management strategies, such as using a package manager or pre-built base images, to ensure reliable and reproducible Docker image builds.
Slow Jenkins pipelines
  • I would start by monitoring Jenkins system metrics, such as CPU utilization, memory usage, disk I/O, and network traffic, to identify potential bottlenecks or resource constraints.
  • Then, I would optimize Jenkins configuration settings, including executor allocation, build queue management, and plugin usage, to improve resource utilization and mitigate performance issues.
  • Additionally, I might consider scaling Jenkins horizontally or vertically to handle increased workload demands.
Security
  • Integrate a security vulnerability scanning tool (e.g., Clair, Trivy, OWASP ZAP) into the Jenkins pipeline.
  • Perform scanning as a post-build step or parallel task.
  • Analyze container images, application code, or infrastructure configurations for known vulnerabilities.
  • Configure the pipeline to fail builds or trigger alerts based on predefined security thresholds.
Multibranch Pipeline
  • Use Multibranch Pipeline Jobs in Jenkins to automatically create and manage pipelines for each Git branch.
  • Define consistent pipeline configurations using a Jenkinsfile stored in each branch to ensure uniformity across pipelines.
  • Configure branch-specific build triggers to automatically trigger pipelines when changes are pushed to the respective branches.
  • Implement shared libraries for common pipeline logic to reduce duplication across branches.
  • Use environment-specific stages or conditions within the Jenkinsfile to differentiate branch-specific behavior (e.g., development, staging, production).
  • Enable pipeline pruning to remove outdated or deleted branch pipelines automatically.
Manage credentials
  • Use Jenkins Credentials Manager to securely store sensitive credentials (e.g., API keys, cloud provider secrets) and avoid hardcoding them in the pipeline.
  • Restrict access to credentials by assigning them only to specific Jenkins jobs or users that require them.
  • Integrate with external secret management tools (e.g., HashiCorp Vault, AWS Secrets Manager) for dynamic and secure retrieval of credentials at runtime.
  • Leverage environment variables to inject credentials into the pipeline steps without exposing them in logs or source code.
  • Use role-based access control (RBAC) on the external service to grant minimal, necessary permissions for Jenkins tasks.
  • Rotate credentials regularly and enforce expiration policies to minimize the risk of credential misuse.
Notification
  • Use Jenkins plugins for notifications, such as Slack Notification, Email Extension, or Microsoft Teams Notifications, to integrate with your team’s communication tools.
  • Configure notifications in the Jenkinsfile using the relevant plugin, ensuring notifications are triggered on specific pipeline events (e.g., success, failure, unstable builds).
  • Set up email notifications for build failures or status changes by configuring Jenkins to send emails to the team or specific individuals.
  • Use webhook-based integrations to send real-time notifications to services like Slack or custom monitoring platforms.
  • Customize notification messages to include key details like build name, number, status, and a direct link to the Jenkins job.
  • Implement different notification levels (e.g., notify only on failures, or alert key stakeholders for critical pipelines).
Jenkins Audit Logging
  • Use the Audit Trail Plugin to log all Jenkins activities, including user actions and configuration changes.
  • Configure the plugin to store logs in a secure and accessible location, such as a centralized log management system.
  • Implement Role-Based Access Control (RBAC):
    • Use the Role Strategy Plugin to define and enforce access controls based on user roles and responsibilities.
    • Ensure that only authorized personnel have access to sensitive pipeline configurations and credentials.
Disaster Recovery Plan
  • Deploy Jenkins in a Clustered Environment:
    • Use Jenkins Operations Center or a similar solution to manage multiple Jenkins masters and distribute the load.
    • Configure Jenkins to run on multiple masters with a shared file system or database. Use Redundant Infrastructure:
  • Deploy Jenkins on a cloud platform with auto-scaling and load balancing capabilities to handle increased load and ensure availability.
  • Implement multi-availability zone or multi-region deployments to protect against localized failures.
Jenkinsfile
pipeline { agent { docker { image 'abhishekf5/maven-abhishek-docker-agent:v1' args '--user root -v /var/run/docker.sock:/var/run/docker.sock' // mount Docker socket to access the host's Docker daemon } } stages { stage('Checkout') { steps { sh 'echo passed' //git branch: 'main', url: 'https://github.com/iam-veeramalla/Jenkins-Zero-To-Hero.git' } } stage('Build and Test') { steps { sh 'ls -ltr' // build the project and create a JAR file sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn clean package' } } stage('Static Code Analysis') { environment { SONAR_URL = "http://34.201.116.83:9000" } steps { withCredentials([string(credentialsId: 'sonarqube', variable: 'SONAR_AUTH_TOKEN')]) { sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn sonar:sonar -Dsonar.login=$SONAR_AUTH_TOKEN -Dsonar.host.url=${SONAR_URL}' } } } stage('Build and Push Docker Image') { environment { DOCKER_IMAGE = "abhishekf5/ultimate-cicd:${BUILD_NUMBER}" // DOCKERFILE_LOCATION = "java-maven-sonar-argocd-helm-k8s/spring-boot-app/Dockerfile" REGISTRY_CREDENTIALS = credentials('docker-cred') } steps { script { sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && docker build -t ${DOCKER_IMAGE} .' def dockerImage = docker.image("${DOCKER_IMAGE}") docker.withRegistry('https://index.docker.io/v1/', "docker-cred") { dockerImage.push() } } } } stage('Update Deployment File') { environment { GIT_REPO_NAME = "Jenkins-Zero-To-Hero" GIT_USER_NAME = "iam-veeramalla" } steps { withCredentials([string(credentialsId: 'github', variable: 'GITHUB_TOKEN')]) { sh ''' git config user.email "abhishek.xyz@gmail.com" git config user.name "Abhishek Veeramalla" BUILD_NUMBER=${BUILD_NUMBER} sed -i "s/replaceImageTag/${BUILD_NUMBER}/g" java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml git add java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml git commit -m "Update deployment image to version ${BUILD_NUMBER}" git push https://${GITHUB_TOKEN}@github.com/${GIT_USER_NAME}/${GIT_REPO_NAME} HEAD:main ''' } } } } }