DevSecOps Practice Project Documentation

Step 1: Setting Up the Environment

1.1 Create an EC2 Instance

1. Log in to AWS Management Console:

   • Navigate to the EC2 Dashboard.

2. Launch a New Instance:

   • Click on "Launch Instance".

   • Choose the Amazon Machine Image (AMI): Ubuntu Server 20.04 LTS.

   • 

3. Select Instance Type:

   • Choose t2.large(2 vCPUs, 8 GiB Memory) to handle the necessary downloads and operations.

   • 

     Select the key pair you have create to connect with the cloud machine

   • 

4. Configure Network Settings:

   1. 

5. Configure Instance Details:

   • Keep the default settings unless specific configurations are needed.

6. Add Storage:

   • Set the storage to 10 GB.

   • 

7. Add Tags (Optional):

   • Add any tags if needed for identification.

8. Configure Security Group:

   • Create a new security group or select an existing one.

   • Ensure ports 22 (SSH), 80 (HTTP), and 443 (HTTPS) are open.

9. Review and Launch:

   • Review the instance configuration.

   • Click "Launch".

   • Launch the instance.

1.2 Connect to the EC2 Instance

1. Open your EC2 instance

   

2. Select and click on Connect

   

3. Open Terminal (on your local machine).

4. Connect to the Instance:

   • Be in the folder, where your key pair file is saved

   • Use the above commands one by one to connect via SSH:

   • for e.g

       ssh -i "your-key-pair.pem" ubuntu@your-ec2-instance-public-dns
     

Step 2: Setting Up Docker and Docker Compose

2.1 Update the Ubuntu Packages

1. Update Package Index:

   • Run the following commands to update the package index:

       sudo apt-get update
       sudo apt-get upgrade -y
     

2.2 Install Docker

1. Install Docker:

   • Run the following command to install Docker:

       sudo apt-get install docker.io -y
     

2. Verify Docker Installation:

   • Check Docker version to verify installation:

       docker --version
     

2.3 Install Docker Compose

1. Install Docker Compose:

   • Run the following command to install Docker Compose:

       sudo apt-get install docker-compose -y
     

2. Verify Docker Compose Installation:

   • Check Docker Compose version to verify installation:

       docker-compose --version
     

2.4 Resolve Docker Permission Issues

1. Check Docker Permissions:

   • Run docker ps to check if you have the necessary permissions. If you face a permission denied error, follow the next steps.

2. Identify the Current User:

   • Run the following command to identify the current user:

       whoami
     

3. Add the User to Docker Group:

   • Add your user to the Docker group:

       sudo usermod -aG docker $USER
     

4. Reboot the Instance:

   • Reboot the instance to apply the group changes:

       sudo reboot
     

   • Note: The reboot will take approximately 1 minute.

5. Reconnect to the Instance:

   • After the reboot, reconnect to your EC2 instance using the same SSH command:

       ssh -i "your-key-pair.pem" ubuntu@your-ec2-instance-public-dns
     

6. Verify Docker Permissions:

   • Run docker ps again to ensure the permission issue is resolved.

---

Next Step is to install Jenkins...

Jenkins is an open-source automation server that is widely used for continuous integration and continuous delivery (CI/CD). It helps automate the parts of software development related to building, testing, and deploying, facilitating continuous integration and continuous delivery. Here’s a more detailed explanation:

What is Jenkins?

Overview

Jenkins is an open-source tool written in Java with plugins built for Continuous Integration purposes. It is used to build and test software projects continuously, making it easier for developers to integrate changes to the project and for users to obtain a fresh build. It also allows you to continuously deliver your software by integrating with a large number of testing and deployment technologies.

Key Features

1. Easy Installation: Jenkins can be installed through native system packages, Docker, or even run standalone on any machine with a Java Runtime Environment (JRE) installed.

2. Extensible: Jenkins can be extended via its plugin architecture, providing nearly infinite possibilities for what it can do.

3. Distributed: Jenkins can distribute build/test loads to multiple machines, helping drive builds across multiple platforms faster.

4. Support for Multiple Languages and Technologies: Jenkins supports a range of tools and languages, making it a versatile tool for developers working in different ecosystems.

5. Rich Plugin Ecosystem: Jenkins boasts a huge library of plugins that enhance its functionality, allowing it to integrate with other CI/CD tools and systems.

Core Concepts

1. Pipeline: A Jenkins pipeline is a suite of plugins that supports implementing and integrating continuous delivery pipelines into Jenkins. It provides a DSL (Domain-Specific Language) to model deployment pipelines as code.

2. Job: A job in Jenkins represents a runnable task. This could be building your software, running tests, or deploying an application.

3. Node: A node is a machine that Jenkins runs on. A node can be the master (the central server) or a slave (a machine that runs build jobs).

4. Build: A build is the result of one run of a job. Jenkins archives these builds and can display reports and logs for each.

Basic Workflow

1. Commit: Developers commit code changes to the source code repository.

2. Build: Jenkins detects the changes in the repository and automatically pulls the latest version. It then builds the project (e.g., compiles the code).

3. Test: Jenkins runs automated tests to verify the integrity of the build.

4. Deploy: If the build and tests are successful, Jenkins can deploy the application to a staging or production environment.

5. Feedback: Jenkins provides feedback to the developers about the build status, test results, and deployment status.

Installing Jenkins

To install Jenkins on your EC2 instance, you can follow these steps:

1. Visit --> https://www.jenkins.io/doc/book/installing/linux/

   

2. Install Java (required for Jenkins):

    sudo apt update
    sudo apt install fontconfig openjdk-17-jre
   

3. Add Jenkins Repository:

    sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
      https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
    echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
      https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
      /etc/apt/sources.list.d/jenkins.list > /dev/null
   

4. Update the package index again:

    sudo apt-get update
   

5. Install Jenkins:

    sudo apt-get install jenkins
   

6. Check Jenkins Status

    systemctl status jenkins
   

7. Start Jenkins:

    sudo systemctl start jenkins
   

8. Enable Jenkins to start at boot:

    sudo systemctl enable jenkins
   

9. Open Jenkins Web Interface:

   • Open your web browser and go to http://your-ec2-instance-public-dns:8080.

   • Jenkins runs on port 8080, go to your instance and on security tab

     • 

   • In security tab, move to security group

     • 

   • Edit inbound rules, and open port 8080, allow from anywhere

   • Copy your ip address for cloud machine and add :8080 port number, and your jenkins is running

     • 

   • Now to Unlock Jenkins through password, go to terminal of your cloud machine, type

   •   sudo cat /var/lib/jenkins/secrets/initialAdminPassword
     

   • Install suggested Plugins

     • 

Next Steps

Once Jenkins is installed and set up, we can start creating jobs and pipelines to automate our CI/CD processes.

Through Jenkins, now we have to install below mentioned plugins

• Sonarqube Scanner

• Sonar Quality Gates

• OWASP Dependency-Check

• Docker

Go to manage Jenkins

As click on the installation, do check on restart Jenkins

Next Step, we would need to integrate Sonarqube with Jenkins

This Integration is shown after Trivy Setup in this Document

Next Step would be to setup Sonar Cube Server

SonarQube is an open-source platform used for continuous inspection of code quality. It performs automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on 20+ programming languages. Here’s a more detailed explanation:

What is SonarQube?

Overview

SonarQube is an open-source tool for continuous code quality inspection. It supports multiple programming languages and integrates seamlessly with CI/CD pipelines to ensure that code meets the required quality standards. By analyzing the source code, it provides detailed reports on various aspects of code quality, including potential bugs, code smells (maintainability issues), and security vulnerabilities.

Key Features

1. Multi-Language Support: SonarQube supports more than 20 programming languages, including Java, JavaScript, TypeScript, C#, C/C++, Python, and more.

2. Static Code Analysis: SonarQube performs static code analysis to identify potential bugs, vulnerabilities, and code smells without executing the code.

3. Quality Gates: Quality gates are a set of conditions that the code must meet to be considered of good quality. These gates can be customized to fit the needs of a project.

4. Security Vulnerabilities Detection: It identifies potential security risks and vulnerabilities in the codebase.

5. Integrations: SonarQube integrates with various build tools (Maven, Gradle, Ant), CI/CD pipelines (Jenkins, GitLab CI, Travis CI), and other development tools (GitHub, Bitbucket, Azure DevOps).

6. Code Coverage: SonarQube provides detailed reports on code coverage, helping teams to ensure that their tests cover the necessary parts of the code.

7. Developer Feedback: It provides developers with immediate feedback on code quality issues directly in their IDEs or pull requests.

8. Customizable Rules: SonarQube allows customization of code analysis rules to match the coding standards and guidelines of a project.

Core Concepts

1. Project: In SonarQube, a project represents the codebase of a particular application or component that you want to analyze.

2. Quality Profiles: A set of rules that define what needs to be checked during the code analysis.

3. Quality Gates: Criteria that a project must meet to be considered of acceptable quality. If any condition is not met, the quality gate fails.

4. Issues: Problems found during the analysis, categorized as bugs, vulnerabilities, and code smells.

Basic Workflow

1. Code Analysis: Developers write code and commit it to a source control system (e.g., GitHub).

2. Continuous Integration: A CI tool (e.g., Jenkins) triggers a build, which includes running SonarQube analysis on the code.

3. Inspection Results: SonarQube analyzes the code and provides detailed reports on issues found, such as bugs, vulnerabilities, and code smells.

4. Feedback Loop: Developers review the issues reported by SonarQube and fix them. The fixed code is then committed back to the repository.

5. Quality Gates: The project’s code must pass the quality gates before it can be merged or deployed.

SonarQube Installation

Prerequisites

• Ensure Docker is installed on your system.

• Allocate sufficient resources to Docker (at least 2GB of RAM).

Step-by-Step Guide

Step 1: Pull the SonarQube Docker Image

Pull the latest long-term support (LTS) community edition of SonarQube:

docker pull sonarqube:lts-community

Step 2: Run the SonarQube Docker Container

Run the SonarQube server using the Docker command:

docker run -itd --name sonarqube-server -p 9000:9000 sonarqube:lts-community

This command will:

• Start SonarQube in a detached mode (-itd).

• Name the container sonarqube-server.

• Map port 9000 of the container to port 9000 on your host machine (-p 9000:9000).

Step 3: Verify SonarQube is Running

Check if the container is running:

docker ps

You should see sonarqube-server listed in the running containers.

Step 4: Access the SonarQube Web Interface

Open a web browser and navigate to http://localhost:9000.

Or if using Cloud Machine, than again this time we have to open port 9000 like we did before for jenkins port 8080

Step 5: Log In to SonarQube

Use the default credentials to log in:

• Username: admin

• Password: admin

Step 6: Configure SonarQube

After logging in, you will be prompted to change the default password. Follow the on-screen instructions to do so.

Step 7: Setting Up a Project

1. Create a new project:

   • Click on the “Create new project” button.

   • Provide a project key and name.

2. Generate a token:

   • SonarQube will prompt you to generate a token. This token is required for the scanner to authenticate with the SonarQube server.

   • Save the token as it will be shown only once.

3. Download and configure the scanner:

   • Follow the instructions provided to download and configure the SonarQube scanner. This typically involves installing the scanner and configuring it with the server details and the generated token.

Step 8: Analyzing Your Code

1. Navigate to your project directory.

2. Run the SonarQube scanner. The command will look something like this:

sonar-scanner \
  -Dsonar.projectKey=my_project_key \
  -Dsonar.sources=. \
  -Dsonar.host.url=http://localhost:9000 \
  -Dsonar.login=your_generated_token

Replace my_project_key with your actual project key and your_generated_token with the token you generated earlier.

Step 9: Viewing Analysis Results

After the scanner completes, go back to the SonarQube web interface (http://localhost:9000) and navigate to your project to see the analysis results.

Step 10: Stopping the SonarQube Container

To stop the SonarQube server, use the following command:

docker stop sonarqube-server

To remove the container:

docker rm sonarqube-server

Additional Configuration

Setting Up a Database

By default, the SonarQube container uses an embedded H2 database, which is not recommended for production use. You should configure an external database (like PostgreSQL) for better performance and reliability.

Example of running SonarQube with a PostgreSQL database:

1. Run PostgreSQL container:

docker run -d --name sonarqube-postgres -e POSTGRES_USER=sonar -e POSTGRES_PASSWORD=sonar -e POSTGRES_DB=sonarqube -p 5432:5432 postgres

2. Run SonarQube container with PostgreSQL:

docker run -d --name sonarqube-server --link sonarqube-postgres:db -p 9000:9000 sonarqube:lts-community

3. Update SonarQube configuration:

   • Edit sonar.properties to point to the PostgreSQL database:

# Set these values in sonar.properties file
sonar.jdbc.url=jdbc:postgresql://db:5432/sonarqube
sonar.jdbc.username=sonar
sonar.jdbc.password=sonar

Summary

This guide covers setting up SonarQube using Docker, creating a project, and analyzing your code. It also includes additional steps for configuring an external database for a production environment. This should provide a solid foundation for using SonarQube in your projects.

Next Steps

Once SonarQube is installed and set up, you can start configuring your projects for code analysis.

Our main motive is to send code to sonarqube through Jenkins, and then sonarqube will share the analysis with Jenkins

Next Step is to install Trivy

Trivy is an open-source vulnerability scanner for containers and other artifacts, including Docker images, Kubernetes clusters, and repositories. It is designed to detect vulnerabilities in your application code and dependencies, ensuring your software is secure and compliant with best practices.

What is Trivy?

Overview

Trivy (pronounced "trivy") is an easy-to-use and comprehensive vulnerability scanner for containers and other systems. It is used to scan for vulnerabilities in container images, file systems, and repositories by identifying known security issues in the packages and dependencies used in the applications. Trivy supports a wide range of operating systems and programming languages, making it a versatile tool for developers and DevOps teams.

Key Features

1. Comprehensive Scanning: Trivy performs extensive scanning of container images and file systems to detect vulnerabilities in OS packages and language-specific dependencies (e.g., Ruby, Node.js, Python).

2. Ease of Use: Trivy is simple to set up and use. It requires minimal configuration and can be run with a single command.

3. Wide Language and OS Support: Trivy supports various operating systems (e.g., Alpine, RHEL, CentOS, Ubuntu) and languages (e.g., Go, Java, JavaScript, Python, Ruby, PHP, Rust).

4. Detailed Reports: Trivy provides detailed reports of the vulnerabilities it finds, including severity levels and links to more information.

5. Integration with CI/CD: Trivy can be easily integrated into CI/CD pipelines to ensure that code and dependencies are scanned for vulnerabilities before deployment.

6. Configuration File Support: Trivy can scan configuration files for misconfigurations, helping ensure that your deployments follow best practices.

7. Community and Support: Being open-source, Trivy has an active community and is maintained by Aqua Security, a leading cloud-native security company.

Basic Workflow

1. Install Trivy: Download and install Trivy on your system.

2. Scan Artifacts: Use Trivy to scan container images, file systems, or repositories for vulnerabilities.

3. Review Results: Analyze the detailed reports provided by Trivy to identify and address vulnerabilities.

4. Integrate with CI/CD: Integrate Trivy into your CI/CD pipeline to automate vulnerability scanning during the build and deployment processes.

Step-by-Step Guide to Install Trivy

Step 1: Update the Package List

sudo apt-get update

Step 2: Install Prerequisites

Install necessary packages for Trivy:

sudo apt-get install wget apt-transport-https gnupg lsb-release -y

Step 3: Add the Aqua Security Trivy Repository

1. Download and add the signing key:

wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null

2. Add the Trivy repository:

echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/trivy.list

Step 4: Update the Package List Again

sudo apt-get update

Step 5: Install Trivy

sudo apt-get install trivy -y

Verify the Installation

To verify that Trivy is installed correctly, run:

trivy --version

You should see the version information for Trivy.

Using Trivy

Scan a Docker Image

To scan a Docker image for vulnerabilities, use the following command:

trivy image your-docker-image:tag

Replace your-docker-image:tag with the name and tag of the Docker image you want to scan.

Example

trivy image ubuntu:latest

This command will scan the latest Ubuntu image for vulnerabilities and display the results.

Next Steps

Once Trivy is set up and integrated, you can start scanning your container images, file systems, and repositories regularly to ensure they are free from known vulnerabilities.

Now as above we told , that we need to Integrate Sonarqube with Jenkins, so we have to continue that

Go to Sonarqube Administration Tab

Select Webhooks in Configuration

As you see in the explanation Image, that web hooks would be used as the bridge between Jenkins and Sonarqube, so we are going to create a webhook

After Creating the webhook, go to Security Users, and then we have to create token

Now again go to Jenkins, manage System

Scroll down to Sonarqube Servers

Here we have to add Sonarqube, and when we are going to add sonarqube here, we also have to add the secret token, which we generated from Sonarqube, but to add the secret token, first move to credentials > global > add credentials

Sonarqube and Jenkins are Integrated now