🌻Jenkins Multi Stage CICD Pipeline with Amazon EKS🌻

Here, we will be deploying a Node JS application

We will be setting up a Jenkins pipeline using Jenkins file. The continuous deployment would be done by Github Webhook. Jenkins GitHub Webhook automates the build and deployment of applications when any commit is done to the source code.

Step 1 — Create and set up an EC2 Instance, configure inbound Security Group TCP Port 8080. Connect to EC2 Instance

Step 2 — Install Java

Step 3— Install and setup Jenkins

Step 4— Update visudo and assign administrative privileges to jenkins user

Step 5— Install Docker

Step 6— Install and Setup AWS CLI

Step 7— Install and setup eksctl

Step 8— Creating an EKS Cluster using eksctl

Step 9— Add Docker and Github credentials on Jenkins

Step 10— Build/Deploy and test CI/CD pipeline

Step 11— To setup Jenkins — Github webhook

Step 12 — Cleanup/Deprovision

Now, lets get started and dig deeper into each of these steps:-

Step 1 — Log in to your AWS Console and search for EC2. Click on Launch Instance. You can name it as Jenkins-freestyle

Select the Instance type as T2 Medium. Kindly note that this is not the free instance and a paid one. So, as soon as you are finished working on this project, remember to de-provision or tear it down to avoid any additional costs. Create and download the key

T2 Medium Instance and Jenkins-key

Allow HTTP and HTTPs Traffic from the Internet. Click on Launch EC2 Instance is now in Running State.

Since Jenkins works on Port 8080, configure the Security Group. Add Custom TCP Port 8080 to access Jenkins using the Public IP Address of the EC2 Instance. Inbound Security Group would be now modified for our Jenkins.

Next, go ahead and login to the EC2 console. Connect to it either using the console or via SSH Key Pair that we earlier downloaded.

Step 2— Install Java

$ sudo apt update
$ sudo apt install openjdk-11-jre
$ java --version

The output would look like this

Java is now installed

Step 3— Install and setup Jenkins

$ wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
$ sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
$ sudo apt-get update
$ sudo apt-get install jenkins

Start Jenkins

$ sudo systemctl enable jenkins
$ sudo systemctl start jenkins
$ sudo systemctl status jenkins

Now, Copy IP address of EC2 Instance and search in browser

Jenkins Terminal

Copy the password path and go to your terminal and run it using cat command

$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the password and run it in browser

Install the suggested plugins,

Create a First Admin User and save it.

Copy this url and paste it in a new tab and save it.

Jenkins is set up successfully now.

Jenkins is now ready!

Step 4— Update visudo and assign administration privileges to jenkins user

Now we have installed the Jenkins on the EC2 instance. To interact with the Kubernetes cluster Jenkins will be executing the shell script with the Jenkins user, so the Jenkins user should have an administration(superuser) role assigned forehand.

Run pipeline with Jenkins User, So we will need to give privileges

Let’s add jenkins user as an administrator and also ass NOPASSWD so that during the pipeline run it will not ask for root password.

Open the file /etc/sudoers in vi mode

$ sudo vi /etc/sudoers

Add the following line at the end of the file

$ jenkins ALL=(ALL) NOPASSWD: ALL

After adding the line save and quit the file with :wq!

Now we can use Jenkins as root user and for that run the following command

$ sudo su - jenkins

Step 5— Install Docker

Remember, all these commands are run as a Jenkins user and not as Ubuntu user.

$ sudo apt install docker.io
$ docker --version
$ docker ps
$ sudo usermod -aG docker jenkins
$ sudo docker ps
$ sudo reboot

Step 6— Install and set up AWSCLI

Remember, we will be using the Jenkins user to install AWSCLI. So, after rebooting if you are in Ubuntu user, use the below command to come into Jenkins

$ sudo su - jenkins

AWS CLI will be able to authenticate EC2 Instance

$ sudo apt install awscli
$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
$ unzip awscliv2.zip
$ sudo ./aws/install --update
$ aws --version

Now after installing AWS CLI, let’s configure the AWS CLI so that it can authenticate and communicate with the AWS environment.

$ aws configure

Login to your AWS Console, goto Security Credentials and create a new Secret access key. Remember, to download this key, as you will not be able to access it later if you haven’t downloaded. Enter all these details

Step 7— Install and setup kubectl

$ curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
$ chmod +x ./kubectl
$ sudo mv ./kubectl /usr/local/bin
$ kubectl version

Step 8— Install and setup eksctl

First, we have to install and setup eksctl using these commands.

$ curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
$ sudo mv /tmp/eksctl /usr/local/bin
$ eksctl version

You can refer to this URL to create an Amazon EKS Cluster. https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.html .

Here I have created cluster using eksctl

You need the following in order to run the eksctl command

1. Name of the cluster : — name first-eks-cluster1

2. Version of Kubernetes : — version 1.24

3. Region : — region us-east-1

4. Nodegroup name/worker nodes : — nodegroup-name worker-nodes

5. Node Type : — nodegroup-type t2.micro

6. Number of nodes: — nodes 2

This command will set up the EKS Cluster in our EC2 Instance. The command for the same is as below,

$ eksctl create cluster --name new-eks-cluster1 --version 1.24 --region us-east-1 --nodegroup-name worker-nodes --node-type t2.micro --nodes 2

Kindly note that it took me almost 20–25 minutes to get this installed and running.

To verify whether the EKS cluster is working or not, you can go back to your AWS dashboard and check.

Step 9— Add Docker and Github credentials on Jenkins

Setup Docker Hub Secret Text in Jenkins

You can set the docker credentials by going into -

Goto -> Jenkins -> Manage Jenkins -> Manage Credentials -> Stored scoped to jenkins -> global -> Add Credentials

Step 10— Build/Deploy and test CI/CD pipeline

Okay, now we can start writing out the Jenkins pipeline for deploying the Node.js Application into the Kubernetes Cluster.

Create new Pipeline: Goto Jenkins Dashboard or Jenkins home page click on New Item

Pipeline Name: Now enter Jenkins pipeline name and select Pipeline

Add pipeline script: Goto -> Configure and then pipeline section.

Copy the below script and paste into Pipeline Script.

node {

    stage("Git Clone"){

        git credentialsId: 'GIT_HUB_CREDENTIALS', url: 'https://github.com/writetoritika/node-todo-cicd.git', branch: 'master' 
    }

     stage("Build") {

       sh 'docker build . -t writetoritika/node-todo-test:latest'
       sh 'docker image list'

    }

    withCredentials([string(credentialsId: 'DOCKER_HUB_PASSWORD', variable: 'PASSWORD')]) {
        sh 'docker login -u writetoritika-p $PASSWORD'
    }

    stage("Push Image to Docker Hub"){
        sh 'docker push writetoritika/node-todo-test:latest'
    }

    stage("kubernetes deployment"){
        sh 'kubectl apply -f deployment.yml'
    }
}

Step 11- To set up Jenkins — GitHub Webhook

Jenkins GitHub Webhook is used to trigger the action whenever Developers commit something into the repository. It can automatically build and deploy applications.

Switch to your GitHub account, go to “Settings” option. Here, select the “Webhooks” option and then click on the “Add Webhook”

It will provide you the blank fields to add the Payload URL where you will paste your Jenkins address, Content type, and other configuration.

Go to your Jenkins tab and copy the URL then paste it in the text field named “Payload URL“, as shown in the image below. Append the “/github-webhook/” at the end of the URL.

Now, goto your Jenkins pipeline, and build now. The output would look like this

To check if the application is working or not, use this command

$ kubectl get deployments
$ kubectl get svc

Step 12 — Deprovision all Resources

Delete EKS Cluster from AWS Console.

Hope you liked my project on deploying Node.JS app on Amazon EKScluster using EC2 Instance, Docker, Jenkins, Kubernetes, Github and lastly Webhook for continuous deployment. Please give your claps and follow me for more interesting DevOps Projects. Thank you! 🙏

Here s the Github Repsoitory for the same.

https://github.com/writetoritika/node-todo-cicd