I decided to write this post, to better detail the configuration and use of the Pod Template, described in the previous post. Jenkins – Agents in Kubernetes. It is important to read the post to understand some of its context. I also highly recommend reading our next post: Jenkins – Shared Libraries, Containers and Manifests.
Something that can go unnoticed and generate a bit of confusion.
The configuration itself is that of containers, when we create a Cloud and configure the Pod Template.
So, to better understand where we are going, it is worth remembering the structure of a service in Kubernetes.
A Kubernetes cluster can have as many nodes as necessary, on each node, several PODs and on each POD, one or more Containers. So the Container is where we actually run what we need, very similar to Docker terminology.
So, when we say that we create a POD with an image “X”, we are actually creating a Container with the image “X”, inside a POD.
Problem.
What does this have to do with Jenkins and Pipelines?
By default, when we run a Pipeline using K8s as an agent and with Pod Template as we configured it in the post mentioned above. Kubernetes creates a POD, but with 2 Containers.
Of course, if you configure more Containers in Pod Template, more Containers will be created, but let’s limit ourselves to the example to avoid confusion.
One of the Containers created has the image and name you indicated in the configuration – img-jenkins (Kubernetes – Base image for Jenkins Agent) and the other, created automatically, is a Container for Java Network Launch Protocol (JNLP).
This creates confusion when running scripts in the Pipeline, some commands can be run in the correct Container (In the example img-jenkins) and others in the jnlp Container. And this causes the pipeline to fail.
How to simulate.
Let’s go to the practical example, I’m going to create a very simple pipeline, but with a sleep of 10m, so we can go to bash and get some information from kubernetes.
pipeline {
options { timestamps ()
skipDefaultCheckout(true)
}
agent {
label 'microk8-agent'
}
stages {
stage('Script') {
steps {
script {
sh "sleep 10m"
}
}
}
}
}
Let’s see what we have in POD:
kubectl get pods --namespace jenkins-devopsdb
NAME READY STATUS RESTARTS AGE
jenkins-7f6767b7b8-q4ch2 1/1 Running 5 (11h ago) 22h
microk8-agent-npssn 2/2 Running 0 51s
The 2/2 information on microk8-agent already indicates 2 Containers, let’s see the description of the POD:
kubectl describe pod microk8-agent-npssn --namespace jenkins-devopsdb
Name: microk8-agent-npssn
Namespace: jenkins-devopsdb
[...]
Containers:
img-jenkins:
Container ID: containerd://3981ff7b992cd55caa19bf54e8e9b70ea1b1fd09b403e79ec77a1c92cb9cefff
Image: registry.devops-db.internal:5000/img-jenkins-devopsdb:2.0
Image ID: registry.devops-db.internal:5000/img-jenkins-devopsdb@sha256:76ca827b2ef7c85708864e0792a4d896c6320d9988be21b839a478f2eb1f2026
Port: <none>
Host Port: <none>
State: Running
Started: Thu, 20 Jun 2024 21:47:50 +0000
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/home/jenkins/agent from workspace-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-grl2q (ro)
jnlp:
Container ID: containerd://adb20ed99439d4370d6d030765383e3e1018ca08cf9cbbc51e1a60df4f9576ca
Image: jenkins/inbound-agent:3248.v65ecb_254c298-2
Image ID: docker.io/jenkins/inbound-agent@sha256:f6dd3c30b4ee628a7243167b2c0b29eb18ee42c65cc2c6a1a4b266f0775439fb
Port: <none>
Host Port: <none>
State: Running
Started: Thu, 20 Jun 2024 21:47:50 +0000
Ready: True
Restart Count: 0
Requests:
cpu: 100m
memory: 256Mi
Environment:
JENKINS_SECRET: ac66be69a434091f7501885013bbe3ec89be81b70b806df6fbce308b2d3b4b87
JENKINS_AGENT_NAME: microk8-agent-npssn
REMOTING_OPTS: -noReconnectAfter 1d
JENKINS_NAME: microk8-agent-npssn
JENKINS_AGENT_WORKDIR: /home/jenkins/agent
JENKINS_URL: http://jenkins.lab.devops-db.info/
Mounts:
/home/jenkins/agent from workspace-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-grl2q (ro)
[...] There are two containers, img-jenkins and jnlp.
But what is the implication of this? Let’s put into practice.
One of the images (registry.devops-db.internal:5000/img-jenkins-devopsdb:2.0) that we configured in the Pod Template Container, we know has Python, the other doesn’t. So let’s change the pipeline and add a “python3 -V” shell call, just to show the version of installed Python.
pipeline {
options { timestamps ()
skipDefaultCheckout(true)
}
agent {
label 'microk8-agent'
}
stages {
stage('Script') {
steps {
script {
sh "python3 -V"
}
}
}
}
}
When we run the pipeline it fails, as shown below, because the command was run in the container with the jnlp image and not in the one we expected img-jenkins.
22:55:00 + python3 -V
22:55:00 /home/jenkins/agent/workspace/infrastructure/pipelines/tests/k8_test@tmp/durable-c713f744/script.sh.copy: 1: python3: not found
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
Agent microk8-agent-bg2vz was deleted, but do not have a node body to cancel
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 127
Finished: FAILURESolution.
To get around this error, we need to inform, in the stages/steps, the container where we want the scripts to be executed (img-jenkins):
pipeline {
options { timestamps ()
skipDefaultCheckout(true)
}
agent {
label 'microk8-agent'
}
stages {
stage('Script') {
steps {
container('img-jenkins') {
script {
sh "python3 -V"
}
}
}
}
}
}And the result:
22:58:48 + python3 -V
22:58:48 Python 3.11.2
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESSThis site uses Akismet to reduce spam. Learn how your comment data is processed.
Leave a Reply