Jenkins – Python Module for Logs in pipelines

In this post, I want to show, in a similar way to the previous one, how to generate logs in pipelines, but this time from a Python script. https://devops-db.com/jenkins-class-for-logs-in-pipelines/

To do this, I’m going to start generating a little more complexity, not yet in the model that I want to have the pipelines, or codes, but already on the way.

First point, because they are Python scripts, it is not possible to have the classes or methods in a Shared Library. What I’m going to use is a python module with generic methods that I’ve already started creating and will be used a lot from now on. I hadn’t published it yet, but here’s a first use.https://github.com/faustobranco/devops-db/tree/master/infrastructure/resources/python-modules/devopsdb

This module has several other classes and methods that are not relevant now, I will just deal with the pipeline_logger, which is, along the lines of the previous one in Groovy, just a module for generating Log entries.

Using the methods described in post https://devops-db.com/devpi-local-repository-of-python-modules/, the module was compiled and uploaded to our DevPi repository.

Second point, the standard image does not have this python module installed, nor should it, because installing it as the pipeline runs, despite having the disadvantage of having more processes and impacting time, is safer, as we will always have the most current version .
The repository/DevPi is not the Python standard, it is a local repository, so it needs a specific pip.conf, with the address and in our case a trusted-host entry.
To do this, for now, I created an entry in the pipelines/resources repository with a pip-devops.conf file and in it the information about our DevPi. https://github.com/faustobranco/devops-db/tree/master/infrastructure/pipelines/resources/pip
This file will be used in Sparse Checkout during the pipeline and will be used to install the module via pip.

Third point, the python script that we are going to run also needs to be done Sparse Checkout for the pipeline workspace in the container so that it can be run. To this end, the file is in another repository, for now, one that I created just for the test scripts. https://github.com/faustobranco/devops-db/tree/master/infrastructure/pipelines/tests/python-log

See that in the pipeline, in the Environment step, I do the sparse checkout of pip-devops.conf and then install the module.

In the Script step, I sparse checkout the test script in Python and then make the call.

The Pipeline result is below, where I highlight some important points, such as the log result left by Python.

@Library('devopsdb-global-lib') _

import devopsdb.utilities.Utilities
def obj_Utilities = new Utilities(this)

pipeline {
    agent {
        docker {
            image 'registry.devops-db.internal:5000/ubuntu_python:3.10.6'
            label 'docker'
            args '-u root'
        }
    }
    options { timestamps ()
        skipDefaultCheckout(true)
    }
    stages {
        stage('Environment') {
            steps {
                script {
                    def str_folder = "${env.WORKSPACE}/pipelines/resources/pip"
                    def str_folderCheckout = "/pip"
                    obj_Utilities.CreateFolders(str_folder)
                    obj_Utilities.SparseCheckout('git@gitlab.devops-db.internal:infrastructure/pipelines/resources.git',
                            'master',
                            str_folderCheckout,
                            'usr-service-jenkins',
                            str_folder)
                    sh "PIP_CONFIG_FILE=${str_folder}${str_folderCheckout}/pip-devops.conf pip install devopsdb"
                }
            }
        }
        stage('Script') {
            steps {
                script {
                    def str_folder = "${env.WORKSPACE}/pipelines/python/log"
                    def str_folderCheckout = "/python-log"
                    obj_Utilities.CreateFolders(str_folder)
                    obj_Utilities.SparseCheckout('git@gitlab.devops-db.internal:infrastructure/pipelines/tests.git',
                            'master',
                            str_folderCheckout,
                            'usr-service-jenkins',
                            str_folder)
                    sh "python3 ${str_folder}${str_folderCheckout}/python-log.py"
                }
            }
        }
        stage('Cleanup') {
            steps {
                cleanWs deleteDirs: true, disableDeferredWipeout: true
            }
        }
    }
}

The pipeline result:

No Caption