gitlab-ci-local: running your tests locally
image: Gitlab CI/CD pipelines
Introduction
First of all let me introduce the term CI for those that are not familiar with: CI is an abbreviation of Continuous Integration and, as the name suggest, refers to the process of integrate your new implementation to main code. In the development process is very important that the new path, that will be integrated, add some features and bug fixes without breaking the previous version. Sometimes this is something that can happen but we must to prevent, as much as we can, the possible errors.
Besides that, is very important to assure that the code that will be integrated is clean (follow the project pattern with good practices, like no spaces in blank lines) and can be built without errors. Of course that those changes could be seem in a Merge Request, for example, but those small things could be fixed automatically. For this we have the CI process.
Gitlab and gitlab-ci.yml
Gitlab enviroment has a great support to continuous integration, named as CI/CD (Continuos Integration and Continuos Deployment). To use it your tests must be defined at gitlab-ci.yml file, in the project root, and it will be automatically read and applied. In this file you can add lint tests, unit tests, define dependencies, build things, define when to run the pipeline and also make the deploy procedure (CD).
Running at gitlab you can see an interface like the presented in the main image of this post (cover), that show the pipeline running and all jobs states and relations. The pipeline is the combination of all tests that are defined at gitlab-ci.yml, and one job represent one pipeline test (in the image you can see five jobs, for example). These terms are important.
So here is an example of a simple gitlab-ci.yml that apply lint and unit test, also defining some rules about when to run the pipeline.
stages:
- lint
- test
lint:
stage: lint
image: python:3.11
script:
- pip install flake8
- flake8 .
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: on_success
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: on_success
- when: never
unit-tests:
stage: test
image: python:3.11
before_script:
- pip install -r requirements.txt
- pip install pytest
script:
- pytest -v
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
when: on_success
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
when: on_success
- when: never
The jobs are defined as lint and unit-tests. Both of them are executed using a python base image (Docker) and receive the definition that:
- If in branch main -> execute
- If in merge request -> execute
- Else -> never (will not be executed)
gitlab-ci-local
Now we get into the main subject of this post. Recently, one notification appear in one of the repositories that I'm a developer about the limits of CI/CD resource. For no paying accounts you can have pipeline limits about minutes and jobs quantity. This receive my attention because I observe that I was running a lot of pipelines just because was easier to see tests errors instead to run all of them one at time locally.
Searching for a solution I found this project that resolved my several pipelines problem. Just installing and running gitlab-ci-local you can simulate your pipeline, also base in gitlab-ci.yml, before to push it into your remote repository at gitlab. Why to do that? Some answers for this questions:
- Prevent to run a lot of pipelines at CI/CD;
- See the errors before to push them and need to fix all;
- Run the tests individually, not writing the commands but just informing the test;
- Etc.
You can think that, if gitlab-ci-local already checks the code, CI/CD could just be avoided, but this is not true. In development you have a lot of persons working in the same project, so is necessary to centralize the tests. But use it locally is very useful.
Installing
npm install -g gitlab-ci-local // to install it
gitlab-ci-local --version // just to see if the installation is fine
Running
Go to your project folder that contains gitlab-ci.yml at root path and just run the command below that all your tests will be executed. Is nice to ponit again that gitlab-ci-local will follow gitlab-ci.yml instructions so, if you define for example that the pipeline must run only when pushed to main, you need to go to main branch to run it.
gitlab-ci-local
You can also run with some parameters:
gitlab-ci-local build // only execute the job build
gitlab-ci-local --list-all // to list all jobs that can be executed
gitlab-ci-local --help // to see all options
See the example that shows the tests running for the gitlab-ci.yml file below, that creates and runs a python docker image, recreate the code, test it and create the executable file.
image: python:3
stages:
- linting
- deploy
before_script:
- pip install flake8 pylint pandas chardet PyYAML==6.0.2 openpyxl customtkinter
flake8:
stage: linting
script:
- flake8 --show-source --statistics --extend-ignore=F403,F405 .
pylint:
stage: linting
script:
- pylint --disable=C0114,C0115,C0116,C0301,W0703,R0902,R0903,R0915,W0719,W0511,W0401,W0621 *.py
pages:
stage: deploy
script:
- mkdir -p public
- cp "$(ls dist/*.exe | sort -V | tail -n1)" public/
artifacts:
paths:
- public
only:
- main
Looking to your repository you'll see a new folder called .gitlab-ci-local. You can add to .gitignore file.
Comments
Post a Comment