Gitlab

https://docs.gitlab.com/omnibus/docker/
https://gitlab.com/

GitLab is a complete DevOps platform With GitLab, you get a complete CI/CD toolchain in a single application. One interface. One conversation. One permission model. Thousands of features. You'll be amazed at everything GitLab can do today. And we're just getting started.

You can get Gitlab up and running by running the following command in the hashiqube source repository

Duration 15 - 30 minutes
Your Docker daemon should at least have 12G Ram (Gitlab takes 6G alone, Minikube 2G and 2G for the Operating System in Hashiqube)
We will bring up the Docker daemon and Minikube and then deploy Gitlab ontop of Minikube using Helm

During the install process you can open the Kubernetes Dashboard once Minikube has been installed - Initally you will see some red as Gitlab pods and services start up, later this will all turn to green.

http://localhost:10888/

Provision

bash docker/docker.sh
bash minikube/minikube.sh
bash gitlab/gitlab.shCopy to clipboardErrorCopiedCopy to clipboardErrorCopied

vagrant up --provision-with basetools,docker,docsify,minikube,gitlabCopy to clipboardErrorCopiedCopy to clipboardErrorCopied

docker compose exec hashiqube /bin/bash
bash hashiqube/basetools.sh
bash docker/docker.sh
bash docsify/docsify.sh
bash minikube/minikube.sh
bash gitlab/gitlab.shCopy to clipboardErrorCopiedCopy to clipboardErrorCopied

The above Gitlab provision will look like this

You can now login to Gitlab at http://localhost:5580 and login with Username root and the password printed out example: jMh629reoQ7FqtillBmLQZPY69JUStSFATXD11T5wMk39NtNezqIKohcIIwoxwvl

You can follow along with the rest of the documentation here:
Locally: http://localhost:3333/gitlab/README?id=you-are-here
Hashiqube.com: https://hashiqube.com/gitlab/README?id=you-are-here

You are here

You have just ran the Gitlab provisioner and Gitlab should now be installed.

Please login with username root and the password that was printed out

Now you should be logged into Gitlab, and you will see a page looking like this

Now let's create our first project in Gitlab
Click on Create a Project -> Create a Blank project and enter the project name as test

Select the namespace as root
Make it a Public repository

And click Create Project

Our Test Repository has been created

Now we need to add our SSH key to our User Profile in Gitlab, we can then clone this repository locally, using our SSH keys as authentication.

To do that, please navigate to Top Left, Click on your Profile Icon image and click on Preferences

Now please navigate to SSH Keys

On your laptop in a Terminal Window, please do: cat ~/.ssh/id_rsa.pub It should look something like this

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIgkrDFTASaZYyJfGd0swmUZaol3JSZmO6D74230CtZjz/YrHq4QL34r5z4oPz9gO8f0l6kN8Hz6BEhAIzArg8kDI9lrsPXypCqlahE49cuzNH3C5GEC9MHo46U6+ZG2IOwCXjiP5ohnXyG8ho8N1BVpDa0xyVevy4COg4malaLVcRwP73YwgxiitNLWcq8k8nB [email protected]Copy to clipboardErrorCopiedCopy to clipboardErrorCopied

Copy that key, we will enter it now, navigate back to Gitlab

Be sure to remove the expiry date and click Add Key

Clone Test Repo

Now we have a method to authenticate to Gitlab, now we can clone our Test repository we created earlier!

Navigate back to the home page of Gitlab, and click on your test repository.

Now, to the right you will see a Clone Button, please click on that, and Copy the Clone with SSH link.

The HTTP Link does not work currently, there is a BUG and i was unable to add the port 5580 in there. So please don't use that, it won;t work.

I will clone this to my ~/workspace/personal/test directory, and I will use this command git clone ssh://git@localhost:32022/root/test.git

The output is below

Gitlab Pipeline

Now that we have cloned the Test repository locally, we can add files to it. For our first file we are going to add a .gitlab-ci.yml Gitlab Pipeline file.

So create a new file in the test reposiroty directory, called .gitlab-ci.yml with the following content

variables:
  REPOSITORY_URL: xxxxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/container

stages:
  - test
  - build
  - dev
  - stg
  - prd

test:
  stage: test
  script:
    - echo 'Here you can run tests'

build:
  stage: build
  script:
    - echo 'After Test stage was successful, here you can run build your container'

dev:
  stage: dev
  script:
    - echo 'After Build stage was successful, here you can run your Development environment deployment'

stg:
  stage: stg
  script:
    - echo 'After Dev stage was successful, here you can run your Staging environment deployment'

prd:
  stage: prd
  script:
    - echo 'After Stg stage was successful, here you can run your Production environment deployment'Copy to clipboardErrorCopiedCopy to clipboardErrorCopied

Here is a screenshot of what I did, and the commands will be below that you can copy and past.

cd test

nano .gitlab-ci.yml

git status

On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
    .gitlab-ci.yml

nothing added to commit but untracked files present (use "git add" to track)Copy to clipboardErrorCopiedCopy to clipboardErrorCopied

git add .gitlab-ci.yml

git commit -am "adding .gitlab-ci.yml pipeline file"

[main 0fe4e09] adding .gitlab-ci.yml pipeline file
 1 file changed, 35 insertions(+)
 create mode 100644 .gitlab-ci.ymlCopy to clipboardErrorCopiedCopy to clipboardErrorCopied

git push

Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 10 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 573 bytes | 573.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To ssh://localhost:32022/root/test.git
   29c151b..0fe4e09  main -> mainCopy to clipboardErrorCopiedCopy to clipboardErrorCopied

Now when you navigate back to the Gitlab Test Reposiroty and you refresh the page, you will see that our file have been added.

In the next section we will add a Runner to run our pipeline, enjoy!

Runners

https://docs.gitlab.com/runner/register/index.html#docker

Let's add a Gitlab runner for our test project, please navigate to Settings -> CI/CD

Now please expand Runners

and click on New Project Runner

You can leave everything default, but please Click Run Untagged Jobs

And click on Create Runner.

Remember that BUG I mentioned above? https://gitlab.com/gitlab-org/charts/gitlab/-/issues/4205

Well you will now get a blank page, because it redirects us to the URL: http://localhost/root/test/-/runners/1/register?platform=linux

BUT IT SHOULD ACTUALLY BE:

http://localhost:5580/root/test/-/runners/1/register?platform=linux

So please add the :5580 after localhost in the URL adrress bar and press enter

Register Runner

You see the text in Register Runner, Step 1? Copy that, mine looks like below, BUT LOK CLOSELY, You see the BUG again? It's missing the port!! :5580

WRONG!!

gitlab-runner register  --url http://localhost  --token glrt-NRYUnqLZ2yzyutC1MYVVCopy to clipboardErrorCopiedCopy to clipboardErrorCopied

So remember to add the port there so that it looks like

CORRECT!!

gitlab-runner register  --url http://localhost:5580  --token glrt-NRYUnqLZ2yzyutC1MYVVCopy to clipboardErrorCopiedCopy to clipboardErrorCopied

Now let's head over to our Hashiqube instance, in our hashiqube directory, we can do

vagrant ssh

And we'll register our runner in the Hashiqube instance, be sure to remember to add the port in there

Now you can do step 3 where we run the runner

gitlab-runner run

When you now go back to your Gitlab page you will see You've created a new runner!

You can now click on Go to the Runners page and you will see it is registered and ready to accept jobs

Run Pipeline

Ok, so we have a Test Project, We have a registered Runner and we have a Pipeline, so let's run our pipeline. Please navigate on your left to Build - Pipelines

and you will see in the next screen your pipeline is running

To see what the Pipeline Jobs did, please navigate to Jobs on the left and then click on one of those jobs

And that's it for now folks, thank you for taking the time to go through this tutorial with me and thank you for using Hashiqube, I hope you had fun!

Gitlab Provisioner

#!/bin/bash

# https://docs.gitlab.com/charts/installation/version_mappings.html#previous-chart-versions
GITLAB_HELM_CHART_VERSION=7.2.4 # Which is Gitlab v16.2.4

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Cleanup"
echo -e '\e[38;5;198m'"++++ "
sudo docker stop gitlab
sudo docker rm gitlab
sudo docker stop gitlab-runner
sudo docker rm gitlab-runner
yes | sudo docker system prune -a
yes | sudo docker system prune --volumes
sudo --preserve-env=PATH -u vagrant helm list
sudo --preserve-env=PATH -u vagrant helm uninstall gitlab
sudo --preserve-env=PATH -u vagrant helm list

arch=$(lscpu | grep "Architecture" | awk '{print $NF}')
if [[ $arch == x86_64* ]]; then
  ARCH="amd64"
elif  [[ $arch == aarch64 ]]; then
  ARCH="arm64"
fi
echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"CPU is $ARCH"
echo -e '\e[38;5;198m'"++++ "

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"Ensure Minikube is running"
echo -e '\e[38;5;198m'"++++ "
if pgrep -x "minikube" >/dev/null
then
  echo "Minikube is running"
else
  echo -e '\e[38;5;198m'"Minikube is not running, launching"
  sudo bash /vagrant/minikube/minikube.sh
fi

# https://docs.gitlab.com/charts/development/minikube/ Gitlab via minikube
# https://gitlab.com/gitlab-org/charts/gitlab
# https://docs.gitlab.com/ee/administration/troubleshooting/kubernetes_cheat_sheet.html
# https://docs.gitlab.com/runner/install/kubernetes.html

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ helm repo add gitlab https://charts.gitlab.io/"
echo -e '\e[38;5;198m'"++++ "
sudo --preserve-env=PATH -u vagrant helm repo add gitlab https://charts.gitlab.io/

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ helm repo update"
echo -e '\e[38;5;198m'"++++ "
sudo --preserve-env=PATH -u vagrant helm repo update

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ helm search repo gitlab"
echo -e '\e[38;5;198m'"++++ "
sudo --preserve-env=PATH -u vagrant helm search repo gitlab

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Launch Gitlab on minikube using Helm Charts"
echo -e '\e[38;5;198m'"++++ "

# BUG: https://gitlab.com/gitlab-org/charts/gitlab/-/issues/4205
# https://docs.gitlab.com/charts/charts/globals.html
# https://docs.gitlab.com/charts/charts/globals.html#configure-host-settings
# https://helm.sh/docs/helm/helm_upgrade/
# https://docs.gitlab.com/charts/installation/deployment.html
# https://docs.gitlab.com/charts/installation/command-line-options.html
# https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/doc/installation/command-line-options.md
echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Helm install gitlab"
echo -e '\e[38;5;198m'"++++ "
sudo --preserve-env=PATH -u vagrant helm install --version $GITLAB_HELM_CHART_VERSION \
  --namespace default gitlab \
  --timeout 600s \
  --set global.edition=ce \
  --set global.hosts.https=false \
  --set global.hosts.domain=localhost \
  --set global.hosts.hostSuffix="" \
  --set global.hosts.externalIP=$(sudo --preserve-env=PATH -u vagrant minikube ip) \
  --set global.hosts.gitlab.name=localhost \
  --set gitlab-runner.install=false \
  --set gitlab-runner.gitlabUrl="localhost:5580" \
  --set registry.enabled=false \
  --set gitlab.webservice.registry.enabled=false \
  --set gitlab.sidekiq.registry.enabled=false \
  --set gitlab-runner.runners.privileged=true \
  --set redis.resources.requests.memory=128Mi \
  -f https://gitlab.com/gitlab-org/charts/gitlab/raw/master/examples/values-minikube-minimum.yaml gitlab/gitlab

# INFO: Other flags I have tried during this process 
# --set global.hosts.domain=localhost \
# --set global.hosts.gitlab.name=localhost \
# --set global.hosts.gitlab.hostnameOverride=localhost \
# --set global.hosts.ssh=localhost \
# --set global.hosts.kas.name=localhost \
# --set global.hosts.minio.name=localhost \
# --set global.workhorse.host=localhost \
# --set global.webservice.serviceName=webservice-default \
# --set global.webservice.port=5580 \
# --set gitlab.webservice.service.workhorseExternalPort=5580 \
# --set global.hosts.shell.port=32022 \
# --set global.hosts.externalIP=$(sudo --preserve-env=PATH -u vagrant minikube ip) \
# --set global.hosts.gitlab.hostnameOverride=localhost \
# --set gitlab-runner.gitlabUrl=localhost:5580 \
# --set global.webservice.serviceName=localhost \
# --set global.workhorse.host=localhost \
# --set global.workhorse.serviceName=gitlab-webservice-default \
# --set global.hosts.gitlab.servicePort=5580 \
# --set global.workhorse.port=5580 \
# --set global.hosts.gitlab.name=localhost \
# --set global.hosts.gitlab.hostnameOverride=localhost \
# --set global.hosts.hostSuffix="" \

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Stretch your legs, get a coffee or a drink, ETA 10m.."
echo -e '\e[38;5;198m'"++++ Waiting for Gitlab to become available.."
echo -e '\e[38;5;198m'"++++ See Minikube Dashboard for details: http://localhost:10888"
echo -e '\e[38;5;198m'"++++ "

attempts=0
max_attempts=15
while ! ( sudo --preserve-env=PATH -u vagrant kubectl get po | grep gitlab-webservice | tr -s " " | cut -d " " -f3 | grep Running ) && (( $attempts < $max_attempts )); do
  attempts=$((attempts+1))
  sleep 60;
  echo -e '\e[38;5;198m'"++++ Waiting for Gitlab to become available, (${attempts}/${max_attempts}) sleep 60s"
  sudo --preserve-env=PATH -u vagrant kubectl get po | grep gitlab
  sudo --preserve-env=PATH -u vagrant kubectl get events | grep -e Memory -e OOM
done

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Waiting for Gitlab to stabalize, sleep 60s"
echo -e '\e[38;5;198m'"++++ "
sleep 30;
sudo --preserve-env=PATH -u vagrant kubectl get po | grep gitlab
sudo --preserve-env=PATH -u vagrant kubectl get events | grep -e Memory -e OOM

# https://gitlab.com/gitlab-org/charts/gitlab/-/issues/2572 Error 422
# https://stackoverflow.com/questions/67084554/how-to-kubectl-port-forward-gitlab-webservice
attempts=0
max_attempts=20
while ! ( sudo netstat -nlp | grep "0.0.0.0:5580" ) && (( $attempts < $max_attempts )); do
  attempts=$((attempts+1))
  sleep 60;
  echo -e '\e[38;5;198m'"++++ "
  echo -e '\e[38;5;198m'"++++ kubectl port-forward -n default service/gitlab-webservice-default 5580:8181 --address=\"0.0.0.0\", (${attempts}/${max_attempts}) sleep 60s"
  echo -e '\e[38;5;198m'"++++ "
  sudo --preserve-env=PATH -u vagrant kubectl port-forward -n default service/gitlab-webservice-default 5580:8181 --address="0.0.0.0" > /dev/null 2>&1 &
done

attempts=0
max_attempts=20
while ! ( sudo netstat -nlp | grep "0.0.0.0:80" ) && (( $attempts < $max_attempts )); do
  attempts=$((attempts+1))
  sleep 60;
  echo -e '\e[38;5;198m'"++++ "
  echo -e '\e[38;5;198m'"++++ kubectl port-forward -n default service/gitlab-webservice-default 80:8181 --address=\"0.0.0.0\", (${attempts}/${max_attempts}) sleep 60s"
  echo -e '\e[38;5;198m'"++++ "
  sudo --preserve-env=PATH -u vagrant kubectl port-forward -n default service/gitlab-webservice-default 80:8181 --address="0.0.0.0" > /dev/null 2>&1 &
done

attempts=0
max_attempts=20
while ! ( sudo netstat -nlp | grep "0.0.0.0:8181" ) && (( $attempts < $max_attempts )); do
  attempts=$((attempts+1))
  sleep 60;
  echo -e '\e[38;5;198m'"++++ "
  echo -e '\e[38;5;198m'"++++ kubectl port-forward -n default service/gitlab-webservice-default 8181:8181 --address=\"0.0.0.0\", (${attempts}/${max_attempts}) sleep 60s"
  echo -e '\e[38;5;198m'"++++ "
  sudo --preserve-env=PATH -u vagrant kubectl port-forward -n default service/gitlab-webservice-default 8181:8181 --address="0.0.0.0" > /dev/null 2>&1 &
done

# https://gitlab.com/gitlab-org/charts/gitlab/-/issues/2572 Error 422
# https://stackoverflow.com/questions/67084554/how-to-kubectl-port-forward-gitlab-webservice
attempts=0
max_attempts=20
while ! ( sudo netstat -nlp | grep 32022 ) && (( $attempts < $max_attempts )); do
  attempts=$((attempts+1))
  sleep 60;
  echo -e '\e[38;5;198m'"++++ "
  echo -e '\e[38;5;198m'"++++ kubectl port-forward -n default service/gitlab-gitlab-shell 32022:32022 --address=\"0.0.0.0\", (${attempts}/${max_attempts}) sleep 60s"
  echo -e '\e[38;5;198m'"++++ "
  sudo --preserve-env=PATH -u vagrant kubectl port-forward -n default service/gitlab-gitlab-shell 32022:32022 --address="0.0.0.0" > /dev/null 2>&1 &
done

attempts=0
max_attempts=30
while ! ( curl -s -w '%{http_code}' -o /dev/null "http://localhost:5580/users/sign_in" | grep 200) && (( $attempts < $max_attempts )); do
  attempts=$((attempts+1))
  sleep 60;
  echo -e '\e[38;5;198m'"++++ Waiting for Gitlab web interface to become ready, (${attempts}/${max_attempts}) sleep 60s"
  curl -s -w '%{http_code}' -o /dev/null "http://localhost:5580/users/sign_in"
done

# https://docs.gitlab.com/runner/install/linux-manually.html
echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Installing Gitlab-Runner"
echo -e '\e[38;5;198m'"++++ "
curl -s https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo DEBIAN_FRONTEND=noninteractive apt-get install -qq --allow-downgrades --assume-yes gitlab-runner=16.3.0 < /dev/null > /dev/null

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Gitlab CE http://localhost:5580 and login with Username: root and below password: "
sudo --preserve-env=PATH -u vagrant kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo

echo -e '\e[38;5;198m'"++++ "
echo -e '\e[38;5;198m'"++++ Please login to Gitlab and create a project called test"
echo -e '\e[38;5;198m'"++++ Please follow the rest of the instructions here: http://localhost:3333/#/gitlab/README?id=you-are-here"
echo -e '\e[38;5;198m'"++++ "Copy to clipboardErrorCopiedCopy to clipboardErrorCopied