Manage OpenStack using Terraform and GitLab

date

‣

slug

terraform-gitlab-openstack

status

Published

GitLab and Terraform state

The goal is to achieve collaboration through Terraform, so you need to have a centralized state file. GitLab has a managed state for Terraform. With this feature, you can enable individuals to manage OpenStack collaboratively.

Create a GitLab group and project

view all groups

Create a group by clicking on New group and then on Create group.

Create Group

Name the group to generate a unique group URL, and invite your team to work with you.

Name Group

After creating a group, create a project by clicking Create new project, and then Create blank project:

Create from blank project

Name your project. GitLab generates a unique project URL for you. This project contains the repository for your Terraform scripts and Terraform state.

Create a personal access token

The repository needs a personal access token to manage this Terraform state. In your profile, select Edit Profile:

Edit profile

Click Access Token in the side panel to access a menu for creating an access token. Save your token because you can't view it again.

Access Token

Clone the empty repository

On a computer with direct access to your OpenStack installation, clone the repository and then change to the resulting directory:

Create the backend .tf and provider file

Create a backend file to configure GitLab as your state backend:

This provider file pulls the provider for OpenStack:

Because you've declared a variable in the provider, you must declare it in a variable file:

Because you're initially working locally, you must set those variables to make it work:

These details are available on your rc file on OpenStack.

Initialize the project in Terraform

Initializing the project is quite different because you need to tell Terraform to use GitLab as your state backend:

To view the gitlab-project-id, look in the project details just above the Project Information tab in the side panel. It's usually your project name.

Project ID

For me, it's 42580143.

Use your username for gitlab-username. Mine is ajohnsc.

The gitlab-personal-access-token is the token you created earlier in this exercise. In this example, I use wwwwwwwwwwwwwwwwwwwww. You can name your-unique-state-name anything. I used homelab.

Here is my initialization script:

To use the file:

The output is similar to this:

terraform init

Test the Terraform script

This sets the size of the VMs for my OpenStack flavors:

The settings for my external network are as follows:

Router settings look like this:

Enter the following for images:

Here is a Demo tenant:

When complete, you will have this file structure:

Issue plan

After the files are complete, you can create the plan files with the terraform plan command:

After all plan files have been created, apply them with the terraform apply command:

After applying the infrastructure, return to GitLab and navigate to your project. Look in Infrastructure → Terraform to confirm that the state homelab has been created.

Gitlab State file

Destroy the state to test CI

Now that you've created a state, try destroying the infrastructure so you can apply the CI pipeline later. Of course, this is purely for moving from Terraform CLI to a Pipeline. If you have an existing infrastructure, you can skip this step.

You now have a state everyone can use. You can provision using a centralized state. With the proper pipeline, you can automate common tasks.

Set up a GitLab runner

Your OpenStack cluster isn't public-facing, and the OpenStack API isn't exposed. You must have a GitLab runner to run GitLab pipelines. GitLab runners are services or agents that run and perform tasks on the remote GitLab server.

On a computer on a different network, create a container for a GitLab runner:

Now register it with your project in your GitLab project's Settings → CI/CD panel:

Gitlab runner register

Scroll down to Runners → Collapse:

Gitlab runner registration

The GitLab runner registration token and URL are required. Disable the shared runner on the right side to ensure it works on the runner only. Run the gitlab-runner container to register the runner:

Upon success, your GitLab interface displays your runner as valid. It looks like this:

Specific Runner

You can now use that runner to automate provisioning with a CI/CD pipeline in GitLab.

Set up the GitLab pipeline

Now you can set up a pipeline. Add a file named .gitlab-ci.yaml in your repository to define your CI/CD steps. Ignore the files you don't need, like .terraform directories and sensitive data like variable files.

Here's my .gitignore file:

Here are my CI pipeline entries in .gitlab-ci.yaml:

The process starts by declaring that every step and stage is under the homelab tag, allowing your GitLab runner to run it.

Next, the variables are set on the pipeline. The variables are only present when the pipeline is running:

There's a cache that saves specific files and directories upon running from stage to stage:

These are the stages that the pipeline follows:

This declares what to do before any stages are run:

In the prepare stage, the tf-init initializes the Terraform scripts, gets the provider, and sets its backend to GitLab. Variables that aren't declared yet are added as environment variables later.

In this part, the CI job tf-validate and the stage validate run Terraform to validate that the Terraform scripts are free of syntax errors. Variables not yet declared are added as environment variables later.

Next, the CI job tf-build with the stage build creates the plan file using terraform plan and temporarily saves it using the artifacts tag.

In the next section, the CI job tf-deploy with the stage deploy applies the plan file.

There are variables, so you must declare them in Settings → CI/CD → Variables → Expand.

Add all the variables required:

So for this example, I used the following:

And it is masked GitLab for its protection.

The last step is to push the new files to the repository:

View the results

View your new pipelines in the CI/CD section of GitLab.

On the OpenStack side, you can see the resources created by Terraform.

The networks:

The flavors:

The images:

The project:

The user:

Next steps

Terraform has so much potential. Terraform and Ansible are great together. In my next article, I'll demonstrate how Ansible can work with OpenStack

reference:

https://opensource.com/article/23/2/manage-openstack-using-terraform-and-gitlab