Infrastructure as Code with Pulumi and vSphere
humblelab TheHumbleLab

Infrastructure as Code with Pulumi and vSphere

· Read in about 15 min · (3176 words) ·
IaC vSphere Pulumi

NOTE: The final state of the resources created in this blog post are located on my GitHub. Feel free to hijack, reuse, do something interesting! The good stuff is in index.ts in case you don't read the rest of this post :)

What This Blog Post Is Not

Blogging about vendor tech is always tricky. You throw a blog up, and people have a tendency to think you're telling everyone THE RIGHT way to do it. I'm an advocate for tools in the user's tool belt. This post, like many of my posts, is about a tool you can use. I'm not slinging mud on any product. I'm a nerd who loves tech. This post is not me telling you what you have to use to be successful; it's telling you about something fun you can play with.

And now, play on!

Introduction

I've been doing a lot with Pulumi over the past few months. I really enjoy the interaction model of creating my Infrastructure as Code using native languages that I've worked with (i.e. TypeScript and Python most commonly). Pulumi has language bindings across all of the major cloud providers, many of them quickly stood up using a Terraform bridge that allows them to quickly prototype new Pulumi bindings from existing providers. This approach allows Pulumi to very quickly bring providers that already have been built into functional programming languages.

Just to be clear - not all providers are created using the Terraform bridge. For example, the Pulumi Kubernetes provider which Joe Beda did a great TGI Kubernetes on, and their set of AWS providers (named Pulumi Crosswalk) were designed from the ground up.

For the purposes of this post, I wanted to give a tour of getting started with the vSphere provider. The vast majority of vSphere customers still deploy machines using “Right Click/Create New VM”. There's millions of better ways to do this - today, we'll do it in Pulumi!

Getting Started

Before anything, we'll want to go sign up for an account at Pulumi's website. There's a few things that this opens up. First, Pulumi by default uses a remote state. A state file is generated whenever a Pulumi program is run and represents the state of the deployed components from your actual program. There are a number of reasons why you might want to store remotely vs locally - stories for another time.

Another thing that signing on Pulumi's website gives is the ability to track the status and resources associated with your current stack (you can think of this as a deployment). You can quickly view the associated outputs, configuration details, run activities, deployed resources and more. Summed up. It's a good idea to sign up. Using the website we can also store secrets remotely as needed.

Another cool thing I want to explicitly call out is that within the Pulumi UI, you can easily view all properties of deployed objects as well as their dependent objects. This makes it easier when we're building outputs later on!

Getting back to the setup… we'll also need to use brew to install the Pulumi CLI. We do this using homebrew with brew install pulumi. As mentioned earlier, Pulumi has language bindings for a number of native languages

  • JavaScript
  • TypeScript
  • Golang
  • Python

In the case of this project, I've decided to work with the TypeScript bindings. I'll also want need to install NodeJS using brew install NodeJS. Finally - with Node installed, we're going to run an npm install @pulumi/vsphere to install the Pulumi provider.

With all of this in place, we can start building our first Pulumi program!

Building Our Program

Pulumi refers to the Infrastructure as Code that deploys your applications as “programs”, since they are ultimately compiled code at the end of the day. From your terminal create a new directory, move into that directory, and issue the new program command.

mkdir pulumi-on-vsphere; cd pulumi-on-vsphere
pulumi new typescript

You'll be prompted to configure your Pulumi install, which will prompt you for a few items that are needed including an API token, and some naming components for your stacks. Once this is completed, Pulumi will scroll through scaffolding your program. When that's done, you should see the following structure in your directory.

Pulumi.yaml
index.ts
node_modules/
package-lock.json
package.json
tsconfig.json

The main file we're going to work in is index.ts, however before that, we need to configure the details needed to connect to our vSphere endpoint. To do this, we'll use the pulumi config set command.

pulumi config set vsphere:allowUnverifiedSsl true
pulumi config set vsphere:password VMware123! --secret
pulumi config set vsphere:user administrator@vsphere.local
pulumi config set vsphere:vsphere_server hlcorevc01.humblelab.com

This will result in configuration file being generated within your directory. In my environment, it's Pulumi.dev.yaml. Within that, you can see all configuration items you've set. Since we specified secret on our vSphere password, it's encrypted.

From here, i'll open the index.ts and take it for a spin. Right now, all it has in it is import * as pulumi from "@pulumi/pulumi";. Well thats pretty boring. Let's do something mildly interesting, and create a new folder!

Edit the file, and add the following import line for the vSphere package.

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

In order to create a folder, Pulumi requires the Datacenter ID. This is a result of the underlying Terraform provider requiring this information (and ultimately, the API for vCenter requiring it). We don't know this off the top of our heads (although it's pretty easy to find out) - but we can use Pulumi to get this for us by using the pulumi.output method.

Update the file to match the below

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

let dc = pulumi.output(vsphere.getDatacenter({
    name: "Core"
}));

export let dcid = dc.id

First, we're declaring a variable called dc that will hold the results of the following execution. Pulumi.output is a helper for pulling data about objects “out”. Were using the vSphere package we imported, and calling the “getDatacenter” method. In VS Code (or any mature IDE), you can hold your mouse over the vsphere.getDatacenter code to see the required information for execution. As shown below, you should see example usage for the data source in question.

If we wanted to see more complex data to construct our own calls out of (outside of the basic example provided), we could right click on the getDatacenter portion of the function in VS Code, and select “Peek Definition” which will let us see the real requirements of the function in question.

You should get used to the concept of “peeking the definition”. This is extremely useful across all of the Pulumi providers and will give you a path to head down when you're unsure which properties are needed. In the screenshot above, we can see that a required argument is GetDataCenterArgs. Within that interface, we can see that it's looking for “name” which will be a string. We can also see code comment indicating additional details.

Getting back to the code, we're diving a bit into functional programming now. The variable “dc” that's been established actually “holds” a number of properties within it (hence it being an object). In this case, we're exporting the “id” value, which will tell Pulumi to display this value to back to us. We export a new variable named dcid, and tell it to show us the dc.id value.

If you run a pulumi up this should execute and return us the datacenter ID. This concept is going to be more useful later, but for now go ahead and delete it. It's not super useful to return this as an output.

This is a very simple example, because we're just “getting” an existing resource. Let's chain this into another call that allows us to create something useful, like a folder!

Creating a Folder with Pulumi

Diving more into functional programming, we're going to use something known as a “constructor” to build a new object. The constructor in this case is the new vSphere.folder() constructor. This constructor has a number of requirements, which you can again see by starting to type your code, and then holding your mouse over the vsphere.Folder portion for more details.

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

let dc = pulumi.output(vsphere.getDatacenter({
    name: "Core"
}));

let folder = new vsphere.Folder()

Immediately we'll see that the code is unhappy (assuming we're working in an IDE).

Again, diving into functional programming, we're using something known as a “constructor” to build a new object. The constructor in this case is the new vSphere.folder() function. This constructor has a number of requirements, which you can again see by holding your mouse over the vsphere.Folder code.

The constructor requires a few components to build the object, which are called out when we highlight. A name (which is specifically called out indicating an argument for ‘name’ was not provided), the Folder Arguments, and any other custom resources options from Pulumi that we want to include. We're not including any custom resources, but we absolutely need to include our Folder Arguments. You can either look these up on the vSphere Terraform Provider or choose to peek the definition as I talked about above.

By the definition, we need to provide a name (“Pulumi Builds” in my case), Datacenter ID, Path, and the Folder type. Let's update our program to include these components.

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

let dc = pulumi.output(vsphere.getDatacenter({
    name: "Core"
}));

let folder = new vsphere.Folder("Pulumi Builds", {
    datacenterId: dc.apply(dc => dc.id),
    path: "Pulumi Builds",
    type: "vm",
    });

In the above example, we're telling the provider that the datacenterId key is populated by the results of the dc variable after is created (hence the ‘.apply’). The arrow function (=>) tells the program that the result of this value is going to be the .id property of the resulting variable after it's computed by the program.

When we run a pulumi up this is compiled and executed resulting in a folder creation.

Our pulumi up command shows that our resource was created, and gives us a link to the actual stack within the Pulumi website that we can view the history.

Let's fast forward onto a more complex example!

Creating a Virtual Machine with Pulumi

There are a number of values we need to satisfy in order to create a Virtual Machine with Pulumi…

  • Resource Pool
  • Datastore
  • Folders
  • Template (for clone operations)
  • Network
  • Disk configuration
  • Virtual Machine configurations (i.e. DNS, gateways, etc…)

Many of these properties can be discovered the same way as the Datacenter ID example we did above were discovered (by calling the appropriate data source). Updating our program with the below detail detail to capture the needed data. Remember as you build your data sources to use the peek definition ability often to find your way.

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

let dc = pulumi.output(vsphere.getDatacenter({
    name: "Core"
}));

let folder = new vsphere.Folder("Pulumi Builds", {
    datacenterId: dc.apply(dc => dc.id),
    path: "Pulumi Builds",
    type: "vm",
    });

let cluster = dc.apply(dc => vsphere.getComputeCluster({
    datacenterId: dc.id,
    name: "Tenant"
}));

In my case, I'm using a cluster with the name of Tenant that lives within the Core datacenter. We're going to call a constructors to create a resource pool for the machine to live in as part of the stack deployment. We'll call this resource pool “Pulumi Resources”.

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

let dc = pulumi.output(vsphere.getDatacenter({
    name: "Core"
}));

let folder = new vsphere.Folder("Pulumi Builds", {
    datacenterId: dc.apply(dc => dc.id),
    path: "Pulumi Builds",
    type: "vm",
    });

let cluster = dc.apply(dc => vsphere.getComputeCluster({
    datacenterId: dc.id,
    name: "Tenant"
}));

let resourcePool = new vsphere.ResourcePool("Pulumi Resources", {
    parentResourcePoolId: cluster.apply(cluster => cluster.resourcePoolId),
});

Similar to before with our folder, we're calling a constructor for a resource pool creation. Based on resource definition, we give it a name and tell it to create the resource pool against our earlier defined cluster. We feed it the resource pool ID from the Tenant cluster (using the same methods we've picked up along the way). If we run this program using pulumi up -y it now creates the resource pool.

Something interesting you might notice is that we're iterating on the existing deployment. We don't need to create a new folder each time. Pulumi (thanks in this specific case to the underlying Terraform provider) knows that the folder has already been created, but also can see that the resource pool is missing and creates it. Welcome to desired state configuration!

Another fun experiment with this is to change the name of a deployed resource, and rerun the program. You'll notice the resource is automatically deleted and recreated to match the desired state. All of these changes are then tracked on pulumi.com as part of your stack (as we can see in the screenshot I included of the run).

We're going to kick this into high-gear by quickly grabbing the remaining values we need

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

let dc = pulumi.output(vsphere.getDatacenter({
    name: "Core"
}));

let folder = new vsphere.Folder("Pulumi Builds", {
    datacenterId: dc.apply(dc => dc.id),
    path: "Pulumi Builds",
    type: "vm",
    });

let cluster = dc.apply(dc => vsphere.getComputeCluster({
    datacenterId: dc.id,
    name: "Tenant"
}));

let resourcePool = new vsphere.ResourcePool("Pulumi Resources", {
    parentResourcePoolId: cluster.apply(cluster => cluster.resourcePoolId),
});

let datastoreId = dc.apply(dc => vsphere.getDatastore({
    datacenterId: dc.id,
    name: "vsanDatastore"
}));

let networkId = dc.apply(dc => vsphere.getNetwork({
    datacenterId: dc.id,
    name: "VM Network"
}));

let template = dc.apply(dc => vsphere.getVirtualMachine({
    datacenterId: dc.id,
    name: "vsan_ubuntu_18"
}));

We collect our target Datastore (vSAN in this case), our target Network (VM Network), and the template we want to consume (vsan_ubuntu_18). With these values we can move on to actually creating our virtual machine. For simplicity, I've included my baseline build below. While this is still written in typescript, using Pulumi, all of these primitives are based on the vSphere Terraform provider. As I mentioned previously, this means you can use the Terraform documentation directly for the necessary properties - or follow the steps I gave previously around looking at the actual constructor code by peeking the definition (are you seeing a consistent theme here yet?).

let master01 = new vsphere.VirtualMachine("pulumi01", {
    resourcePoolId: resourcePool.id,
    datastoreId: datastoreId.id,
    folder: masters.path,
    numCpus: 2,
    memory: 2048,
    guestId: template.guestId,
    networkInterfaces: [{
        networkId: networkId.id,
        adapterType: template.networkInterfaceTypes[0],
    }],
    disks: [{
        label: "disk0",
        size: template.disks[0].size,
        eagerlyScrub: template.disks[0].eagerlyScrub,
        thinProvisioned: template.disks[0].thinProvisioned,
    }],
    clone: {
        templateUuid: template.id,
        customize: {
            dnsServerLists: ["192.168.1.5"],
            dnsSuffixLists: ["humblelab.com"],
            ipv4Gateway: "192.168.1.1",
            linuxOptions: {
                domain: "humblelab.com",
                hostName: "master01"
            },
            networkInterfaces: [{
                dnsDomain: "humblelab.com",
                dnsServerLists: ["192.168.1.5"]
            }]
        }
    },
});

Let's break down what's happening above. First, we create a new vsphere Virtual Machine using the constructor. This constructor requires several properties to create successfully (as we discussed previously).

We fill in the necessary values we gathered earlier - Resource Pool, Datastore, Network, Folder, and the type of template we're using. We configure our network interfaces, which are an array. Since we're only setting up 1 interface, there's only one object provided - which includes the network ID and the type of adapter were using. We then configure our disks - including the specific disk configuration we want (gathering some of this information directly form the template).

Finally, we configure our clone operations. We call out our specific template's ID, and then fill in the data for our customization specification. This includes values like our DNS settings, default gateway, domain, and hostname.

Let's take a look at the completed Pulumi program now..

import * as pulumi from "@pulumi/pulumi";
import * as vsphere from "@pulumi/vsphere"

let dc = pulumi.output(vsphere.getDatacenter({
    name: "Core"
}));

let folder = new vsphere.Folder("Pulumi Builds", {
    datacenterId: dc.apply(dc => dc.id),
    path: "Pulumi Builds",
    type: "vm",
    });

let cluster = dc.apply(dc => vsphere.getComputeCluster({
    datacenterId: dc.id,
    name: "Tenant"
}));

let resourcePool = new vsphere.ResourcePool("Pulumi Resources", {
    parentResourcePoolId: cluster.apply(cluster => cluster.resourcePoolId),
});

let datastoreId = dc.apply(dc => vsphere.getDatastore({
    datacenterId: dc.id,
    name: "vsanDatastore"
}));

let networkId = dc.apply(dc => vsphere.getNetwork({
    datacenterId: dc.id,
    name: "VM Network"
}));

let template = dc.apply(dc => vsphere.getVirtualMachine({
    datacenterId: dc.id,
    name: "vsan_ubuntu_18"
}));

let pulumivm = new vsphere.VirtualMachine("pulumi01", {
    resourcePoolId: resourcePool.id,
    datastoreId: datastoreId.id,
    folder: folder.path,
    numCpus: 2,
    memory: 2048,
    guestId: template.guestId,
    networkInterfaces: [{
        networkId: networkId.id,
        adapterType: template.networkInterfaceTypes[0],
    }],
    disks: [{
        label: "disk0",
        size: template.disks[0].size,
        eagerlyScrub: template.disks[0].eagerlyScrub,
        thinProvisioned: template.disks[0].thinProvisioned,
    }],
    clone: {
        templateUuid: template.id,
        customize: {
            dnsServerLists: ["192.168.1.5"],
            dnsSuffixLists: ["humblelab.com"],
            ipv4Gateway: "192.168.1.1",
            linuxOptions: {
                domain: "humblelab.com",
                hostName: "master01"
            },
            networkInterfaces: [{
                dnsDomain: "humblelab.com",
                dnsServerLists: ["192.168.1.5"]
            }]
        }
    },
});

We will save, and execute our pulumi up -y. While it runs, you should be able to watch live as the object is created (as shown in the screenshot below).

If you open vSphere, you should see the machine creating using the specifications you outlined. Upon completion, you'll receive a message indicating the build is done - but we don't know any of the connectivity details, lets add 1 more thing the bottom of our program…

export let pulumivmip = pulumivm.defaultIpAddress

Save, and execute a pulumi up -y one last time. You should see the stack update to include a new output value (that could subsequently be used in other programs) of the VM's IP address.

In VS Code, you should have been able to see other property types that could be returned in addition to the defaultIpAddress. Experiment with returns and see what you can find!

Wrapping Up

I've been working in the “machine and platform deployment” business for a very long time, as many of us have. I've done it manually, I've scripted it, I've used the API's directly, I've used overlay management tools. What continues to blow my mind is that the majority of customers still deploy machines in the least efficient way possible - Right Click and Deploy new VM from vCenter's interface.

Infrastructure as Code tools like Pulumi (and others in the space, i.e. Terraform from HashiCorp) add a ton of value on top of traditional deployment methods. Being able to store your infrastructure deployments in source control (i.e. GitHub/GitLab) opens up the door to a number of collaboration concepts. When you partner this in with some form of a CI/CD tool (i.e. CircleCI) - thing's get really interesting. Being able to submit a PR to change an infrastructure deployment, and having something like CircleCI automatically deploy the resources when changes are merged creates a pretty powerful platform for delivering infrastructure ot end users.

From a Pulumi specific perspective - working directly with native languages brings all the power of working directly in real code. Being able to store configuration values, and drive decisions on the way workloads are provisioned off of those values is pretty awesome. What if you want to deploy to a specific network based on defined configuration values? What if you want to be able to loop over an array and make deployment decisions based on the configurations? What if you want to take the programs you build, and include them in other higher level functions? All of this becomes possible when working in functional programming.