Working With The VMware Cloud on AWS REST API and Python

Working With The VMware Cloud on AWS REST API and Python

· Read in about 11 min · (2252 words) ·

VMC API Developer Center

For those purely interested in the code example; here you go! GIST of VMware Cloud on AWS API Example Code - I have also created examples on the VMware Code Sample exchange here.

Introduction

I’ve been playing with the REST API for VMware Cloud on AWS a lot lately for a variety of reasons. This has given me a good view into a lot of use-cases where programmatic access would be preferred to the “point and click” of the GUI (although the GUI is pretty nice…that ClarityUI though!).

I tend to leverage Python for calling REST API’s as it’s an extremely easy and flexible language to get started with. It’s a quick setup, and typically I don’t need to install any additional modules. The “Requests” module, which is the most common module used for interacting with http(s) endpoints, is included in Python by default.

As a quick side note, VMware actually has a Python SDK (as well as Java) for interacting with the API directly without using REST. You can check it out here if you want to get a bit more Pythonic with your API adventures

Let’s jump in and get started!

Getting Started - Pulling your OAuth Key

In order to get started, you’ll need an active VMware Cloud Services account. With an active account we pull down an OAuth key which will allow us to authenticate to the API programmatically. This key is effectively your credentials for the environment, so you should probably guard this with your life. We’ll be “exporting” this key in linux so that we don’t have to statically place the key in our code.

To get our OAuth key, log into the https://console.cloud.vmware.com url and select your name on the top right, from the drop down - we’ll select “OAuth Refresh Token”.

OAuth Refresh Token

After this, we will select “New Token” or “Create a new token” on the OAuth Refresh Token page.

New Token

Remember what I mentioned before, this is a VERY important code. Store it safely, store it wisely! Now that we have this token, we can start diving into the service!

Constructing Our REST Calls

These examples will be done on Python 3.6 for Ubuntu in Windows. Yeah, I use the Linux shell in Windows. Come at me, haters :). We will use some F String’s in here - so it’s a good idea to know what those do. They are a handy way to make your code a little bit cleaner.

In Linux, you can use the export command to store variables in memory, and call them within Python using the OS module. The benefit to this is that they aren’t being stored long term in a static file somewhere where people can read them, and when the shell ends - it’s gone! In this case, we will do export oauthkey='yourkeydigitshere' and then launch our Python shell. F strings were added in Python 3.6, so if you aren’t running that you’ll want to statically assign the variables and concatenate as needed!

import requests 
import os 

oauth = os.environ['oauthkey']
baseurl = 'https://console.cloud.vmware.com/csp/gateway'
uri = '/am/api/auth/api-tokens/authorize'
headers = {'Content-Type':'application/json'}
payload = {'refresh_token': oauth}
r = requests.post(f'{baseurl}{uri}', headers = headers, params = payload)
print(r.status_code)

Taking a step back to explain whats happening here…

  • We import the requests module to work with our API endpoints, and we import the os module to interact with environment variables. Both of these modules have extensive capabilities outside of what I just mentioned - but this is what we are using them for in this post
  • We setup the oauth variable map to the environment variable that we set outside of python using the os.environ method
  • We establish baseurl as a variable, specifically as the “root” URL we are going to be working with.
  • We setup the uri variable as the API endpoint we are attempting to to call, in this case, we are calling the API endpoint for token authorization
  • We begin constructing our headers and our parameters. For our headers variable, we set our content type thats expected to ‘application/json’. We don’t actually NEED to do this; but setting a content type is best practice when working with API’s. We also set our “body” payload to be a JSON object containing our OAuth key, also referred to as a “refresh token” within VMware Cloud
  • Finally, we print out the status code of the return, so we know if we got in. We’re expecting a status code of “200” response. AnythingElse == ‘bad’. See what I did there?

How did I know what URL to use for authentication? In the newest release of VMware Cloud on AWS, the team released the Developer Center, an entire UI built around the API integrations.

VMC API Developer Center

From within this view, we can browse to the “Authentication Operations” menu and use this to see details about the various possible calls to the authentication service.

VMC API Developer Center

Huge props to Alan Renouf and his team for getting this feature in - it’s a huge help to those of us that are trying to work with VMware Cloud Services from an API perspective! Now back to our scheduled programming…

When our “script” runs, with our OAuth key appropriately applied; we should get that 200 response code with no problem. If we print(r.json()) in this case, we will see our access token return, along with how long it takes to expire, the token type, and some permission related values. COOL! We’ve authenticated!

Python Shell

How can we make this better? Can’t we check for a failure before printing? Also, this works fine as a script - but what if we wanted to build something a little bit extensible? Or reuse this code throughout our program? Does it make sense for us

Improving our API Login - “if” Statements and Functions

We can perform proactive checks using an if statement to check and see “if” the response is what we want it to be. As you might imagine, for the un-indoctrinated, using an if statement is a lot like saying “if something is THIS, do SOMETHING, otherwise we can do THAT”. In this case, the update is quite simple. We can add the following block to our code to make it check during the login process, and return an error if it fails to login successfully.

if r.status_code != 200:
    print(f'Unsuccessful Login Attempt. Error code {r.status_code}')
else:
    print('Login successful. ') 

This is very easy to read, and an extremely common example of an “if” statement in Python with REST API’s. If the status code returned when “r” is executed is anything other than 200, print the error and tell us the code. Otherwise, print that the login was successful.

Also, best practice typically says that creating functions is a GOOD THING when it comes to code. This will let us run a simple command, such as “login”, while feeding it a value to successfully login and return our authentication credentials to us. The best part of this, we can setup the formatting of the returned response so that it’s in a format that we can feed into OTHER API calls.

def login():
    key = os.environ['oauthkey']
    baseurl = 'https://console.cloud.vmware.com/csp/gateway'
    uri = '/am/api/auth/api-tokens/authorize'
    headers = {'Content-Type':'application/json'}
    payload = {'refresh_token': key}
    r = requests.post(f'{baseurl}{uri}', headers = headers, params = payload)
    if r.status_code != 200:
        print(f'Unsuccessful Login Attmept. Error code {r.status_code}')
    else:
        print('Login successful. ')
        auth_json = r.json()['access_token']
        auth_Header = {'Content-Type':'application/json','csp-auth-token':auth_json}
        return auth_Header

The big gotcha here is being aware of your spacing. Python requires consistency in this. You can see that we do a simple def login() to start things off. This declares our function. Next, we implement the same code as our earlier script, with the addition of our status response check. Once our check passes successfully (200 response), we print out that the login was successful and we build our authentication header. To build this, we pull the “access_token” key from the response JSON, and build it into a new header object called “auth_Header” where we set our “content-type” and the “csp-auth-token” key. The return statement at the end “returns” this header when the call is successful.

Calling the VMware Cloud on AWS REST API

Now that we have a login function defined, we can get even more creative with our API call. The earlier base URL we were using is for the VMware Cloud Services API. One of the core components of this is the authentication service. What it does NOT include is individual Cloud Services, such as the VMware Cloud on AWS offering. For that, we need to use a DIFFERENT URL.

We’ve abstracted the majority of the login code now, which is going to drastically shorten our next calls. Lets try a simple one to list all Organizations that my VMwware Cloud Services user account is a member of.

If we look at the developer center, under the “Organizations” menu, we can see the URI we need to call to pull the Org data down.

Organization API

One thing not documented in the Developer Center currently (Feedback provided on getting this added! Thanks Alan!) is that the base URL for the VMware Cloud on AWS calls is “https://vmc.vmware.com/vmc/api". As you can see in the developer center, we need to append “/orgs” to the base URL. In order to call this API, we also need to include our authentication header in the actual call. With our previous login function we built, this makes things a lot simpler!

auth_header = login()
orgList = requests.get('https://vmc.vmware.com/vmc/api/orgs', headers = auth_header)
print(orgList.status_code)

Success! When we run this code, we’re able to see our status code returned is 200.

Authentication Login

The VMware Cloud on AWS API returns JSON so it’s very easy to parse out with Python. We can move through the returned JSON using a simple loop to provide us the name, and Organization ID for each of the objects. The organization ID is needed for calls that go deeper into VMware Cloud on AWS - so its kind of important.

auth_header = login()
orgList = requests.get('https://vmc.vmware.com/vmc/api/orgs', headers = auth_header)
for org in orgList.json():
    print("Org Name: " + org['display_name'])
    print("Org ID: " + org['id'])

We’ve introduced a for loop in here, as I mentioned above. We are basically saying “for every org in the returned orglist’s JSON object, print data from it”. We parse out our JSON, calling the specific keys attached to our for loop. When we fire off the call, using everything we’ve built so far - we are returned the below content! AWESOME! DARK MAGICS!

Org List API Result

“GET”ing our SDDC’s

For our final concept, we’re going to take the data we’ve called above and return an SDDC. We can see in the response that we were returned 2 Organizations from our previous call. We’re going to work with the second organization in that list. It’s a little bit deceiving if you’ve never worked with a scripting or coding language before, but 1 actually refers to the 2nd item in the array (0 being the first). Lets play with our code some more!

auth_header = login()
orgList = requests.get('https://vmc.vmware.com/vmc/api/orgs', headers = auth_header)
orgID = orgList.json()[1]['id']
print(orgID)

When we run this block - we can confirm that our OrgID is returned as expected. In order to “GET” and SDDC, we need to return to the developer center to understand the API URI we need to call.

Dev Center Get SDDCs

There’s a ton of options in the SDDCs menu, but we ultimately just want the main SDDCS URI. We can see from the call that e need to feed the organization id into the API URI. Good thing we established how to do that in the previous section! Since we’re confident this API call is going to work, lets also include

auth_header = login()
orgList = requests.get('https://vmc.vmware.com/vmc/api/orgs', headers = auth_header)
orgID = orgList.json()[1]['id']
sddcList = requests.get(f'https://vmc.vmware.com/vmc/api/orgs/{orgID}/sddcs', headers = auth_header)
if sddcList.status_code != 200:
    print('API Error')
else:
    for sddc in sddcList.json():
        print("SDDC Name: " + sddc['name'])
        print("SDDC Create Date: " + sddc['created'])
        print("SDDC Provider: " + sddc['provider'])
        print("SDDC Region: " + sddc['resource_config']['region'])

What a long way we’ve come; and now we’re at the end! With these modifications to our script, we are able to parse through the SDDC result an grab some basic data about the cluster SDDC thats been created. Use the f-string concept we talked about earlier to feed in our OrgID to the SDDC “GET” API. Once we get the value back, we parse throug the JSON and pull out the relevant information that we wanted. We use a for loop here - but ultimately there is only one SDDC in the object, so it only prints out once. If we had multiple SDDCs though, it would print them all!

Get SDDCs

Success! We’ve finally reached our actual vCenter cluster living in AWS and are able to return data from it!

Conclusion

There are a lot of advanced concepts being covered here - but you can see its very easy to get to a place where we are pulling relevant data out around our infrastructure. So far we’ve been exclusively focused on “GET” API calls, but there’s a whole lot of PUT, POST, and DELETE out there too that we haven’t touched yet.

Imagine crafting a Dispatch function that added hosts or removed them via FaaS? Imagine doing it in Lambda? vRealize Automation? Imagine deploying new datacenters when you on-boarded a new project. There are tons of options that we can play with.

Other Research For Over Achievers

  • Prettytables Python module to build attractive tables
  • Use the API to pull data around NSX components in VMware Cloud on AWS
  • Build out additional functions to further “parameterize” our code