Automating APIs with Python and Go (Part I)

Automating APIs with Python and Go (Part I)

Chris EvansCode, Data Protection, DevOps, Enterprise, HCI

I’ve been writing software since the late 1970s when I started out with BASIC on the Commodore PET.  Over the years I’ve written in dozens of languages from machine code to high-level bespoke languages like SAS.  I’ve even written niche solutions like an SNA to TCP/IP conversion tool in IBM mainframe assembler and a distributed file system using FUSE and C++. 

Although I was never a “professional coder”, at various times in my career, I’ve needed to write a script, develop some software tools or use code to improve the “day job”.  Probably the most recent of these requirements was to put automation and scripting around storage management.

This year, I’m looking to do more hands-on work and exploit the lab environments I have available.  In an analyst role, it’s easy to talk about technology without actually being involved in the challenges that accompany delivering solutions.  You’ll see a lot more focus in 2020 onwards in proving out the realities of software and hardware.  This direction will be biased towards Open Source and cloud-native solutions. 

APIs

I recently received an evaluation Scale Computing HE150 cluster, which is aimed at the Edge Computing market.  I’ve talked about Scale Computing before, and I recommend checking out the excellent Tech Field Day presentations from the company.  The HE150 cluster is managed through a GUI but also supports an unofficial API.  As part of my automation efforts, I’m looking to develop a set of CLI functions to drive the API more easily.

HYCU

I’m also looking at other solutions in the data centre that offer API management.  One of these is HYCU, a software-defined backup solution I’ve previously installed and written about.  Again, HYCU has an API that provides the ability to manage data protection through automation.  I wrote last year about how important APIs are for backup, so HYCU (and data protection automation in general) is also on the list.

REST

What do we mean by APIs in the context of Scale, HYCU and other solutions?   Typically, we’re referring to API calls that are implemented over HTTP(S) and so accessible across either a local network or the wider Internet.  The best features of REST APIs are that we don’t need any bespoke technology to implement them and that they operate with well-documented data structures like XML or JSON. 

[I once wrote some credit card authorisation software to process transactions for major retailers that relied on X.25 and an essentially undocumented API, so be thankful for XML & JSON, despite their shortcomings.]

Python & Go

Why choose Python and Go as the two main languages to code in?  Firstly, Python is everywhere and becoming a ‘standard’ tool for developers and others involved in coding ML/AI applications.  Python interpreters are available on every O/S platform, so code should run with little or no alteration across any deployment. 

I think the case for Go (or Golang) is a bit more interesting.  As a compiled language (compared to Python’s interpreted design), Go offers the opportunity to build efficient executables that are more easily download and consumed by a wider audience.  There’s an obvious trade-off here in supportability versus ease of use.  However, from an Enterprise perspective, the idea of immutable versioned code is generally a preferred route to follow.

GitHub

I will aim to share all of my code online with a Creative Commons model.  Feel free to steal, copy, amend and do what you like with the source.  However, I set only two requirements:

  • Please attribute where the source code originated, in case others want to learn more.
  • Use at your own risk.  I place no warrants or guarantees on the use of any of the code or software binaries produced. 

A Simple Example

So, let’s get on with it!  Here’s some simple code that will query a Scale Computing cluster (any that supports the API) and return a list of virtual machines.  This code is stored in “getVMs.py”.

import requests
import urllib3
# import libraries for HTTP calls and URL exception processing
#
# disable warnings for insecure certificate
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
#
# call API with fixed user/password & ignore no certificate
resp = requests.get('https://172.16.10.1/rest/v1/VirDomain', verify=False, auth = ('apiuser','PasswordChanged23'))
#
# check the response - < 400 is good, otherwise error out
#
if resp.status_code < 400:
  print("{0:40s}{1:10s}{2:8s}{3:12s}{4:10s}".format("VM Name","Status","CPUs","Memory (MB)","Operating System"))
  for vms in resp.json():
    print("{0:40s}{1:10s}{2:<8d}{3:<12d}{4:10s}".format(vms["name"],vms["state"],vms["numVCPU"],vms["mem"]/1048576,vms["operatingSystem"]))
else: 
  print("Error code {} received from API call.".format(resp.status_code))

In this example, I’ve been a little lazy and hard-coded the target URL and user/password combination (we’ll take these out in future development).  Key elements:

  • Python is supported by a rich set of libraries; we just import the ones that offer support for HTTP transactions.
  • Connectivity is managed through hard-coded values.  Most API calls offer a token approach that obfuscates the user/password combination (future enhancement)
  • The output from the API call is a structured JSON object consisting of multiple virtual machine objects.  The code iterates through them and prints a formatted line for each.
Output from getVMs.py python code

This code is an incredibly simple example but demonstrates the power of both API structures and Python.  Look out for future posts that will look at making the code parameter-driven and actually more useful!

Post #d670. Copyright (c) 2007-2020 Brookend Ltd. No reproduction in whole or part without permission.