Scheduled Severless Startup
Using AWS lambda to automate boot up and down of a Digital Ocean droplet.
The Problem
As part of the early Covid-19 response, Digital Ocean donated some free credit to us to host an app for a local food delivery scheme.
To make that credit go as far as possible and to minimise power consumption, we'd like to power up and down the servers according to a schedule.
The Solution
Here's how to do that with AWS Lambda, with cloudfront events. We iterate on that to use the Serverless Framework.
TLDR; Have a look at the companion repo.. It contains an example dockerized web app and the shell scripts for starting containers and creating the service.
This article covers how to automate with a Digital Ocean droplet so to follow along with the code, you can create one using the example from a previous article.
We will examine the cURL statements, convert those into Python. We can use AWS Lambda to execute the Python in a serverless environment. Then, the AWS lambda functions can be triggered by Cloudwatch events to a schedule so machines can be brought online only during operating hours.
We can notify any interested parties using Microsoft Teams - a topic for a subsequent post.
Prerequisites
You can always check the companion repo.
You will need to:
- .. have
jq
installed to format JSON responses- Download JQ for your OS
- .. have a server on which a dockerized web app will start automatically when the server is restarted
- See previous article on how to create one with DigitalOcean.
- .. have installed Boto3 with your AWS account credentials so we can work with the AWS Python SDK.
cURL statements to boot down and up
As covered in a previous post, our web app will restart when rebooted using a systemd service.
Rather than ssh or the cloud provider's control panel, the server can be started and stopped using cURL.
In the case of DigitalOcean:
Find out the ID of the server:
$ curl \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer '$DIGITAL_OCEAN_ACCESS_TOKEN'' \
"https://api.digitalocean.com/v2/droplets?name=scheduled-serverless" | jq '.droplets[] | {id:.id, name:.name, status: .status}'
The output gives us the ID
{
...
},
{
"id": 195786885,
"name": "scheduled-serverless",
"status": "active"
}
And then power it down:
$ curl -X POST \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer '$DIGITAL_OCEAN_ACCESS_TOKEN'' \
-d '{"type":"power_off"}' \
"https://api.digitalocean.com/v2/droplets/195786885/actions" | jq '.[] | {id:.id, status:.status, type:.type}'
Which tells us it's in progress
{
"id": 955320153,
"status": "in-progress",
"type": "power_off"
}
The equivalent cURL for powering on the server is virtual identical
$ curl -X POST \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer '$DIGITAL_OCEAN_ACCESS_TOKEN'' \
-d '{"type":"power_on"}' \
"https://api.digitalocean.com/v2/droplets/195786885/actions" | jq '.[] | {id:.id, status:.status, type:.type}'
The web app from the example is running on port 80. We know the server's IP address by referring to the control panel. Let's confirm that the service is running by verifying the output in a browser.
import os
import sys
import requests
TOKEN = os.getenv('DIGITAL_OCEAN_ACCESS_TOKEN','')
DROPLET_NAME = 'scheduled-serverless'
headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {TOKEN}',
}
if TOKEN == '':
sys.exit('Your environment should have the `DIGITAL_OCEAN_ACCESS_TOKEN exported.')
def get_droplet():
response = requests.get('https://api.digitalocean.com/v2/droplets', headers=headers, params={})
return [d for d in response.json().get('droplets',{}) if d.get('name','') == DROPLET_NAME][0]
DROPLET = get_droplet()
print(f'Droplet ID: {DROPLET.get("id")}')
Now we have the ID, let's define the functions for bringing the servers down and back up.
def power_off() -> dict:
data = '{"type":"power_off"}'
url = f'https://api.digitalocean.com/v2/droplets/{DROPLET.get("id")}/actions'
response = requests.post(url, headers=headers, data=data)
return response.json()
def power_on() -> dict:
data = '{"type":"power_on"}'
url = f'https://api.digitalocean.com/v2/droplets/{DROPLET.get("id")}/actions'
response = requests.post(url, headers=headers, data=data)
return response.json()
action_resp = power_off()
print(action_resp['action']['type'])
print(action_resp['action']['status'])
print('----')
# action_resp = power_on()
# print(action_resp['action']['type'])
# print(action_resp['action']['status'])
# Popping on a new on/off timestamp on a firebase stored stack
import requests
import arrow
start: dict = {'off': ['2020-08-11 09:04'], 'on': ['2020-08-11 09:03']}
# get data
url = 'https://wordsdothowappeddotcom.firebaseio.com/scheduled-serverless-startup.json?print=pretty'
r = requests.get(url, data={}, headers={},)
data = r.json()
print(data)
# replace data
print(data['off'][0])
Quality of Life enhancements
When the target service is unavailable, it's quite unfriendly to show a blank 500 screen. A subsequent post outlines a solution for a forwarding service which can display an Open/Closed for business page to the user, with 'Opening times'
As part of what's become known as 'Chatops', it's useful to update team members and stakeholders about the status of the server. A future addition will be to use the Microsoft Teams API to notify interested parties.
Resources used: with thanks 💚
- Curl converter Github repo
-
AWS Lambda and Secret Management Overview Blog post from Espagon
-
Sharing Secrets with AWS Lambda Using AWS Systems Manager Parameter Store AWS Compute blog post