Terratest Good Practices: Table-Driven Tests

Hello!

Terratest is a common way to run integration tests against terraform modules. I use it on many of the modules I develop. If you haven’t used it before, check out its quickstart for an example of how it works.

For simple cases, the pattern in that quickstart is all you need. But, bigger modules mean more tests and pretty soon you can end up swimming in all the cases you have to define. Go has a tool to help: table-driven tests. Here’s what you need to get them set up for terratest (Dave Cheney also has a great article on them if you want to go deeper).

First, let’s look at a couple simple tests that aren’t table-driven:

package test

import (
	"testing"

	"github.com/gruntwork-io/terratest/modules/terraform"
	"github.com/stretchr/testify/assert"
)

func TestOutputsExample(t *testing.T) {
	terraformOptions := &terraform.Options{
		TerraformDir: ".",
	}
	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)

	one := terraform.Output(t, terraformOptions, "One")
	assert.Equal(t, "First.", one)
	two := terraform.Output(t, terraformOptions, "Two")
	assert.Equal(t, "Second.", two)
}

Easy. Just repeat the calls to terraform.Output and assert.Equal for each test and assert it’s what you expect. Not a problem, unless you have dozens or hundreds of tests. Then you end up with a lot of duplication.

You can de-duplicate the repeated calls by defining your test cases in a slice of structs (the “table”) and then looping over the cases. Similar to adaptive modeling. Like this:

package test

import (
	"testing"

	"github.com/gruntwork-io/terratest/modules/terraform"
	"github.com/stretchr/testify/assert"
)

func TestOutputsTableDrivenExample(t *testing.T) {
	terraformOptions := &terraform.Options{
		TerraformDir: ".",
	}
	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)

	outputTests := []struct {
		outputName    string
		expectedValue string
	}{
		{"One", "First."},
		{"Two", "Second."},
	}

	for _, testCase := range outputTests {
		outputValue := terraform.Output(t, terraformOptions, testCase.outputName)
		assert.Equal(t, testCase.expectedValue, outputValue)
	}
}

Now, there’s just one statement each for terraform.Output and assert.Equal. With only two tests it actually takes a bit more code to use a table, but once you have a lot of tests it’ll save you.

That’s it! That’s all table-driven tests are. Just a routine practice in Go that work as well in terratest as anywhere.

Happy testing,

Adam

Need more than just this article? We’re available to consult.

You might also want to check out these related articles:

CloudWatch Logs: Preventing Orphaned Log Groups

Hello!

When you need to publish logs to CloudWatch (e.g. from a lambda function), you need an IAM role with access to CloudWatch. It’s tempting to use a simple policy like the one in the AWS docs. You might write a CloudFormation template like this:

# Don't use this!

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  DemoRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: '/'
      Policies:
      - PolicyName: lambda-logs
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:DescribeLogStreams
            - logs:PutLogEvents
            Resource: arn:aws:logs:*:*:*

  DemoFunction:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          def handler(event, context):
              print('Demo!')
      FunctionName: demo-function
      Handler: index.handler
      Role: !GetAtt DemoRole.Arn
      Runtime: python3.7

Obviously, the role is too permissive: arn:aws:logs:*:*:*

But, there’s another problem: it grants logs:CreateLogGroup.

Here’s what happens:

  1. Launch a stack from this template
  2. Run demo-function
  3. Because we granted it permission, demo-function automatically creates /aws/lambda/demo-function log group in CloudWatch Logs
  4. Delete the stack
  5. CloudFormation doesn’t delete the /aws/lambda/demo-function log group

CloudFormation doesn’t know about the function’s log group because it didn’t create that group, so it doesn’t know anything needs to be deleted. Unless an operator deletes it manually, it’ll live in the account forever.

It seems like we can fix that by having CloudFormation create the log group:

DemoLogGroup:
  Type: AWS::Logs::LogGroup
  Properties:
    LogGroupName: /aws/lambda/demo-function
    RetentionInDays: 30

But, if the function still has logs:CreateLogGroup I’ve seen race conditions where the stack deletes the group before the lambda function and the function recreates that group before it gets deleted.

Plus, there aren’t any errors if you forget to define the group in CF. The stack launches. The lambda function runs. We even get logs, they’ll just be orphaned if we ever delete the stack.

That’s why it’s a problem to grant logs:CreateLogGroup. It allows lambda (or EC2 or whatever else is logging) to log into unmanaged groups.

All resources in AWS should be managed by CloudFormation (or terraform or whatever resource manager you use). Including log groups. So, you should never grant logs:CreateLogGroup except to your resource manager. Nothing else should need that permission.

And that’s the other reason: lambda doesn’t need logs:CreateLogGroup because it should be logging to groups that already exist. You shouldn’t grant permissions that aren’t needed.

Here’s the best practice: always manage your CloudWatch Logs groups and never grant permission to create those groups except to your resource manager.

Happy automating!

Adam

Need more than just this article? We’re available to consult.

You might also want to check out these related articles:

Terraform: Get Data with Python

Update 2017-12-26: There’s now a more complete, step-by-step example of how to use terraform’s data resource, pip, and this decorator in the source.

Good morning!

Sometimes I have data I need to assemble during terraform’s apply phase, and I like to use Python helper scripts to do that. Awesomely, terraform natively supports using Python to populate the data resource:

data "external" "cars_count" {
  program = ["python", "${path.module}/get_cool_data.py"]

  query = {
    thing_to_count = "cars"
  }
}

output "cars_count" {
  value = "${data.external.cars_count.result.cars}"
}

A slick, easy way to drop out of terraform and use Python to grab what you need (although it can get you in to trouble if you abuse it).

The Python script has to follow a protocol that defines formats, error handling, etc. It’s minimal but it’s fiddly, plus if you need more than one external data script it’s better to modularize than copy and paste, so I wrote a pip-installable decorator that implements the protocol for you. The source is also an example you can follow if you’d rather implement it yourself than add a dependency. Here’s how you use it:

from terraform_external_data import terraform_external_data

@terraform_external_data
def get_cool_data(query):
    return {query['thing_to_count']: '3'}

if __name__ == '__main__':
    get_cool_data()

It’s available on PyPI, just pip install terraform_external_data.

Happy terraforming!

Adam

Need more than just this article? We’re available to consult.