Simple reusable Terraform modules for AWS VPC and server group

In earlier blogs, we developed simple Terraform code to create an AWS VPC and a set of three EC2 instances, intended to run some testing of the MongoDB community collection for Ansible. Here we will convert these into reusable Terraform modules, so the implementation will be more flexible to try different options without duplicating lots of code. With the working code we already have, this is quite easy!

Design

We will simply move the existing code into a new directory structure and choose which variables to break out for input to the module. We can name the modules “aws-simple-vpc” and “aws-server-group” to reflect ideas on the level of abstraction. We choose how much logic to build into each module and the interface.

Existing directories:

New directories:

Interface for aws-simple-vpc module

Looking through the code, we can pull out the following variables:

vpc_name
vpc_cidr_block
vpc_subnets (list of objects)
- cidr_block
- availability_zone

variable "vpc_name" {
  description = "Name of the VPC"
  type = string
  default = "labvpc"
}

variable "vpc_cidr_block" {
  description = "CIDR block for the VPC"
  default = "10.54.0.0/16"
}

variable "vpc_subnets" {
  description = "List of subnets in the VPC"
  type = list(object({
      cidr_block = string
      availability_zone = string
  }))
}

Interface for aws-server-group module

We can observe that the module does not need to be specific to MongoDB servers and pull out the following generalized variables:

vpc_name
ami_id
instance_type
key_name
server_name (prefix to add serial number from server count)
server_count (support any number of servers)
server_offset (allow names to start at a higher number)
app_port_start
app_port_end
app_cidr_block

variable "vpc_name" {
  description = "Name of the VPC"
  type = string
  default = "labvpc"
}

variable "ami_id" {
  description = "AMI for servcer creation"
}

variable "instance_type" {
  description = "EC2 instance type for server"
  type = string
  default = "t3.micro"
}

variable "key_name" {
  description = "Existing key pair for server"
  type = string
}

variable "server_name" {
  description = "Server name prefix"
  type = string
}

variable "server_count" {
  description = "Start of port range for application access"
  type = number
  default = 1
}

variable "server_offset" {
  description = "Starting number for server names"
  type = number
  default = 0
}

variable "app_port_start" {
  description = "Start of port range for application access"
  type = number
  default = 0
}

variable "app_port_end" {
  description = "Start of port range for application access"
  type = number
  default = 0
}

variable "app_cidr_block" {
  description = "CIDR block for the application access"
  default = "0.0.0.0/0"
}

Code for Reusable Modules

The provider definition moves out of the module into the caller, and we substitute the variables in place of hard-coded values in the resource definitions. We will review a few of the interesting code changes here and refer to the public repository for the full implementation.

Code Highlights for aws-simple-vpc

Instead of defining the provider, indicate the provider requirement.

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "3.25"
    }
  }
}

Instead of enumerating just three subnets as in the previous implementation, use a count based on the input variable.

# Subnets
resource "aws_subnet" "public" {
  count = length(var.vpc_subnets)
  vpc_id = aws_vpc.vpc.id
  cidr_block = var.vpc_subnets[count.index].cidr_block
  availability_zone = var.vpc_subnets[count.index].availability_zone
  map_public_ip_on_launch = "true"

  tags = {
    Network = "Public"
    Name = "${var.vpc_name}-public-${var.vpc_subnets[count.index].availability_zone}"
  }
}
[...]
# Subnet Associations to Route Table
resource "aws_route_table_association" "srta" {
  count = length(var.vpc_subnets)

  subnet_id = element(aws_subnet.public.*.id, count.index)
  route_table_id = aws_vpc.vpc.default_route_table_id 
}

Code Highlights for aws-server-group

We can’t hard code the variable number of subnets anymore, so we query them from the VPC.

# Get the subnet IDs
data "aws_subnet_ids" "vpc_subnets" {
  vpc_id = data.aws_vpc.vpc.id
}

Now we use the server count and distribute across the available subnets, using Terraform logic.

# EC2
resource "aws_instance" "server_group" {
  count = var.server_count
  ami = var.ami_id
  instance_type = var.instance_type
  key_name = var.key_name
  security_groups = [aws_security_group.secgrp.id]
  subnet_id = tolist(data.aws_subnet_ids.vpc_subnets.ids)[count.index % length(tolist(data.aws_subnet_ids.vpc_subnets.ids))]
  tags = {
    Name = "${var.server_name}${count.index + var.server_offset}"
  }
}

Using the Modules

Now it is simple to use the modules by passing variables. Start by creating a new directory, named “myvpc”, for example. Then create a main.tf file. This will initialize the AWS provider and call the module, using the relative path on the local filesystem. Here we recreate the same VPC from the previous post.

provider "aws" {
  region = "us-east-1"
}

module "my_simple_vpc" {
  source = "../modules/aws-simple-vpc"

  vpc_name = "labvpc"
  vpc_cidr_block = "10.54.0.0/16"
  vpc_subnets = [
      {
          cidr_block = "10.54.14.0/24"
          availability_zone = "us-east-1a"
      },
      {
          cidr_block = "10.54.16.0/24"
          availability_zone = "us-east-1b"
      },
      {
          cidr_block = "10.54.24.0/24"
          availability_zone = "us-east-1c"
      }
  ]
}

Use this code from the repository, initialize on the first run.

# Go to the directory where the code is stored
cd labtools/terraform/aws/myvpc
# Initialize Terraform setup
terraform init

Now the build and tear down can be repeated, as needed. (The “plan” and “show” commands are optional. Make sure all of the object in the VPC are cleaned up before issuing the “destroy”.)

# Show what changes Terraform will make
terraform plan
# Apply the changes
terraform apply
# Show state information for the resources
terraform show
# Remove the resources
terraform destroy

Now to use the servers, the process is the same. Create a new directory named “mymongo” for example, and create the main.tf file. Here we pull the AMI selection into the caller and pass the ami_id, and the output from the module needs to be passed through to the output for the user.

provider "aws" {
  region = "us-east-1"
}

# Latest Ubuntu 20.04 AMI
data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}

module "my_server_group" {
  source = "../modules/aws-server-group"

  vpc_name = "labvpc"

  server_name = "mongo"
  server_count = 3

  ami_id = data.aws_ami.ubuntu.id
  instance_type = "t2.micro"
  key_name = "npxlab"

  app_port_start = 27017
  app_port_end = 27017
  app_cidr_block = "0.0.0.0/0"

}

output "ec2_dns_names" {
    value = [module.my_server_group.ec2_dns_names]  
}

We will take advantage of these reusable modules in a future post, to avoid repeating all the code from the previous post, just to build EC2 instances with a different AMI.