Synthesizing Tiered VPC in Terraform

Jan 12, 2021 in TERRAFORM
vpc observe-abstract-synthesize create-iterate-combine cloud-networking
7 min read

Today’s Mathematics

Dynamic VPC x Tiered Subnet Calculator = Tiered VPC

Observing Abstractions

In hindsight, the Dynamic VPC module should instead be called Simple VPC. It provides a general VPC tier including a public subnet (/24), a private subnet (/24) and NAT Gateways per AZ. The interface is simple because there is minimal focus on subnetting due to the internal subnet generation.

Routing is automatic and the module outputs can be consumed for peering. The structured naming for resource tagging is a must have for readability in the AWS console.

However, the biggest draw backs are that subnets of various valid sizes can’t be added or removed from an AZ. Flexibility in subnet creation is needed to take advantage of baseline VPC features like network segmentation.

The Tiered Subnet Calculator idea had several takeways that were crucial to iterating on Dynamic VPC. Subnet generation makes subnetting less dynamic and knowing your subnetting is the key to architecting networks. The best thing to come out of the calculator was the structured output.

calculated_tiers = [
  {
    "acl" = "public"
    "azs" = {
      "a" = "10.0.0.0/24"
      "b" = "10.0.1.0/24"
      "c" = "10.0.2.0/24"
      "d" = "10.0.3.0/24"
    }
    "name" = "app"
    "network" = "10.0.0.0/20"
  },
  {
    "acl" = "private"
    "azs" = {
      "a" = "10.0.16.0/24"
      "b" = "10.0.17.0/24"
      "c" = "10.0.18.0/24"
      "d" = "10.0.19.0/24"
    }
    "name" = "db"
    "network" = "10.0.16.0/20"
  }
]

Each object is basically a tiered VPC structure. It’s much easier to think about tiers when scaling out VPCs because we can narrow down the context (ie app, db, general) and maximize the use of smaller network sizes. Routing between VPCs should be easier now that Transit Gateway exists.

Synthesis

The calculator output was abstracted into a tiered VPC object then integrated into the interface for Dynamic VPC. This fusion created the Tiered VPC module.

The tiered object requires at least one public subnet. All Public subnets will route out the Internet Gateway. Private subnets can be nulled out and when they are defined they will route out the NAT gateway relative to the AZ.

Subnet generation has been removed because subnets should be subnetted correctly before module instansiation. All subnets will have randomly generated name for better labeling. I chose not to do subnet CIDR validation against the network CIDR because the aws_subnet resource will error on invalid subnets during creation.

variables.tf

# snippet
variable "tier" {
  type = object({
    name    = string
    network = string
    azs = map(object({
      public  = list(string)
      private = list(string)
    }))
  })

  validation {
    condition = length([
      for az in var.tier.azs : true
      if length(az.public) > 0
    ]) == length(var.tier.azs)
    error_message = "There must be at least 1 public subnet per az."
  }
}

The shared public route table for all AZs will update automatically when public subnets are add or removed.

tier = {
  azs = {
    a = {
      private = null
      public = ["10.47.0.0/28"] # 10.47.0.0/24 chopped up into /28
    }
  }
  name    = "general"
  network = "10.47.0.0/20"
}

I can define the private subnet layer at any time. Again, NAT Gateways are created per AZ which is why we need at least one public subnet per AZ to create them in. The relative AZ’s private route tables also update automatically when private subnets are added or removed.

tier = {
  azs = {
    a = {
      private = ["10.47.11.0/24", "10.47.12.0/24"]
      public  = ["10.47.0.0/28"] # 10.47.0.0/24 chopped up into /28
    }
  }
  name    = "general"
  network = "10.47.0.0/20"
}

Adding more subnets and another AZ:

provider "aws" {
  region = "us-west-2"
}

variable "env_prefix" {
  default = "test"
}

variable "region_az_labels" {
  type = map(string)

  default = {
    us-west-2  = "usw2"
    us-west-2a = "usw2a"
    us-west-2b = "usw2b"
    us-west-2c = "usw2c"
  }
}

locals {
  general_tier = {
    azs = {
      a = {
        private = ["10.47.11.0/24", "10.47.12.0/24"]
        public  = ["10.47.0.0/28", "10.47.0.16/28"] # 10.47.0.0/24 chopped up into /28
      },
      c = {
        private = null
        public  = ["10.47.6.0/24", "10.47.7.0/24"]
      },
    }
    name    = "general"
    network = "10.47.0.0/20"
  }
}

module "general_vpc" {
  source = "git@github.com:JudeQuintana/terraform-modules.git//networking/tiered_vpc?ref=v1.1.1"

  tier             = local.general_tier
  env_prefix       = var.env_prefix
  region_az_labels = var.region_az_labels
}

The resource naming would be as follows:

<env_prefix>-<region|az>-<tier_name>-<public|private>-<pet_name|misc_label>

VPC:

TEST-usw2-general

IGW:

TEST-usw2-general

Public Subnets:

TEST-usw2a-general-public-squirrel
TEST-usw2a-general-public-warthog
TEST-usw2c-general-public-bat
TEST-usw2c-general-public-jackal

Public Route Table:

TEST-usw2-general-public-all (contains shared route for all public subnets across azs)

Private Subnets:

TEST-usw2a-general-private-bass
TEST-usw2a-general-private-catfish

Private Route Table:

TEST-usw2a-general-private (contains all private subnets relative to the AZ)

EIP:

TEST-usw2a-general-private

NATGW:

TEST-usw2a-general-private

Three Flavors of Output

I wasn’t sure of the best way to output the attributes required for peering so I cooked up a few marvelous flavors to make ya mouth water! Like oh shit!

Separate collections:

private_route_table_ids = {
  a = "rtb-0c517e8e8a0cfd1a7"
}
private_subnet_ids = {
  a = [
    "subnet-0aeef4439292f37d0",
    "subnet-0d722e62b5316094c",
  ]
}
public_route_table_ids = {
  a = "rtb-07490f656c15c1ba4"
  c = "rtb-07490f656c15c1ba4"
}
public_subnet_ids = {
  a = [
    "subnet-08560aef6093bed8b",
    "subnet-0f3b2e567e19f9ba0",
  ]
  c = [
    "subnet-091d97b6ffce14a88",
    "subnet-09e3c3f1f1363f37f",
  ]
}

Bundled Tier Object:

tier_bundle = {
  a = {
    private = [
      "10.47.11.0/24",
      "10.47.12.0/24",
    ]
    private_ids = [
      "subnet-0aeef4439292f37d0",
      "subnet-0d722e62b5316094c",
    ]
    private_route_table_id = "rtb-0c517e8e8a0cfd1a7"
    public = [
      "10.47.0.0/28",
      "10.47.0.16/28",
    ]
    public_ids = [
      "subnet-08560aef6093bed8b",
      "subnet-0f3b2e567e19f9ba0",
    ]
    public_route_table_id = "rtb-07490f656c15c1ba4"
  }
  c = {
    private                = []
    private_ids            = []
    private_route_table_id = null
    public = [
      "10.47.6.0/24",
      "10.47.7.0/24",
    ]
    public_ids = [
      "subnet-091d97b6ffce14a88",
      "subnet-09e3c3f1f1363f37f",
    ]
    public_route_table_id = "rtb-07490f656c15c1ba4"
  }
}

Structured Tier Object:

tier_structured = {
  a = {
    private = [
      {
        id     = "subnet-0aeef4439292f37d0"
        subnet = "10.47.11.0/24"
      },
      {
        id     = "subnet-0d722e62b5316094c"
        subnet = "10.47.12.0/24"
      },
    ]
    private_route_table_id = "rtb-0c517e8e8a0cfd1a7"
    public = [
      {
        id     = "subnet-08560aef6093bed8b"
        subnet = "10.47.0.0/28"
      },
      {
        id     = "subnet-0f3b2e567e19f9ba0"
        subnet = "10.47.0.16/28"
      },
    ]
    public_route_table_id = "rtb-07490f656c15c1ba4"
  }
  c = {
    private                = []
    private_route_table_id = null
    public = [
      {
        id     = "subnet-091d97b6ffce14a88"
        subnet = "10.47.6.0/24"
      },
      {
        id     = "subnet-09e3c3f1f1363f37f"
        subnet = "10.47.7.0/24"
      },
    ]
    public_route_table_id = "rtb-07490f656c15c1ba4"
  }
}

Module Iteration

Terraform 0.13 enables use of for_each on modules. Let’s see it in action!

terraform {
  required_version = ">= 0.13"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}

# default provider
provider "aws" {
  region = "us-east-1"
}

provider "aws" {
  region = "us-west-2"
  alias  = "usw2"
}

variable "env_prefix" {
  default = "test"
}

variable "region_az_labels" {
  type = map(string)

  default = {
    us-east-1  = "use1"
    us-east-1a = "use1a"
    us-east-1b = "use1b"
    us-east-2c = "use1c"
    us-west-2  = "usw2"
    us-west-2a = "usw2a"
    us-west-2b = "usw2b"
    us-west-2c = "usw2c"
  }
}

locals {
  tiers = [
    {
      azs = {
        a = {
          private = ["10.0.8.0/24", "10.0.9.0/24"]
          public  = ["10.0.0.0/24", "10.0.1.0/24"]
        },
        b = {
          private = ["10.0.11.0/24", "10.0.12.0/24"]
          public  = ["10.0.3.0/24", "10.0.4.0/24"]
        },
      }
      name    = "app"
      network = "10.0.0.0/20"
    },
    {
      azs = {
        a = {
          private = ["10.0.16.0/24", "10.0.17.0/24"]
          public  = ["10.0.28.0/28"] # 10.0.28.0/24 chopped up into /28
        },
        b = {
          private = ["10.0.20.0/24", "10.0.21.0/24"]
          public  = ["10.0.28.16/28"] # 10.0.28.0/24 chopped up into /28
        },
      }
      name    = "db"
      network = "10.0.16.0/20"
    },
    {
      azs = {
        a = {
          private = ["10.47.11.0/24", "10.47.12.0/24"]
          public  = ["10.47.0.0/28", "10.47.0.16/28"] # 10.47.0.0/24 chopped up into /28
        },
        c = {
          private = null
          public  = ["10.47.6.0/24", "10.47.7.0/24"]
        },
      }
      name    = "general"
      network = "10.47.0.0/20"
    }
  ]
}

module "usw2_vpcs" {
  source = "git@github.com:JudeQuintana/terraform-modules.git//networking/tiered_vpc?ref=v1.1.1"
  for_each = { for t in local.tiers : t.name => t }

  providers = {
    aws = aws.usw2
  }

  tier             = each.value
  env_prefix       = var.env_prefix
  region_az_labels = var.region_az_labels
}

Takeaways

Terraform is all about transforming collections. Filtering and building related data structures for resources to lookup against later works out well. Learn to express yourself with for and wield unlimited cosmic power.

Nested for loops are necessary for iterating over deep data structures. But if you find yourself writing them on shallow maps or objects then think about how simple for loops yield simple data structures. Being concise goes a long way.

What you feed into for_each directly influences module behavior in a fluid way. Module flexibility depends on how well data structures are organized to create resources or dynamic blocks, all of which result from concise collection transformations. Utilize objects to create abstractions and filter attributes into data structures that make sense for related resources.

Caveats

You wont be able to see the generated pet names for subnet naming on plan, only on apply. Using ignore_changes facilitates adding and removing AZs and subnets properly but a plan won’t show drift if there are changes via the AWS console.

Even though you can delete subnets, remember that the NAT Gateways get created in the first subnet in public subnet list for the AZ.

This module will cost money! (ie NAT Gateways, EIP)

Final Thoughts

Thanks for reading my take on yet another VPC abstraction, the Tiered VPC. Looking forward to YOLO’ing it in production!

~jq1

Feedback

What did you think about this post? jude@jq1.io