Slappin' chrome on the WIP

Reflection in the Rear-View:

Initially, I thought the TNT Architecture project and Super Router project were “good enough” to scale VPCs and TGWs in AWS.

But, after looking back on what was built there were so many areas for improvement that I could not resist the extended refinement and discovery.

I went ahead and gave the most useful Terraform networking modules a clean chrome wrapping with encoded engravings.

Most of the work went into internal module changes and consolidating my favorite patterns to use.

This includes new Tiered VPC-NG features for maximum subnet flexibility and a brand new Super Intra VPC Security Group Rules module for cross region VPC access!

Introducing the high powered Shokunin version v1.4.6 (and later) for all modules used in TNT Architecture Demo and Super Router Demo so be sure to check out the new features.

There are breaking changes going from v1.4.5 to v1.4.6 (and later) so best to start fresh.

The next time you’re cruising in the cloud, take a spin in the choice ride!

~jq1 #StayUp #AoD

“No more livin’ hard. Barbeques every day, drivin’ fancy cars.” - Dr. Dre

“The way the wheels spin cuttin’ through the still wind, outsiders lookin’ in with the ill grin.” - Q-Tip

General Refactoring Across Modules:

• Being explicit goes a long way so automatic naming via the random provider was removed across the board.
• Moved most variables into object configurations in each module so now it’s easier to configure them with for_each.
• Many moar validations and preconditions for better interface guard rails especially the cross region checks.
• Doubled down on consistent naming patterns for resources, data sources, variables, local variables and tags.
• Simplified transform output requirements.
• Removed risky for loop assumptions leading to stronger transform patterns.
• base.tf is the new main now that main.tf is used only for document generation with terraform-docs.

Tiered VPC-NG:

• Can access subnet id by subnet name map via output.
• Can rename public and private subnets directly without forcing a new subnet resource.
• Public subnets now have a special attribute option.
  • Only one can have special = true which enables 3 things:
    • Associate a NAT Gateway if enable_natgw = true.
    • Use for associating VPC attatchments when Tiered VPC is passed to a Centralized Router (one in each AZ).
    • Existing public subnets can be rearranged in any order in their repective subnet list without forcing new resources.
  • The trade off is always having to allocate one public subnet per AZ, even if you don’t need to use it (ie using private subnets only).
• Important:
  • All VPC names should be unique across regions (validation enforced).
  • I highly recommend allocating a small public subnet like a /28 and setting it’s special attribute to true for each AZ.

Examples (TNT | Super Router):

Before:

locals {
  tiered_vpc = {
    name    = "app"
    network = "10.0.0.0/20"
    azs = {
      a = {
        private = ["10.0.0.0/24", "10.0.1.0/24", "10.0.2.0/24"]
        public  = ["10.0.3.0/24", "10.0.4.0/24"]
      }
    }
  }
}

After:

locals {
  new_tiered_vpc = {
    name         = "app"
    network_cidr = "10.0.0.0/20"
    azs = {
      a = {
        private_subnets = [
          { name = "cluster1", cidr = "10.0.0.0/24" }
        ]
        public_subnets = [
          { name = "random1", cidr = "10.0.3.0/28" },
          { name = "haproxy1", cidr = "10.0.4.0/26" },
          { name = "natgw", cidr = "10.0.10.0/28", special = true }
        ]
      }
    }
  }
}

Centralized Router:

• Can set name directly.
• Can blackhole routes for specific network cidrs.
• Important:
  • All centralized router names should be unique across regions (validation enforced).

Examples (TNT | Super Router):

module "centralized_router" {
  source = "git@github.com:JudeQuintana/terraform-modules.git//networking/transit_gateway_centralized_router_for_tiered_vpc_ng?ref=v1.4.6"

  env_prefix       = var.env_prefix
  region_az_labels = var.region_az_labels
  centralized_router = {
    name            = "gambit"
    amazon_side_asn = 64512
    blackhole_cidrs = ["172.16.8.0/24"]
    vpcs            = module.vpcs
  }
}

Generate Routes to Other VPCs (function):

• Refactored interface and internals but behavior is the same.
  • Only used as a nested module function in Centralized Router to generate routes to other VPCs.

Example (Centralized Router):

Before:

variable "vpcs" {
  description = "map of tiered_vpc_ng objects"
  type = map(object({
    network                      = string
    az_to_private_route_table_id = map(string)
    az_to_public_route_table_id  = map(string)
  }))
}

After:

variable "vpcs" {
  description = "map of tiered_vpc_ng objects"
  type = map(object({
    network_cidr            = string
    private_route_table_ids = list(string)
    public_route_table_ids  = list(string)
  }))
}

Super Router:

• Can set name directly.
• Can blackhole routes for specific network cidrs.
• Continues to build a decentralized hub spoke topology.

Example (Super Router):

module "super_router_usw2_to_use1" {
  source = "git@github.com:JudeQuintana/terraform-modules.git//networking/tgw_super_router_for_tgw_centralized_router?ref=v1.4.6"

  providers = {
    aws.local = aws.usw2 # local super router tgw will be built in the aws.local provider region
    aws.peer  = aws.use1 # peer super router tgw will be built in the aws.peer provider region
  }

  env_prefix       = var.env_prefix
  region_az_labels = var.region_az_labels
  super_router = {
    name            = "professor-x"
    blackhole_cidrs = local.blackhole_cidrs
    local = {
      amazon_side_asn     = 64521
      centralized_routers = module.centralized_routers_usw2
    }
    peer = {
      amazon_side_asn     = 64522
      centralized_routers = module.centralized_routers_use1
    }
  }
}

Intra VPC Security Group Rule:

• Refactored internals but behavior is the same.
  • Can set security group rules for VPCs within same region.
  • It’s recommended that the Routers are built before the Security Group Rules to enforce validations that are too awkward to handle in this module.

Examples (TNT | Super Router):

module "intra_vpc_security_group_rules" {
  source = "git@github.com:JudeQuintana/terraform-modules.git//networking/intra_vpc_security_group_rule_for_tiered_vpc_ng?ref=v1.4.6"

  for_each = { for r in local.intra_vpc_security_group_rules : r.label => r }

  env_prefix       = var.env_prefix
  region_az_labels = var.region_az_labels
  intra_vpc_security_group_rule = {
    rule = each.value
    vpcs = module.vpcs
  }
}

NEW Super Intra VPC Security Group Rules:

• Can set security group rules for cross region VPCs.
• Important:
  • It’s recommended that the Routers are built before the Security Group Rules to enforce validations that are too awkward to handle in this module.

Example (Super Router):

module "super_intra_vpc_security_group_rules_usw2_to_use1" {
  source = "git@github.com:JudeQuintana/terraform-modules.git//networking/super_intra_vpc_security_group_rules?ref=v1.4.6"

  providers = {
    aws.local = aws.usw2
    aws.peer  = aws.use1
  }

  env_prefix       = var.env_prefix
  region_az_labels = var.region_az_labels
  super_intra_vpc_security_group_rules = {
    local = {
      intra_vpc_security_group_rules = module.intra_vpc_security_group_rules_usw2
    }
    peer = {
      intra_vpc_security_group_rules = module.intra_vpc_security_group_rules_use1
    }
  }
}

Thank you for taking the time to read my work.

It’s been over 3 years in the making and painstaking at times but well worth the effort.

Not just for the knowledge itself but to help others on their way is part of the mission.

I think it’s important to express your ideas to the world because you have much insight to share.

Have fun, be kind, and keep building.

Much gratitude.

~jq1