I’ve been thinking about adding support for building tiered subnets of any valid size for the next iteration of the Dynamic VPC Module (which ended up being called Tiered VPC-NG). It occurred to me that auto subnet generation inside the module actually makes the subnetting less dynamic.
Furthermore, auto subnet calculation should be in assistance to the process of allocating subnets and should not be fed directly as input to the VPC module. This is due to the fact that order matters only for the subnetting calculation. So when tiers or AZs are changed, added or removed, the calculation will shift for none, some or all tiers.
Removing the subnet generation will simplify the module itself and it will reinforce the notion that engineers should know their subnetting when architecting networks. With all that in mind, I created a Tiered Subnet Calculator module to assist with allocating subnets per AZ per network tier.
tiers.auto.tfvars
base_cidr_block = "10.0.0.0/16"
tiers = [
{
name = "app"
acl = "public"
newbit = 4
},
{
name = "db"
acl = "private"
newbit = 4
},
{
name = "worker"
acl = "private"
newbit = 4
},
{
name = "lbs"
acl = "public"
newbit = 4
}
]
az_newbits = {
a = 4
b = 4
c = 4
d = 4
}
variables.tf
variable "base_cidr_block" {
description = "Large starting CIDR block ie 10.0.0.0/16"
type = string
}
variable "tiers" {
description = "Networking tiers"
type = list(object({
name = string
acl = string
newbit = number
}))
}
variable "az_newbits" {
description = "New bits to add to calculated cidr blocks for subnets per AZ"
type = map(number)
}
main.tf
locals {
# generate top level networks for each tier based on tier newbit + base_cidr_block mask ie /4 + /16 = /20
tier_networks = zipmap(var.tiers[*].name, cidrsubnets(var.base_cidr_block, var.tiers[*].newbit...))
# generate a subnet based on each az newbit per tier network ie /4 + /20 = /24
tier_subnets = { for t, n in local.tier_networks : t => cidrsubnets(n, values(var.az_newbits)...) }
# generate azs to subnet map per tier
tier_az_subnets = { for t, s in local.tier_subnets : t => zipmap(keys(var.az_newbits), s) }
# build new tiers list with their associated network and az to subnets map
tiers_with_subnets_per_az = [
for t in var.tiers : {
name = t.name
acl = t.acl
network = lookup(local.tier_networks, t.name)
azs = lookup(local.tier_az_subnets, t.name)
}]
}
output "calculated_tiers" {
value = local.tiers_with_subnets_per_az
}
terraform apply
$ terraform apply
...
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
calculated_tiers = [
{
"acl" = "public"
"azs" = tomap({
"a" = "10.0.0.0/24"
"b" = "10.0.1.0/24"
"c" = "10.0.2.0/24"
"d" = "10.0.3.0/24"
})
"name" = "app"
"network" = "10.0.0.0/20"
},
{
"acl" = "private"
"azs" = tomap({
"a" = "10.0.16.0/24"
"b" = "10.0.17.0/24"
"c" = "10.0.18.0/24"
"d" = "10.0.19.0/24"
})
"name" = "db"
"network" = "10.0.16.0/20"
},
{
"acl" = "private"
"azs" = tomap({
"a" = "10.0.32.0/24"
"b" = "10.0.33.0/24"
"c" = "10.0.34.0/24"
"d" = "10.0.35.0/24"
})
"name" = "worker"
"network" = "10.0.32.0/20"
},
{
"acl" = "public"
"azs" = tomap({
"a" = "10.0.48.0/24"
"b" = "10.0.49.0/24"
"c" = "10.0.50.0/24"
"d" = "10.0.51.0/24"
})
"name" = "lbs"
"network" = "10.0.48.0/20"
},
]
If you want to see each tier transform you can open the
terraform console and call them to see their output.
$ terraform console
> local.tier_networks
{
"app" = "10.0.0.0/20"
"db" = "10.0.16.0/20"
"lbs" = "10.0.48.0/20"
"worker" = "10.0.32.0/20"
}
> local.tier_subnets
{
"app" = tolist([
"10.0.0.0/24",
"10.0.1.0/24",
"10.0.2.0/24",
"10.0.3.0/24",
])
"db" = tolist([
"10.0.16.0/24",
"10.0.17.0/24",
"10.0.18.0/24",
"10.0.19.0/24",
])
"lbs" = tolist([
"10.0.48.0/24",
"10.0.49.0/24",
"10.0.50.0/24",
"10.0.51.0/24",
])
"worker" = tolist([
"10.0.32.0/24",
"10.0.33.0/24",
"10.0.34.0/24",
"10.0.35.0/24",
])
}
> local.tier_az_subnets
{
"app" = tomap({
"a" = "10.0.0.0/24"
"b" = "10.0.1.0/24"
"c" = "10.0.2.0/24"
"d" = "10.0.3.0/24"
})
"db" = tomap({
"a" = "10.0.16.0/24"
"b" = "10.0.17.0/24"
"c" = "10.0.18.0/24"
"d" = "10.0.19.0/24"
})
"lbs" = tomap({
"a" = "10.0.48.0/24"
"b" = "10.0.49.0/24"
"c" = "10.0.50.0/24"
"d" = "10.0.51.0/24"
})
"worker" = tomap({
"a" = "10.0.32.0/24"
"b" = "10.0.33.0/24"
"c" = "10.0.34.0/24"
"d" = "10.0.35.0/24"
})
}
Filtering tiers easy too.
locals {
private_tiers = [for t in local.tier_subnets_per_az : t if t.acl == "private"]
}
Now I can start tweaking the tiers list of objects and az_newbits map
to generate different tiered network configurations.
tiers.auto.tfvars
base_cidr_block = "10.0.0.0/16"
tiers = [
{
name = "app"
acl = "public"
newbit = 6
},
{
name = "db"
acl = "private"
newbit = 6
},
{
name = "worker"
acl = "private"
newbit = 4
},
]
az_newbits = {
b = 2
c = 4
d = 4
}
$ terraform apply
...
You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
calculated_tiers = [
{
"acl" = "public"
"azs" = tomap({
"b" = "10.0.0.0/24"
"c" = "10.0.1.0/26"
"d" = "10.0.1.64/26"
})
"name" = "app"
"network" = "10.0.0.0/22"
},
{
"acl" = "private"
"azs" = tomap({
"b" = "10.0.4.0/24"
"c" = "10.0.5.0/26"
"d" = "10.0.5.64/26"
})
"name" = "db"
"network" = "10.0.4.0/22"
},
{
"acl" = "private"
"azs" = tomap({
"b" = "10.0.16.0/22"
"c" = "10.0.20.0/24"
"d" = "10.0.21.0/24"
})
"name" = "worker"
"network" = "10.0.16.0/20"
},
]
I’m able to take this output, add or remove AZs and subnets that may have not been in the original calculation. I can chop up networks as I see fit.
tiers = [
{
"acl" = "public"
"azs" = {
"b" = "10.0.0.0/24"
"c" = "10.0.1.0/26"
}
"name" = "app"
"network" = "10.0.0.0/22"
},
{
"acl" = "private"
"azs" = {
"c" = "10.0.5.0/26"
"d" = "10.0.5.64/26"
}
"name" = "db"
"network" = "10.0.4.0/22"
},
{
"acl" = "private"
"azs" = {
"b" = "10.0.16.0/22"
"d" = "10.0.21.0/24"
"c" = "10.0.22.0/24"
}
"name" = "worker"
"network" = "10.0.16.0/20"
},
]
Also, I can further validate tiered network ranges with ipcalc.
$ ipcalc 10.0.16.0/20
Address: 10.0.16.0 00001010.00000000.0001 0000.00000000
Netmask: 255.255.240.0 = 20 11111111.11111111.1111 0000.00000000
Wildcard: 0.0.15.255 00000000.00000000.0000 1111.11111111
=>
Network: 10.0.16.0/20 00001010.00000000.0001 0000.00000000
HostMin: 10.0.16.1 00001010.00000000.0001 0000.00000001
HostMax: 10.0.31.254 00001010.00000000.0001 1111.11111110
Broadcast: 10.0.31.255 00001010.00000000.0001 1111.11111111
Hosts/Net: 4094 Class A, Private Internet
A more detailed break down of subnets within a tiered network.
$ ipcalc 10.0.16.0/20 /24
Address: 10.0.16.0 00001010.00000000.0001 0000.00000000
Netmask: 255.255.240.0 = 20 11111111.11111111.1111 0000.00000000
Wildcard: 0.0.15.255 00000000.00000000.0000 1111.11111111
=>
Network: 10.0.16.0/20 00001010.00000000.0001 0000.00000000
HostMin: 10.0.16.1 00001010.00000000.0001 0000.00000001
HostMax: 10.0.31.254 00001010.00000000.0001 1111.11111110
Broadcast: 10.0.31.255 00001010.00000000.0001 1111.11111111
Hosts/Net: 4094 Class A, Private Internet
Subnets after transition from /20 to /24
Netmask: 255.255.255.0 = 24 11111111.11111111.11111111. 00000000
Wildcard: 0.0.0.255 00000000.00000000.00000000. 11111111
1.
Network: 10.0.16.0/24 00001010.00000000.00010000. 00000000
HostMin: 10.0.16.1 00001010.00000000.00010000. 00000001
HostMax: 10.0.16.254 00001010.00000000.00010000. 11111110
Broadcast: 10.0.16.255 00001010.00000000.00010000. 11111111
Hosts/Net: 254 Class A, Private Internet
2.
Network: 10.0.17.0/24 00001010.00000000.00010001. 00000000
HostMin: 10.0.17.1 00001010.00000000.00010001. 00000001
HostMax: 10.0.17.254 00001010.00000000.00010001. 11111110
Broadcast: 10.0.17.255 00001010.00000000.00010001. 11111111
Hosts/Net: 254 Class A, Private Internet
...
15.
Network: 10.0.30.0/24 00001010.00000000.00011110. 00000000
HostMin: 10.0.30.1 00001010.00000000.00011110. 00000001
HostMax: 10.0.30.254 00001010.00000000.00011110. 11111110
Broadcast: 10.0.30.255 00001010.00000000.00011110. 11111111
Hosts/Net: 254 Class A, Private Internet
16.
Network: 10.0.31.0/24 00001010.00000000.00011111. 00000000
HostMin: 10.0.31.1 00001010.00000000.00011111. 00000001
HostMax: 10.0.31.254 00001010.00000000.00011111. 11111110
Broadcast: 10.0.31.255 00001010.00000000.00011111. 11111111
Hosts/Net: 254 Class A, Private Internet
Subnets: 16
Hosts: 4064
The moral of the story is Know Thy Subnetting.
~jq1
Update May 6th 2023:
Things have changed for the Tiered Subnet Calculator module since Terraform v0.13.0 like the resulting order (lexical sorting) of a set of objects with for.
variable "tiers" {
description = "Networking tiers"
type = set(object({
name = string
acl = string
newbit = number
}))
}
Using a list of objects will keep it’s current order so I can use the var.base_cidr_block as a starting point for generating network subnets for AWS.
variable "tiers" {
description = "Networking tiers"
type = list(object({
name = string
acl = string
newbit = number
}))
}
Updated the article with newer outputs from Terraform v1.3.9+, examples
using terraform apply instead of terraform refresh (deprecated), and fixes
from the PR.
~jq1 #StayUp
Update:
I know I said auto subnet calculation should not be fed directly as input to the VPC module but that is just my opinion.
It doesn’t mean you shouldn’t try or that it hasn’t already been looked into.
The goal is to challenge our own ideas.