Using Terraform Merge Lists

In Terraform, managing resources and their configurations often involves dealing with multiple lists. These lists might come from different sources, such as outputs from other modules, variables, or data sources.

To handle these efficiently, Terraform provides the merge() function, which allows you to combine multiple lists or maps into one cohesive unit. This function is particularly useful when building dynamic infrastructures or managing complex configurations where multiple lists need to be combined.

In this guide, I'll explain when and why to use merge() in Terraform and walk through practical examples.

Why Use merge()?

When working with Terraform, you may encounter scenarios where you need to:

• Combine multiple lists of values into a single list for easier management.
• Aggregate outputs from different modules or resources into one configuration.
• Simplify the definition of resources that share common attributes.

By using merge(), you can create a more flexible and dynamic infrastructure codebase. This reduces repetition, improves maintainability, and allows for cleaner, more efficient code.

Basic Syntax of merge() Function

The basic syntax of the merge() function in Terraform is:

merge(map1, map2, ...)

Where:

• map1, map2, ... are the maps (or objects) you want to merge together.
• If there are duplicate keys, the value from the later map will override the earlier ones.

Example Syntax:

output "merged_map" {
  value = merge(
    {
      key1 = "value1"
      key2 = "value2"
    },
    {
      key2 = "override_value2"
      key3 = "value3"
    }
  )
}

In this example, the merge() function combines two maps. The second map overrides the value of key2 from the first map.

Example 1: Merging Lists for EC2 Tags

In AWS environments, tagging resources is a best practice for better organization and cost tracking.

In this example, we’ll merge tags for an EC2 instance, where common tags are shared across all instances, but some tags are unique to each instance.

variable "common_tags" {
  type = map(string)
  default = {
    "Environment" = "Production"
    "ManagedBy"   = "Terraform"
  }
}

variable "instance_tags" {
  type = map(string)
  default = {
    "Role"    = "WebServer"
    "Version" = "1.0"
  }
}

resource "aws_instance" "example" {
  ami           = "ami-12345678"
  instance_type = "t2.micro"

  tags = merge(var.common_tags, var.instance_tags)
}

In this example:

• We have two maps: common_tags and instance_tags.
• The merge() function combines them into a single list of tags. The EC2 instance will now have both common tags and instance-specific tags.
• This helps maintain consistency while allowing flexibility.

Example 2: Merging Security Group Rules

Let’s consider a scenario where you need to create a security group with rules coming from different sources, one from a default rule set and another specific to your application. Instead of hardcoding all the rules together, you can merge them dynamically.

variable "default_security_group_rules" {
  type = list(object({
    protocol = string
    from_port = number
    to_port   = number
    cidr_blocks = list(string)
  }))

  default = [
    {
      protocol    = "tcp"
      from_port   = 22
      to_port     = 22
      cidr_blocks = ["0.0.0.0/0"]
    }
  ]
}

variable "app_security_group_rules" {
  type = list(object({
    protocol = string
    from_port = number
    to_port   = number
    cidr_blocks = list(string)
  }))

  default = [
    {
      protocol    = "tcp"
      from_port   = 80
      to_port     = 80
      cidr_blocks = ["0.0.0.0/0"]
    },
    {
      protocol    = "tcp"
      from_port   = 443
      to_port     = 443
      cidr_blocks = ["0.0.0.0/0"]
    }
  ]
}

resource "aws_security_group" "web" {
  name = "web-sg"

  dynamic "ingress" {
    for_each = merge(var.default_security_group_rules, var.app_security_group_rules)
    content {
      protocol    = ingress.value.protocol
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      cidr_blocks = ingress.value.cidr_blocks
    }
  }
}

In this example:

• Here, we define two sets of security group rules: default rules for SSH access and application-specific rules for HTTP and HTTPS.
• The merge() function allows us to combine these two lists, dynamically generating the security group rules.
• This approach simplifies the management of security group rules and reduces duplication of code.

Example 3: Merging Lists for Dynamic DNS Entries

In a scenario where you manage DNS entries for multiple services in different environments, you might need to merge DNS configurations dynamically.

Let’s look at an example where we define DNS entries for a staging and production environment and then merge them for use.

variable "staging_dns_entries" {
  type = list(object({
    name = string
    value = string
  }))
  
  default = [
    { name = "staging.example.com", value = "192.168.1.1" }
  ]
}

variable "production_dns_entries" {
  type = list(object({
    name = string
    value = string
  }))
  
  default = [
    { name = "www.example.com", value = "192.168.1.2" },
    { name = "api.example.com", value = "192.168.1.3" }
  ]
}

resource "aws_route53_record" "dns" {
  for_each = merge(var.staging_dns_entries, var.production_dns_entries)
  
  zone_id = "Z1234567890"
  name    = each.value.name
  type    = "A"
  ttl     = 300
  records = [each.value.value]
}

In this example:

• We define separate DNS entries for staging and production environments.
• Using merge(), we combine these two lists and dynamically create Route 53 DNS records for both environments.
• This makes the infrastructure code adaptable to different environments without hardcoding multiple configurations.

Example 4: Merging Multiple Resource Outputs

In large Terraform projects, you may have outputs from different modules or resources that need to be merged into a single list or map for further processing.

For example, merging IP addresses from different AWS EC2 instances into a single list:

output "all_instance_ips" {
  value = merge(module.app1_instance.public_ips, module.app2_instance.public_ips)
}

In this example, outputs from two different instance modules (app1_instance and app2_instance) are merged to create a single list of IPs.

Conclusion

The merge() function in Terraform is a powerful tool that helps streamline infrastructure management by dynamically combining lists or maps from multiple sources. This not only simplifies code but also makes your infrastructure configurations more scalable and maintainable.