Infrastructure as Code: Beyond Terraform

Terraform's Limitations

Terraform is the default for good reason: broad provider support, mature ecosystem, and declarative syntax. But it has limits. State management at scale becomes painful. Dynamic configuration requires workarounds. Testing is minimal. Teams hit these limits and look for alternatives.

The state file is Terraform's Achilles heel. At small scale, a single state file in S3 with DynamoDB locking is sufficient. At enterprise scale, state management becomes a full-time concern.

Dynamic configuration is another limitation. Terraform's HCL is declarative, which is a strength for simple configurations but a weakness for complex logic. Conditional resource creation requires count and for_each workarounds. Teams that need dynamic configuration, complex abstractions, or comprehensive testing find HCL limiting.

Pulumi: General-Purpose Languages

Pulumi lets you write infrastructure in TypeScript, Python, Go, or C#. This means loops, conditionals, functions, and abstractions that Terraform's HCL struggles with. Teams with strong software engineering culture often prefer Pulumi for its expressiveness and testability.

Pulumi's key advantage is the programming model. You can use standard software engineering practices: unit tests, integration tests, type checking, and code reuse. A Pulumi program is just a program; it can import libraries, use design patterns, and be refactored like any other code.

Abstraction Power: Define a "WebService" class that encapsulates load balancer, auto-scaling group, security group, SSL certificate, and DNS record. Teams instantiate this class with parameters and get a complete, best-practice web service.

Complexity Trade-off: Pulumi programs are more powerful but also more complex. Debugging requires understanding both the program logic and the infrastructure state. Teams without strong software engineering skills may find Pulumi overwhelming.

AWS CDK: Cloud-Native Abstractions

The Cloud Development Kit provides high-level constructs that compile to CloudFormation. It is the fastest way to build well-architected AWS infrastructure if you stay in the AWS ecosystem. The constructs library encodes AWS best practices automatically.

The L2 constructs are the sweet spot. They are pre-built, opinionated components that encode AWS best practices. A "Vpc" construct creates a VPC with public and private subnets, NAT gateways, and routing tables. A "Database" construct creates an RDS instance with encryption, backup, and monitoring.

L3 Construct Example: A "WebApplication" construct might create VPC, ECS cluster, Fargate service, ALB, Route 53 record, CloudFront distribution, and WAF rules — a complete production-ready infrastructure in a single construct.

AWS-Only Limitation: CDK compiles to CloudFormation; it does not support Azure, GCP, or on-premises infrastructure. For multi-cloud organisations, CDK is one tool among many, not a universal solution.

Crossplane: Kubernetes-Native Control

Crossplane turns Kubernetes into a universal control plane. You define infrastructure as Kubernetes resources, and Crossplane reconciles them with cloud providers. This unifies application and infrastructure management under one API and one GitOps workflow.

The Kubernetes resource model is the foundation. You define a "Database" resource with spec parameters (engine, version, size, region). Crossplane watches this resource and creates the corresponding cloud resource. The resource status reflects the cloud resource state: creating, available, failed.

Composition Power: Define composite resources that combine multiple cloud resources into a single API. A "WebApplication" composite might create VPC, subnets, security groups, ECS cluster, Fargate service, ALB, Route 53 record, and CloudFront distribution — all from a single YAML file.

The GitOps integration is natural. Since Crossplane resources are Kubernetes resources, they are managed by ArgoCD or Flux just like application resources. The same GitOps workflow manages both applications and infrastructure. This is the holy grail of platform engineering: a single control plane, a single API, a single workflow.

State Management at Scale

All IaC tools need state. Terraform uses state files; Pulumi uses a service; Crossplane uses etcd. At enterprise scale, state becomes the critical concern. Use remote state with locking, encryption, and versioning.

Terraform state management has three levels. Level 1: local state (single developer, not suitable for teams). Level 2: remote state in S3 with DynamoDB locking (small teams, simple workflows). Level 3: Terraform Cloud or self-hosted alternatives (enterprise teams, complex workflows, governance requirements).

Terraform Cloud Value: Policy enforcement (Sentinel) prevents costly mistakes — write policies that block certain resource types, enforce tagging, or limit instance sizes.

Crossplane State Limits: etcd has size limits (typically 8MB per object) and performance limits. For large estates, shard resources across multiple clusters or use external state stores.

Our Recommendation

Stay with Terraform unless you have specific needs: complex logic (Pulumi), AWS-only with rapid prototyping (CDK), or unified Kubernetes control (Crossplane). The switching cost is high; the benefit must be clear.

The decision framework: AWS-only and rapid prototyping → CDK. Complex abstractions with strong engineering skills → Pulumi. All-in on Kubernetes with unified GitOps → Crossplane. Otherwise, stay with Terraform.

Regardless of tool, invest in state management. Use remote state with locking, encryption, and versioning. Implement policy enforcement. And train the team: infrastructure as code is a skill, not just a tool.