GitOps: Declarative Infrastructure Management

The GitOps Principle

GitOps uses Git repositories as the single source of truth for declarative infrastructure and applications. Changes are made via pull requests, not direct commands. Automated agents (ArgoCD, Flux) reconcile the actual state with the desired state defined in Git.

The principle is simple: if you want to change the infrastructure, you change the Git repository. The agent detects the change and applies it to the cluster. If someone changes the cluster directly, the agent detects the drift and reverts it.

GitOps Advantages: Audit trail (every change is a Git commit with author, timestamp, message), rollback (reverting a commit reverts the infrastructure), review (pull requests require approval), and reproducibility (the same commit produces the same state anywhere).

ArgoCD vs Flux

ArgoCD provides a web UI for visualising application state, supports multiple Git sources, and integrates with Helm and Kustomize. Flux is more lightweight, Git-native, and follows a stricter GitOps philosophy. Both are production-ready; the choice depends on team preferences for UI vs CLI workflows.

ArgoCD: More popular choice, especially for teams new to GitOps. The web UI provides visibility into application state, sync status, and history. Supports application grouping, project-based access control, and automated sync policies. Better for teams that need a UI and easier onboarding.

Flux: Preferred by teams with strong Git-native workflows. No UI — all interaction is through Git and the command line. This is a design choice: Flux believes Git is the interface. More lightweight, smaller attack surface, better multi-tenancy support, and tighter integration with GitHub/GitLab.

We recommend ArgoCD for most teams starting with GitOps, and Flux for teams with specific requirements around multi-tenancy or security.

Reconciliation Patterns

The reconciliation loop continuously compares cluster state with Git state. Drift (manual changes, failures) is automatically corrected. This guarantees that the cluster always matches the defined state. Self-healing infrastructure without manual intervention.

Drift Detection: When someone modifies the cluster directly (with kubectl, for example), the agent detects the change on the next reconciliation loop and reverts it. This prevents configuration drift: the gradual accumulation of undocumented changes that makes clusters unmanageable. We have seen clusters where no one knew why a particular setting was configured a certain way, because it had been changed manually years ago.

Self-Healing: If a pod crashes, the Kubernetes scheduler restarts it. If a node fails, the workloads are rescheduled. If the entire cluster fails, the GitOps agent can recreate it from the Git repository. The combination of declarative configuration and automated reconciliation creates resilient infrastructure that recovers from failures without human intervention.

Error Handling: If a Git commit contains an error (invalid YAML, incorrect image tag, missing resource), the agent should detect it and stop applying changes. It should report the error and wait for a fix. Blindly continuing after an error can make things worse. Both ArgoCD and Flux provide robust error handling, but the team's response to errors is equally important.

Secrets Management

Secrets cannot be stored in plain text in Git. GitOps requires a secrets management solution that integrates with the GitOps workflow.

Sealed Secrets: Encrypt secrets using a cluster-specific key. Encrypted secrets can be stored in Git safely. The Sealed Secrets controller decrypts them in the cluster. Simplest approach: secrets are versioned in Git, auditable, and managed through the same pull request workflow. Limitation: the encryption key is cluster-specific.
External Secrets: Fetch secrets from external vaults (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault) and create Kubernetes secrets from them. The reference (not the secret value) is stored in Git. Most secure: secrets never touch Git. Limitation: requires an external vault and network connectivity.
SOPS: Encrypt files with age or PGP keys. Encrypted files can be stored in Git. Multiple keys can decrypt the same file, enabling key rotation and team access. Flexible and works with any file format. Limitation: key management is the team's responsibility.

We recommend Sealed Secrets for teams starting with GitOps, External Secrets for teams with existing vault infrastructure, and SOPS for teams that need cross-platform encryption. All three are production-ready and widely used.

Our Recommendation

Start with ArgoCD for visibility and ease of use. Move to Flux if you need tighter Git-native workflows or want to avoid running a separate UI component. Both tools deliver the core GitOps promise: Git as the single source of truth.

The success factors are not the tool but the practices. Use pull requests for all changes. Review infrastructure changes with the same rigour as code changes. Implement drift detection and automated reconciliation. Manage secrets securely. And train the team: GitOps is a workflow change, not just a tool adoption.

Start Small: Start with a single application. Configure ArgoCD or Flux to manage it. Establish the pull request workflow. Add secrets management. Then expand to more applications. The pattern scales from one application to hundreds, but the fundamentals are the same.

The GitOps Principle

ArgoCD vs Flux

Reconciliation Patterns

Secrets Management

Our Recommendation

Managing infrastructure with Git?