DEVOPS TRIBUNE ENGINEERING INTELLIGENCE FOR THE REAL WORLD
PLATFORM ARCHITECTURE & PORTFOLIO · 21 MIN READ

Real-World DevOps Projects: 5 Production Platform Architectures for Engineers

Moving beyond trivial tutorials: five enterprise platform engineering blueprints implementing multi-region Terraform VPCs, multi-tenant GitOps with Argo CD, distributed observability with OpenTelemetry, progressive rollouts, and automated disaster recovery.

By DevOps Tribune Engineering Staff · Published November 17, 2026

The tech industry's transition from traditional systems administration to modern platform engineering has dramatically raised the expectations placed on engineering portfolios and technical assessments. Hiring managers and principal architects routinely dismiss candidate repositories featuring basic "Hello World" Dockerfiles, unencrypted single-file Terraform configs, or basic Jenkins pipelines executing arbitrary shell scripts on shared virtual machines. These beginner projects fail to demonstrate the operational competencies required in production: failure domain isolation, least-privilege identity federation, zero-downtime progressive rollouts, and structured incident observability.

In high-availability enterprise environments, real-world DevOps is the discipline of architecting resilient systems that survive hardware failures, network partitions, and malicious supply-chain attacks. A production-grade project does not simply spin up a compute node; it partitions subnets across availability zones, enforces AWS IAM Roles for Service Accounts (IRSA), signs container images with cryptographic transparency logs, decrypts secrets in memory via KMS, and establishes automated rollback triggers based on p99 latency regressions.

This technical guide details five comprehensive, production-grade platform engineering projects. Each project addresses an essential operational domain: infrastructure as code foundations, declarative GitOps continuous delivery, distributed multi-signal observability, progressive canary rollouts, and automated disaster recovery. Every blueprint includes production-tested configuration code, architectural trade-off analyses, security guardrails, and verification commands.


1. The Production Portfolio Standard: What Distinguishes Real-World Projects

Before examining individual project architectures, platform engineers must understand the criteria that separate an amateur hobby project from an enterprise-ready platform implementation:

Engineering Dimension Amateur / Tutorial Pattern Enterprise Production Standard
Cloud Identity & Auth Static AWS access keys pasted into repository secrets Short-lived OpenID Connect (OIDC) federation with granular IAM conditions
State Management Local terraform.tfstate committed to git Remote partitioned S3 backend with DynamoDB state locking and KMS encryption
Container Hardening Single-stage build running as root with :latest tag Multi-stage build, non-root user (UID 10001), Trivy CVE scanning, commit SHA tags
Deployment Strategy Manual kubectl apply from developer laptops Automated GitOps reconciliation via Argo CD or Flux with self-healing drift detection
Observability Unstructured stdout logs and static server CPU graphs OpenTelemetry distributed tracing, JSON structured logs, PromQL error budget burn alerts

2. Project 1: Zero-Trust Multi-AZ AWS EKS Platform with OpenTofu / Terraform

The foundation of any cloud-native platform is a hardened network and compute topology. This project builds an enterprise Kubernetes environment across three AWS availability zones, implementing strict network boundaries and cryptographic identity federation.

Architectural Specifications

  • VPC Network Topology: A /16 IPv4 VPC divided into 3 public subnets (for external ALBs), 3 private compute subnets (for EKS worker nodes), and 3 isolated database subnets (with no internet gateways or NAT routes).
  • Outbound Egress Architecture: Highly available NAT Gateways deployed across distinct availability zones to prevent multi-AZ failure coupling.
  • IAM Roles for Service Accounts (IRSA): Eliminates node-level IAM credentials. Individual Kubernetes service accounts assume scoped AWS IAM roles via OIDC token projection.
  • EKS Managed Node Groups: Bottlerocket or Amazon Linux 2023 nodes with IMDSv2 strictly enforced (hop limit: 1) to eliminate SSRF metadata exfiltration attacks.

Core OpenTofu Infrastructure Manifest

# terraform/network_and_eks.tf
terraform {
  required_version = ">= 1.5.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "devopstribune-production-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  public_subnets  = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  private_subnets = ["10.0.10.0/24", "10.0.20.0/24", "10.0.30.0/24"]

  enable_nat_gateway     = true
  single_nat_gateway     = false # Multi-AZ redundancy
  enable_dns_hostnames   = true
  enable_dns_support     = true

  # Essential tags for AWS Load Balancer Controller auto-discovery
  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }
  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 20.0"

  cluster_name    = "devopstribune-production-cluster"
  cluster_version = "1.31"

  # Keep cluster control plane private with endpoint public access restricted to corporate CIDRs
  cluster_endpoint_public_access  = true
  cluster_endpoint_private_access = true

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  eks_managed_node_groups = {
    general_workloads = {
      min_size     = 3
      max_size     = 9
      desired_size = 3

      instance_types = ["m6i.xlarge"]
      capacity_type  = "ON_DEMAND"

      # Security: Enforce IMDSv2 to prevent cloud credential exfiltration
      metadata_options = {
        http_endpoint               = "enabled"
        http_tokens                 = "required"
        http_put_response_hop_limit = 1
      }

      labels = {
        Role = "general-compute"
      }
    }
  }

  enable_cluster_creator_admin_permissions = false
}

IAM Roles for Service Accounts (IRSA) Implementation

To eliminate static AWS access keys, configure Kubernetes service accounts to assume dedicated AWS IAM roles. An OIDC trust policy validates the cluster OIDC issuer URL and restricts role assumption strictly to a specific Kubernetes namespace and service account name:

# AWS IAM Trust Policy for Kubernetes ServiceAccount
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/EXAMPLE"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.us-east-1.amazonaws.com/id/EXAMPLE:sub": "system:serviceaccount:production:storage-manager",
          "oidc.eks.us-east-1.amazonaws.com/id/EXAMPLE:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}

In your Kubernetes manifests, annotate the ServiceAccount with the IAM role ARN. When the pod starts, the AWS EKS pod identity webhook automatically injects temporary AWS STS credentials into the container environment:

# k8s/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: storage-manager
  namespace: production
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/ProductionStorageManagerRole

3. Project 2: Multi-Tenant GitOps Continuous Delivery with Argo CD

In high-maturity engineering organizations, application code and deployment configurations live in separate repositories. This project builds a multi-tenant GitOps continuous delivery system where developer commits automatically trigger immutable release builds, update deployment repos via pull request automation, and synchronize into isolated Kubernetes namespaces.

Two-Repository GitOps Architecture

  1. Application Source Repository (app-repo): Contains business logic, unit tests, and multi-stage Dockerfiles. On merge to main, GitHub Actions builds the image, scans for vulnerabilities, signs the digest with Cosign, and opens an automated PR against the deployment repo updating the image tag.
  2. Platform Deployment Repository (gitops-repo): Contains environment Kustomize overlays (dev, staging, production) and Argo CD Application manifests. Argo CD watches this repo and automatically reconciles changes.

Automated GitHub Actions Image Promotion Workflow

# .github/workflows/promote-image.yml
name: Build, Sign & Promote Artifact

on:
  push:
    branches: [ main ]

permissions:
  contents: read
  id-token: write # For OIDC container signing

jobs:
  build-and-promote:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Source Code
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Extract Metadata & Generate Tag
        id: meta
        run: |
          SHORT_SHA=$(git rev-parse --short HEAD)
          echo "tag=${SHORT_SHA}" >> $GITHUB_OUTPUT

      - name: Build & Push OCI Image
        run: |
          docker build -t ghcr.io/devopstribune/core-api:${{ steps.meta.outputs.tag }} .
          echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
          docker push ghcr.io/devopstribune/core-api:${{ steps.meta.outputs.tag }}

      # Update GitOps Deployment Repo via Automated Commit
      - name: Update GitOps Deployment Repository
        uses: peter-evans/repository-dispatch@ff4505510f45ebfe9d2348398123e9c7442cda37 # v3.0.0
        with:
          token: ${{ secrets.GITOPS_PROMOTION_TOKEN }}
          repository: devopstribune/gitops-deployments
          event-type: update-image-tag
          client-payload: '{"service": "core-api", "tag": "${{ steps.meta.outputs.tag }}"}'

4. Project 3: Production Observability Stack with OpenTelemetry, Prometheus & Grafana

A production platform requires a unified telemetry ingestion pipeline capable of collecting system metrics, distributed traces, and structured logs. This project deploys an end-to-end cloud-native observability pipeline using the OpenTelemetry Collector, Prometheus TSDB, and Grafana.

Observability Pipeline Topology

  • OpenTelemetry Collector (DaemonSet & Gateway): Deployed as a node DaemonSet to collect host metrics and receive application OTLP traces via gRPC (port 4317) and HTTP (port 4318).
  • Batch & Memory Ballast Processors: Buffers telemetry in memory and flushes in batches, eliminating high CPU overhead on worker nodes.
  • Prometheus Remote Write Exporter: Forwards normalized metrics to long-term storage while exporting distributed traces to Jaeger or Tempo.

Production OpenTelemetry Collector Configuration

# k8s/otel-collector-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-config
  namespace: observability
data:
  otel-collector-config.yaml: |
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318
      prometheus:
        config:
          scrape_configs:
            - job_name: 'kubernetes-pods'
              kubernetes_sd_configs:
                - role: pod

    processors:
      memory_limiter:
        check_interval: 1s
        limit_percentage: 75
        spike_limit_percentage: 20
      batch:
        send_batch_size: 1024
        timeout: 5s

    exporters:
      prometheusremotewrite:
        endpoint: "http://prometheus-server.observability.svc:9090/api/v1/write"
      otlp/traces:
        endpoint: "tempo.observability.svc:4317"
        tls:
          insecure: true

    service:
      pipelines:
        metrics:
          receivers: [otlp, prometheus]
          processors: [memory_limiter, batch]
          exporters: [prometheusremotewrite]
        traces:
          receivers: [otlp]
          processors: [memory_limiter, batch]
          exporters: [otlp/traces]

5. Project 4: Progressive Delivery & Automated Canary Analysis with Argo Rollouts

Traditional rolling deployments update pods indiscriminately. If a release introduces a subtle memory leak or intermittent 500 error that unit tests failed to catch, 100% of production users are immediately impacted. This project builds a progressive canary delivery system using Argo Rollouts and Prometheus AnalysisTemplates.

Canary Traffic Progression Workflow

  1. When an updated image is committed, Argo Rollouts deploys a single canary replica alongside stable pods.
  2. The Ingress controller routes exactly 10% of live traffic to the canary version while 90% remains on stable.
  3. An AnalysisTemplate queries Prometheus every 60 seconds, measuring the HTTP 5xx error rate and p99 latency for the canary pod group.
  4. If the error rate remains under 0.1%, traffic scales to 25%, then 50%, and finally 100%.
  5. If the error rate breaches 0.5% at any stage, the rollout aborts immediately, scaling the canary pods to zero and routing 100% of traffic back to the stable baseline without manual human intervention.

Argo Rollout and Automated AnalysisTemplate Manifest

# k8s/canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payment-gateway
  namespace: production
spec:
  replicas: 10
  strategy:
    canary:
      analysis:
        templates:
          - templateName: success-rate-analysis
        args:
          - name: service-name
            value: payment-gateway-canary
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - setWeight: 25
        - pause: { duration: 10m }
        - setWeight: 50
        - pause: { duration: 10m }
  selector:
    matchLabels:
      app: payment-gateway
  template:
    metadata:
      labels:
        app: payment-gateway
    spec:
      containers:
        - name: web
          image: devopstribune/payment-service:v2.1.0
          ports:
            - containerPort: 8080
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate-analysis
  namespace: production
spec:
  metrics:
    - name: success-rate
      interval: 60s
      successCondition: result[0] >= 0.995 # Must maintain 99.5% success rate
      failureLimit: 2
      provider:
        prometheus:
          address: "http://prometheus-server.observability.svc:9090"
          query: |
            sum(rate(http_requests_total{status!~"5.*",app="payment-gateway-canary"}[1m])) 
            / 
            sum(rate(http_requests_total{app="payment-gateway-canary"}[1m]))

6. Project 5: Automated Cluster Backup & Disaster Recovery with Velero

A true platform engineer plans for catastrophic cluster loss: accidental namespace deletion, etcd corruption, or regional AWS outages. This project implements an automated, air-gapped backup and disaster recovery system using Velero.

Disaster Recovery Architecture

  • Cluster State Synchronization: Scheduled hourly backups serialize all Kubernetes custom resources, secrets, configmaps, and deployments into compressed tarballs uploaded to a private, encrypted S3 bucket.
  • Persistent Volume Snapshots: Leverages AWS EBS Container Storage Interface (CSI) volume snapshot APIs to capture point-in-time state of stateful database volumes across availability zones.
  • Automated Recovery Drill: An automated bash and kubectl recovery script that provisions a blank ephemeral cluster, installs Velero, and restores production state from the latest backup, measuring actual Recovery Time Objective (RTO).

Velero Production Backup Schedule and Restore Commands

# Schedule automated daily cluster backup with 14-day retention
velero schedule create daily-production-backup \
  --schedule="0 2 * * *" \
  --include-namespaces production,observability \
  --snapshot-volumes \
  --ttl 336h0m0s

# Simulate Disaster Recovery: Restore production namespace from S3 backup
velero restore create --from-backup daily-production-backup-20261117020000 \
  --restore-volumes=true \
  --wait

# Verify restoration health status
velero restore describe daily-production-backup-20261117020000
kubectl get pods -n production

7. Advanced Production Additions: Multi-Cluster ApplicationSets & SRE Alerting

To elevate these projects into genuine principal-level platform blueprints, incorporate these two advanced operational capabilities:

Argo CD ApplicationSet for Dynamic Multi-Cluster Provisioning

Managing individual Application manifests for dozens of microservices across multiple Kubernetes clusters creates configuration bloat. An ApplicationSet uses generators (such as the Git Directory Generator or Cluster List Generator) to automatically instantiate applications across target clusters:

# k8s/applicationset.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: platform-services
  namespace: argocd
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            environment: production
  template:
    metadata:
      name: '{{name}}-ingress-controller'
    spec:
      project: default
      source:
        repoURL: 'https://github.com/devopstribune/gitops-deployments.git'
        targetRevision: main
        path: 'infrastructure/ingress'
      destination:
        server: '{{server}}'
        namespace: ingress-nginx
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

Prometheus Multi-Burn-Rate Alerting Rules for Error Budgets

Rather than alerting on transient spikes, modern SRE teams implement Google SRE multiwindow multi-burn-rate alerts that fire only when error budgets are rapidly depleting:

# k8s/slo-alerts.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: payment-service-slo-alerts
  namespace: observability
spec:
  groups:
    - name: payment-service-availability
      rules:
        # Critical 1-hour burn alert (14.4x rate consumes 2% of budget in 1 hour)
        - alert: PaymentServiceHighErrorBurnRate
          expr: |
            (
              sum(rate(http_requests_total{job="payment-service",status=~"5.."}[1h]))
              /
              sum(rate(http_requests_total{job="payment-service"}[1h]))
            ) > (14.4 * (1 - 0.999))
            and
            (
              sum(rate(http_requests_total{job="payment-service",status=~"5.."}[5m]))
              /
              sum(rate(http_requests_total{job="payment-service"}[5m]))
            ) > (14.4 * (1 - 0.999))
          for: 2m
          labels:
            severity: page
          annotations:
            summary: "Payment Service error budget burning at 14.4x rate"
            description: "High error rate consumes >2% of monthly budget within 1 hour."

8. How to Defend These Architectures in Platform Engineering Interviews

When presenting these architectures to interview panels and hiring committees, structure your technical responses around trade-offs and operational failure modes:

  • Explain the Why, Not Just the How: Do not merely state that you used EKS and Argo CD. Explain why you selected EKS Managed Node Groups over self-managed EC2 nodes (patching automation, rolling node upgrades) and why you chose Argo CD over direct CI pushes (eliminates long-lived cluster credentials in runners, ensures drift correction).
  • Discuss What Broke and How You Fixed It: Authentic production experience is defined by debugging failures. Discuss real issues you encountered: AWS Load Balancer Controller subnet tagging omissions, cgroup memory limit OOMKilled events, or Argo CD OutOfSync webhook race conditions.
  • Highlight Security Guardrails: Emphasize how your project prevents catastrophe: non-root containers, read-only root filesystems, IMDSv2 metadata protection, and automated container image signing via Sigstore Cosign.

9. How to Showcase DevOps Projects on GitHub and Resumes

Building high-quality infrastructure code is only half the battle; communicating its architectural value to engineering leads and hiring managers is equally critical. Structure project repositories according to this professional standard:

  1. Architecture Diagram: Include a clear SVG or Mermaid diagram at the top of your README.md illustrating network boundaries, data flow, ingress routing, and observability pipelines.
  2. Reproducibility: Provide single-command local deployment instructions using tools like Docker Compose, Minikube, or LocalStack, allowing reviewers to verify your code without incurring AWS billing.
  3. Design Decisions & Trade-Offs: Include an ADR/ (Architectural Decision Records) folder explaining why you chose specific tools (e.g., why OpenTofu over Terraform, why OAC over OAI, why Argo CD over Flux).
  4. Failure Scenarios Handled: Document specific incident failure modes your project actively mitigates (e.g., node termination, memory leaks, secret exposure, broken rollouts).

10. Cloud Cost Optimization & FinOps Governance for Portfolio Projects

A frequent anxiety among engineers building cloud projects is the fear of unexpected billing charges. Platform engineering demands strict financial governance (FinOps). Implement these four automated cost safeguards:

  • Automated Cost Estimation via Infracost: Integrate Infracost into your GitHub Actions pull request pipeline to calculate the exact monthly dollar variance before running tofu apply.
  • Scheduled Node Auto-Stopping: Use Kubernetes CronJobs or AWS EventBridge rules to scale non-production worker nodes to zero outside business hours, reducing EC2 compute bills by over 65%.
  • Karpenter Intelligent Node Consolidation: Replace static cluster autoscalers with Karpenter, which dynamically provisions right-sized Graviton (ARM64) Spot instances and consolidates underutilized nodes automatically.
  • Strict CloudWatch Billing Alarms: Set up an AWS Billing Alarm that triggers SNS email notifications when estimated monthly cloud charges exceed a conservative threshold (e.g., $15.00).
  • Mandatory Cost Allocation Tags: Enforce AWS Resource Tagging Policies requiring Environment, Owner, and Project tags on all provisioned infrastructure. This enables granular cost breakdown dashboards inside AWS Cost Explorer, proving your ability to manage cloud expenditures responsibly.

11. Frequently Asked Questions

How much does it cost to build and test these AWS projects?

By leveraging small instance types (such as t4g.small or m6i.large for EKS worker nodes), utilizing single-AZ development configurations, and tearing down resources via tofu destroy when testing concludes, you can build and validate all five projects within the AWS Free Tier or for under $10–$15 total in cloud testing credits.

Should I build multi-cloud projects or specialize in AWS?

Specialize deeply in one major cloud provider (AWS, GCP, or Azure) first before attempting multi-cloud. The core principles—VPC routing, IAM least privilege, container orchestration with Kubernetes, and Infrastructure as Code—are universally portable. Deep mastery of AWS networking and IAM is far more impressive to hiring managers than shallow overviews across three different clouds.

Are single-container web app deployments acceptable on a senior DevOps resume?

No. Deploying a single application container on an EC2 instance demonstrates basic junior developer competence, not platform engineering. Senior portfolios must showcase infrastructure as code, multi-tenant RBAC, progressive rollouts with automated rollbacks, and high-availability distributed systems design.

Related Platform Architecture & Learning Guides