Amazon EKS Auto Mode with Terraform

Find out how to run your EKS in auto mode making use of Terraform (IaC) and how to deploy a simple 2048 game app with public ALB.

Introduction

Amazon Elastic Kubernetes Service (EKS) Auto Mode is a fully managed Kubernetes deployment option designed to simplify the operational complexities of running containers at scale. With Auto Mode, AWS takes care of infrastructure provisioning, cluster management, and optimization, allowing developers to focus solely on deploying and running their applications.

Unlike traditional EKS clusters, which require users to manage node groups and networking configurations, EKS Auto Mode abstracts these complexities, providing an effortless, cost-optimised, and secure Kubernetes experience.

Key Features of Amazon EKS Auto Mode

Fully Managed Infrastructure

• AWS automatically provisions and scales the underlying compute resources, removing the need for manual node management.
• Ensures high availability and resilience by handling infrastructure failures automatically.

Cost Optimisation

• Auto Mode optimises compute costs by dynamically adjusting resources based on workload demands.
• Supports serverless Kubernetes experience by running workloads efficiently without the need to pre-define instance types or sizes.

Simplified Cluster Networking

• Eliminates the need for users to configure VPCs (you still need a VPC), security groups, and networking policies, making it easier to deploy applications securely.
• AWS manages network connectivity between Kubernetes workloads, reducing operational overhead.

Automatic Scaling

• Provides built-in horizontal and vertical scaling to match application demand without manual intervention.
• Ensures workloads remain responsive while optimising resource utilisation.

Built-in Security and Compliance

• Comes with pre-configured IAM roles, security policies, and encryption settings to meet AWS security best practices.
• Supports AWS PrivateLink for secure communication between services within a VPC.

Seamless Integration with AWS Services

• Natively integrates with AWS services like Amazon RDS, DynamoDB, Amazon S3, and AWS Lambda, making it easier to build cloud-native applications.
• Supports AWS IAM, CloudWatch, and Secrets Manager for centralised security and monitoring.

Zero Cluster Maintenance

• AWS manages patching, updates, and Kubernetes version upgrades, ensuring clusters remain up-to-date and secure.
• Reduces the burden on DevOps teams by automating maintenance tasks.

Developer-Friendly Experience

• Developers can deploy applications using familiar Kubernetes tools (kubectl, Helm, etc.) without worrying about infrastructure complexities.
• Provides an optimised Kubernetes experience, enabling teams to focus on writing and deploying applications rather than managing clusters.

Terraform Implementation of Amazon EKS Auto Mode

locals {
  cluster_name = "sandbox-vpc-eks-test"
}

module "vpc_eks" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.18.1"

  name = "sandbox-vpc-eks-test"

  cidr                  = "10.20.0.0/19"

  azs             = ["eu-west-2a", "eu-west-2b", "eu-west-2c"]
  private_subnets = ["10.20.0.0/21", "10.20.8.0/21", "10.20.16.0/21"]
  public_subnets  = ["10.20.24.0/23", "10.20.26.0/23", "10.20.28.0/23"]

  enable_nat_gateway     = true
  single_nat_gateway     = true
  one_nat_gateway_per_az = false

  enable_vpn_gateway = true

  enable_dns_hostnames = true
  enable_dns_support   = true

  propagate_private_route_tables_vgw = true
  propagate_public_route_tables_vgw  = true

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = "1",
    "mapPublicIpOnLaunch"             = "FALSE"
    "karpenter.sh/discovery"          = local.cluster_name
    "kubernetes.io/role/cni"          = "1"
  }

  public_subnet_tags = {
    "kubernetes.io/role/elb" = "1",
    "mapPublicIpOnLaunch"    = "TRUE"
  }

  tags = {
    "kubernetes.io/cluster/${local.cluster_name}" = "shared"
  }
}

resource "aws_eks_cluster" "cluster" {
  name     = local.cluster_name
  role_arn = aws_iam_role.cluster.arn
  version  = "1.32"

  vpc_config {
    subnet_ids              = module.vpc_eks.private_subnets
    security_group_ids      = []
    endpoint_private_access = "true"
    endpoint_public_access  = "true"
  }

  access_config {
    authentication_mode                         = "API"
    bootstrap_cluster_creator_admin_permissions = false
  }

  bootstrap_self_managed_addons = false

  zonal_shift_config {
    enabled = true
  }

  compute_config {
    enabled       = true
    node_pools    = ["general-purpose", "system"]
    node_role_arn = aws_iam_role.node.arn
  }

  kubernetes_network_config {
    elastic_load_balancing {
      enabled = true
    }
  }

  storage_config {
    block_storage {
      enabled = true
    }
  }
}

resource "aws_iam_role" "cluster" {
  name = "eks-test-cluster-role"

  assume_role_policy = data.aws_iam_policy_document.cluster_role_assume_role_policy.json
}

resource "aws_iam_role_policy_attachments_exclusive" "cluster" {
  role_name = aws_iam_role.cluster.name
  policy_arns = [
    "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy",
    "arn:aws:iam::aws:policy/AmazonEKSComputePolicy",
    "arn:aws:iam::aws:policy/AmazonEKSBlockStoragePolicy",
    "arn:aws:iam::aws:policy/AmazonEKSLoadBalancingPolicy",
    "arn:aws:iam::aws:policy/AmazonEKSNetworkingPolicy",
    "arn:aws:iam::aws:policy/AmazonEKSServicePolicy",
    "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  ]
}

data "aws_iam_policy_document" "cluster_role_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole", "sts:TagSession"]

    principals {
      type        = "Service"
      identifiers = ["eks.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "node" {
  name = "eks-auto-node-example"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = ["sts:AssumeRole"]
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      },
    ]
  })
}

resource "aws_iam_role_policy_attachment" "node_AmazonEKSWorkerNodeMinimalPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodeMinimalPolicy"
  role       = aws_iam_role.node.name
}

resource "aws_iam_role_policy_attachment" "node_AmazonEC2ContainerRegistryPullOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPullOnly"
  role       = aws_iam_role.node.name
}

Sample 2048 Game application deployment with ALB

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: default
  name: deployment-2048
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: app-2048
  replicas: 3
  template:
    metadata:
      labels:
        app.kubernetes.io/name: app-2048
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          minDomains: 2
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app.kubernetes.io/name: app-2048
      containers:
        - image: alexwhen/docker-2048
          imagePullPolicy: Always
          name: app-2048
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  namespace: default
  name: service-2048
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  selector:
    app.kubernetes.io/name: app-2048
---
apiVersion: eks.amazonaws.com/v1
kind: IngressClassParams
metadata:
  name: eks-auto-alb
spec:
  scheme: internet-facing
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: eks-auto-alb
spec:
  controller: eks.amazonaws.com/alb
  parameters:
    apiGroup: eks.amazonaws.com
    kind: IngressClassParams
    name: eks-auto-alb
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: default
  name: ingress-2048-public
  annotations:
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
spec:
  ingressClassName: eks-auto-alb
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: service-2048
                port:
                  number: 80

The above yaml defines deployment, service and ingress resources which will deploy 2048 game application. Please note the importance of IngressClass resource “ingressClassName: eks-auto-alb”, it is required to be implemented otherwise ALB won’t be created.

Final Thoughts

If you’re looking for an easy, serverless-like Kubernetes experience, EKS Auto Mode is the best choice. However, if you need more control over infrastructure and networking, EKS Standard Mode is the better fit.