Amazon EKS, RBAC and IAM access

Using IAM to access Kubernetes cluster and namespaces

Overview

Kubernetes RBAC

In Kuberentes world RBAC API declares four top-level types, they are Role, ClusterRole, RoleBinding and ClusterRoleBinding. Users can interact with these resources as they would with any other API resource (via kubectl, API calls, etc.).

In the RBAC API, a role contains rules that represent a set of permissions. Permissions are purely additive (there are no “deny” rules). A role can be defined within a namespace with a Role, or cluster-wide with a ClusterRole.

A role binding grants the permissions defined in a role to a user or group of users. It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted. Permissions can be granted within a namespace with a RoleBinding, or cluster-wide with a ClusterRoleBinding.

EKS, IAM, Authentication and Authorisation

Namespaces are intended for use in environments with many users spread across multiple teams, or projects. For clusters with a few to tens of users, you should not need to create or think about namespaces at all. Start using namespaces when you need the features they provide. Namespaces provide a scope for names. Names of resources need to be unique within a namespace, but not across namespaces. Namespaces can not be nested inside one another and each Kubernetes resource can only be in one namespace. Namespaces are a way to divide cluster resources between multiple users (via resource quota).

With EKS you can take advantage of using powerful AWS IAM implementation. EKS authentication layers supports both IAM Users and IAM Roles. Amazon EKS uses aws-iam-authenticator for authentication and Kubernetes RBAC for authorisation. It is a tool to use AWS IAM credentials to authenticate to a Kubernetes cluster. The initial work on this tool was driven by Heptio. The project receives contributions from multiple community engineers and is currently maintained by Heptio and Amazon EKS OSS Engineers.

In this story, I would like to demonstrate how to set up an IAM role for authentication and assign a RBAC role to scope the API calls allowed. This will be done for both IAM users and Roles. I will also cover how to allow specific IAM user/role to manage only a single namespace, preventing them from interacting with other namespaces.

Auth ConfigMap

By default, whoever creates an EKS cluster, that IAM User or IAM Role will have full admin access to it. In order to add additional Users/Roles there is a ConfigMap which is used for management of Users or IAM Roles for the Cluster.

Below config is the base configuration for aws-auth ConfigMap which allows worker nodes with InstanceRole (arn:aws:iam::111111111:role/worker-node-instance-role) to join the EKS control plane.

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: arn:aws:iam::111111111:role/worker-node-instance-role
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes

If you decide to add additional access for existing Roles/Users, your config will become something like:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: arn:aws:iam::111111111:role/worker-node-instance-role
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes
    - userarn: arn:aws:iam::111111111:role/marcincuber-role
      username: marcincuber-role
  mapUsers: |
    - userarn: arn:aws:iam::111111111:user/marcincuber
      username: marcincuber

At this point User marcincuber and Role marcincuber-role will be able to call API server with valid credentials. However, they won’t be able to access any resources as RBAC roles are not configured.

Manage specific namespaces with IAM

Create namespace

kubectl create namespace develop

Create IAM role

In the Console, create a role: marcincuber-role, this will ensure that the ConfigMap defined above doesn't need to be changed. There is no need to add IAM permissions. You will only need to add a Trust Relationship:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111:root"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

The above allows anyone in the account to assume the role. This is not ideal and should be restricted as much as possible. Full documentation can be found here.

At this point you have a working IAM role which can be assumed. I will leave the method of assuming role to you.

Create an RBAC Role to namespace

# role.yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: develop
  name: full_access
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]

Apply the role:

kubectl apply -f role.yaml

Create RoleBinding for Role

# rolebinding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: full_access_role_binding
  namespace: develop
subjects:
- kind: User
  name: marcincuber-role
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: full_access
  apiGroup: rbac.authorization.k8s.io

Apply the RoleBinding:

kubectl apply -f rolebinding.yaml

At this point if you assume role arn:aws:iam::111111111:role/marcincuber-role and create kubeconfig for that role you will have full admin access to develop namespace.

Generate kubeconfig for IAM role

To cover the above point, you can simply run:

aws eks update-kubeconfig --name YOUR_CLUSTER_NAME

Manage access using centralised ClusterRole

Similar principals apply here, so we create the following:

# clusterrole.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: cr_full_access
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]
# rolebinding_cr.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: full_access_role_binding_cr
  namespace: develop
subjects:
- kind: User
  name: marcincuber-role
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: full_access
  apiGroup: rbac.authorization.k8s.io

In this case you will grant the same permissions as previously and marcincuber-role will have admin permissions in develop namespace.

ClusterRole and ClusterRoleBinding

In case you want to grant permissions for your IAM role in all namespaces, you can simply use ClusterRoleBinding as follows:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: full_access_cluster_role_binding
subjects:
- kind: User
  name: marcincuber-role
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: cr_full_access
  apiGroup: rbac.authorization.k8s.io

Conclusion

I presented basic examples of using AWS IAM with EKS RBAC. All RBAC resources were covered with examples. It can definitely be used as starting point for many of you and improved based on your requirements. To read more about RBAC please -> Refer to RBAC documentation.

In collaboration with NewRelic a technical blog on EKS and my work can be found -> https://blog.newrelic.com/product-news/news-uk-content-capabilities-amazon-eks-new-relic/