
## Kubernetes as a Cloud Control Plane: Deep Dive into AWS ACK with kind

Kubernetes isn't just about container orchestration anymore — it's become the de facto control plane for everything. With Custom Resource Definitions (CRDs) and controllers, we can now describe and manage virtually any resource declaratively — whether it's an app, an S3 bucket, or an RDS instance. That’s where AWS Controllers for Kubernetes (ACK) come in.

ACK lets us manage AWS resources like RDS or EC2 the same way we manage Deployments and Services: using YAML manifests. The idea is powerful — replace scattered IaC tooling with a unified K8s-native approach. And if you’re already deploying everything through Argo CD, Flux, or Helm, why not include your cloud infra?

But does it actually hold up?

That’s what this post explores — we’ll walk through an end-to-end demo running ACK on a local kind cluster. 

⚙️ Environment Setup

```bash
mkdir ack-demo
cd ack-demo
```

🐳 Start Docker and Create Your Local Cluster
Ensure Docker is running. We’ll use it to spin up a local Kubernetes cluster using kind.

```bash
kind create cluster --config kind.yaml
```

```yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: dot
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |-
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 8080
    hostPort: 8080
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
```

dot.nu:

```bash
#!/usr/bin/env nu

source  scripts/common.nu
source  scripts/kubernetes.nu
source  scripts/ack.nu
source  scripts/crossplane.nu

def main [] {}

def "main setup" [] {

    rm --force .env

    main create kubernetes aws

    kubectl create namespace a-team

    kubectl --namespace a-team apply --filename rds-password.yaml

    main apply ack

    main apply crossplane --preview true --provider aws

    (
        kubectl apply 
            --filename crossplane-providers/cluster-role.yaml
    )

    (
        kubectl apply
            --filename crossplane-providers/provider-kubernetes-incluster.yaml
    )

    kubectl apply --filename dot-sql-config.yaml

    main wait crossplane

    main print source
    
}

def "main destroy" [] {

    main destroy kubernetes aws

    main delete ack
}
```

```bash
devbox shell
chmod +x dot.nu
./dot.nu setup
source .env
```

The kind.yaml file defines networking + node settings tailored for ACK.

## 🧬 ACK CRDs: AWS Infra as K8s Resources

Once the ACK controllers are installed, Kubernetes gains a bunch of new CRDs that represent AWS services — from RDS and S3 to EC2 subnets and VPCs.

Let’s take a look:

```bash
kubectl get crds | grep k8s.aws
```

You’ll see CRDs like:

```bash
dbinstances.rds.services.k8s.aws
vpcs.ec2.services.k8s.aws
subnets.ec2.services.k8s.aws
securitygroups.ec2.services.k8s.aws
...
```

Each CRD represents an AWS resource you can now define using a Kubernetes manifest. Think of it as Terraform, but via kubectl apply.

## 🚀 Deploying the ACK RDS Controller (locally with kind)

Since we’re using kind, not EKS, there’s no IRSA — so we authenticate with static AWS credentials injected into a K8s secret.

You already did this with ./dot.nu setup, but for clarity:

```bash
kubectl create ns ack-system

kubectl create secret generic ack-creds \
  -n ack-system \
  --from-literal=AWS_ACCESS_KEY_ID=<key> \
  --from-literal=AWS_SECRET_ACCESS_KEY=<secret>
```

Install the RDS ACK controller with Helm:

```bash
helm repo add ack https://aws.github.io/eks-charts
helm repo update

helm install ack-rds-controller ack/ack-rds-controller \
  -n ack-system \
  --set aws.region=us-east-1 \
  --set aws.secret.name=ack-creds
```

Now your cluster has a controller watching DBInstance resources and creating matching RDS instances in AWS.

## 🏗️ Defining an RDS PostgreSQL Instance (and dependencies)

Creating a DB in AWS isn’t one resource — it’s VPCs, Subnets, Gateways, RouteTables, and finally the DBInstance.

Here’s a peek from rds.yaml:

```yaml
apiVersion: ec2.services.k8s.aws/v1alpha1
kind: VPC
metadata:
  name: my-db
spec:
  cidrBlock: 11.0.0.0/16
---
apiVersion: ec2.services.k8s.aws/v1alpha1
kind: InternetGateway
metadata:
  name: my-db
spec:
  vpcRef:
    from:
      name: my-db
---
apiVersion: ec2.services.k8s.aws/v1alpha1
kind: Subnet
metadata:
  name: my-db-a
spec:
  cidrBlock: 11.0.1.0/24
  availabilityZone: us-east-1a
  vpcRef:
    from:
      name: my-db
...
apiVersion: rds.services.k8s.aws/v1alpha1
kind: DBInstance
metadata:
  name: my-db
  annotations:
    services.k8s.aws/region: us-east-1
spec:
  dbInstanceClass: db.t3.micro
  engine: postgres
  engineVersion: "16.3"
  allocatedStorage: 20
  masterUsername: masteruser
  masterUserPassword:
    name: my-db-password
    key: password
  dbInstanceIdentifier: my-db
  publiclyAccessible: true
  dbSubnetGroupRef:
    from:
      name: my-db
  vpcSecurityGroupRefs:
    - from:
        name: my-db
Also, create a Secret with the DB password:
```

Also, create a Secret with the DB password:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: my-db-password
stringData:
  password: MySuperSecretPassw0rd!
```

Apply all resources:

```bash
kubectl apply -f rds-password.yaml
kubectl apply -f rds.yaml
```

Check the instance status:

```bash
kubectl get dbinstances.rds.services.k8s.aws
```

Once it's STATUS=available, your RDS instance is live.

## 🔍 Observability in ACK: Where’s the Feedback?

Once you start working with ACK, you’ll quickly hit a snag: visibility into resource states is... primitive at best.

You might try this:

```bash
kubectl -n a-team get \
  vpcs.ec2.services.k8s.aws,internetgateways.ec2.services.k8s.aws,routetables.ec2.services.k8s.aws,securitygroups.ec2.services.k8s.aws,subnets.ec2.services.k8s.aws,dbsubnetgroups.rds.services.k8s.aws,dbinstances.rds.services.k8s.aws
```

Output:

```bash
NAME    ID                       STATE
my-db   vpc-07cdab5bba559b994   available
...
my-db   subnet-0340...          available
...
my-db   db-2NOHJMPDDGYBPY6MH... creating
```
You get barebones info like STATE or STATUS, but it’s inconsistent and often misleading. Some resources use STATE, others use STATUS, and many don’t update accurately.

### 😡 Case Study: Broken Subnet

```bash
kubectl describe subnet.ec2.services.k8s.aws my-db-x
```

Yields:

```bash
Message: api error InvalidParameterValue: Value (us-east-1x) for parameter availabilityZone is invalid
Type:    ACK.Terminal
```

That’s actually helpful — the AZ doesn’t exist. But now try describing a working DBInstance:

```bash
kubectl describe dbinstance.rds.services.k8s.aws my-db
```

You’ll get a dump of AWS metadata with no intuitive indication of what’s actually happening. You have to hunt down .status.dbInstanceStatus and hope it says something like available or configuring-enhanced-monitoring.

No Ready condition. No events. No unified status format. ACK controllers don’t expose standard Kubernetes conditions. That’s not just annoying — it breaks integrations with Argo CD, Crossplane, and others.

Posts tagged with: ack