Crossplane and Localstack

2024-03-21

Crossplane + LocalStack on kind

Goal: Spin up a local Kubernetes cluster with Crossplane, provision AWS resources (S3, SQS, EC2, Lambda + API Gateway) against LocalStack — no real AWS account needed.

🏗️ Why Crossplane + LocalStack?

Challenge | Solution

Cloud bills & IAM friction for dev environments | Emulate AWS with LocalStack inside the cluster
Declarative multi‑cloud IaC in Git | Crossplane CRDs & GitOps
Fast inner‑loop testing for platform engineers | Fast inner‑loop testing for platform engineers

Lab layout

crossplane-localstack-lab/
├── bootstrap.sh              # End‑to‑end cluster + Crossplane + LocalStack installer
├── kind-config.yaml          # (Optional) custom kind config
├── Makefile                  # Convenience targets (up / down / status)
├── manifests/
│   ├── localstack-values.yaml  # Helm values
│   ├── secret.yml              # Fake AWS creds (for ProviderConfig)
│   ├── providerconfig.yml      # Points every provider at LocalStack URL
│   ├── provider‑aws‑*.yml      # Six slim providers (S3, SQS, EC2, Lambda, API GW, IAM)
│   ├── s3-bucket.yml
│   ├── sqs-queue.yml
│   ├── ec2.yml
│   ├── lambda.yml
│   ├── apigw.yml
│   └── role.yml
└── lambda/
    ├── main.py               # basic lambda function
    └── demo-lambda.zip       # artifact (zip of main.py)

⚙️ Prerequisites

Docker ≥ 24 (Desktop or Engine)
kind ≥ 0.22 brew install kind
kubectl ≥ 1.29
helm ≥ 3.14
awslocal (nice‑to‑have wrapper) pip install awscli-local

No AWS credentials required.

Bootstrap

#!/usr/bin/env bash
set -euo pipefail

CLUSTER_NAME=crossplane-localstack-lab
NAMESPACE=crossplane-system
LOCALSTACK_URL="http://localstack.${NAMESPACE}.svc.cluster.local:4566"

info()  { printf '\033[1;32m[+] %s\033[0m\n' "$1"; }
fail()  { printf '\033[1;31m[✗] %s\033[0m\n' "$1" >&2; exit 1; }
need()  { command -v "$1" >/dev/null || fail "'$1' not installed"; }

info "Checking CLIs …"; for c in kind kubectl helm; do need "$c"; done

info "Creating kind cluster '$CLUSTER_NAME' (idempotent) …"
kind get clusters | grep -q "^${CLUSTER_NAME}$" || kind create cluster --name "$CLUSTER_NAME"

info "Installing Crossplane …"
kubectl create ns "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm upgrade --install crossplane crossplane-stable/crossplane -n "$NAMESPACE"

info "Installing LocalStack (Helm) …"
helm repo add localstack-repo https://helm.localstack.cloud
helm upgrade --install localstack localstack-repo/localstack \
  -n "$NAMESPACE" -f manifests/localstack-values.yaml

info "Waiting for LocalStack …"
kubectl wait --for=condition=Ready pod -l app.kubernetes.io/name=localstack -n "$NAMESPACE" --timeout=180s

info "Applying secrets + providers …"
for f in secret provider-aws-s3 provider-aws-sqs provider-aws-ec2 provider-aws-apigw provider-aws-lambda provider-aws-iam; do
  kubectl apply -f manifests/${f}.yml
done

info "Waiting for provider reconciliation …"
for p in s3 sqs; do
  kubectl wait provider.pkg.crossplane.io/provider-aws-${p} --for=condition=Healthy --timeout=120s || fail "provider-${p} unhealthy"
done

info "Applying ProviderConfig + sample resources …"
kubectl apply -f manifests/providerconfig.yml
kubectl apply -f manifests/s3-bucket.yml manifests/sqs-queue.yml manifests/ec2.yml manifests/role.yml manifests/lambda.yml manifests/apigw.yml

info "✅  Crossplane & LocalStack are ready.  Try: \n   kubectl get bucket,queue,instance,function -A\n   awslocal s3 ls --endpoint-url $LOCALSTACK_URL"

Give it a shebang (chmod +x bootstrap.sh) and run:

./bootstrap.sh

📜 2 — Helm Values for LocalStack

service:
  type: ClusterIP
image:
  tag: 3.3.0
# Speed up boot — no persistence needed for dev
persistence:
  enabled: false
startServices: "s3,sqs,ec2,lambda,apigateway,iam"
extraEnv:
  - name: AWS_ACCESS_KEY_ID
    value: test
  - name: AWS_SECRET_ACCESS_KEY
    value: test
  - name: DEFAULT_REGION
    value: us-east-1 # coz localstack is in us-east-1

🗝️ 3 — AWS Cred Secret

Mock secrets for Localstack

apiVersion: v1
kind: Secret
metadata:
  name: localstack-aws-creds
  namespace: crossplane-system
stringData:
  creds: |
    [default]
    aws_access_key_id = test
    aws_secret_access_key = test

🌐 4 — ProviderConfig (Points to LocalStack)

apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: localstack-aws-creds
      key: creds
  endpoint:
    url: http://localstack.crossplane-system.svc.cluster.local:4566
    hostnameImmutable: true
    signingRegion: us-east-1

📦 5 — AWS Providers

S3 Provider

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-s3
spec:
  package: xpkg.upbound.io/crossplane-contrib/provider-aws-s3:v0.47.0

SQS Provider

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-sqs
spec:
  package: xpkg.upbound.io/crossplane-contrib/provider-aws-sqs:v0.47.0

(…repeat for EC2, Lambda, API GW, IAM — change s3 above to the service name.)

🏗️ 6 — AWS Sample Managed Resources

S3 Bucket

apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
  name: crossplane-test-bucket
spec:
  forProvider:
    acl: private
  providerConfigRef:
    name: default

SQS Queue

apiVersion: sqs.aws.upbound.io/v1beta1
kind: Queue
metadata:
  name: crossplane-demo-queue
spec:
  forProvider:
    delaySeconds: 0
    messageRetentionSeconds: 86400
  providerConfigRef:
    name: default

EC2 Instance

apiVersion: ec2.aws.upbound.io/v1beta1
kind: Instance
metadata:
  name: crossplane-demo
spec:
  forProvider:
    ami: "ami-12345678" ## localstack fake ami
    instanceType: t2.micro
    tags:
      Name: crossplane-demo
  providerConfigRef:
    name: default

IAM role & Lambda function

apiVersion: iam.aws.upbound.io/v1beta1
kind: Role
metadata:
  name: demo-lambda-role
spec:
  forProvider:
    assumeRolePolicy: |
      {"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}
  providerConfigRef:
    name: default

## Lambda func
apiVersion: lambda.aws.upbound.io/v1beta1
kind: Function
metadata:
  name: demo-lambda
spec:
  forProvider:
    region: us-east-1
    s3Bucket: crossplane-test-bucket
    s3Key: demo-lambda.zip
    handler: main.handler
    runtime: python3.9
    roleRef:
      name: demo-lambda-role
  providerConfigRef:
    name: default

API Gateway REST API & Integration

apiVersion: apigateway.aws.upbound.io/v1beta1
kind: RestAPI
metadata:
  name: demo-api
spec:
  forProvider:
    name: demo-api
    region: us-east-1
  providerConfigRef:
    name: default
---
apiVersion: apigateway.aws.upbound.io/v1beta1
kind: Deployment
metadata:
  name: demo-api-deploy
spec:
  forProvider:
    restApiIdRef:
      name: demo-api
    stageName: test
  providerConfigRef:
    name: default

(A minimal API that will auto‑wire the root “/” resource to demo-lambda using the provider’s default behaviour. For full path/method resources see the upstream examples.)

Tiny Lambda Code

def handler(event, context):
    return {
        "statusCode": 200,
        "body": "👋  from Crossplane‑managed Lambda!"
    }

pushd lambda && zip demo-lambda.zip main.py && popd

🛠️ 8 — Makefile

up: bootstrap.sh
	./bootstrap.sh

down:
	kind delete cluster --name crossplane-localstack-lab

status:
	kubectl get pkg,providerconfig,bucket,queue,instance,function -A

Testing

# Inside the cluster (CRDs)
kubectl get bucket,queue,instance,function -A

# Against LocalStack emulated AWS
export AWS_ENDPOINT=http://localhost:4566
awslocal --endpoint-url $AWS_ENDPOINT s3 ls
awslocal --endpoint-url $AWS_ENDPOINT sqs list-queues

# Hit the API Gateway - needs work.... localstack doesn't support this yet i think
API_ID=$(awslocal apigateway get-rest-apis --query 'items[?name==`demo-api`].id' --output text)
curl http://localhost:4566/restapis/$API_ID/test/_user_request_/

Clean up

make down # nuke everything