
## 🔍 Overview

Private API Gateway allows you to expose REST APIs that are only accessible inside your VPC. No public internet access. This is ideal for internal microservices, backend systems, or APIs you want fully private.

### Scenario
We'll implement a secure internal **Employee Directory API** using AWS Lambda, API Gateway (private), and VPC Interface Endpoints. The API will be accessible **only inside the VPC**.

## 🧱 Prerequisites

- AWS CLI or Terraform
- A VPC with private subnets
- curl or Postman
- Basic understanding of:
  - API Gateway
  - Lambda
  - IAM
  - VPC endpoints (interface)

## 🛠️ Architecture

```
[ EC2 in VPC ] ---> [ VPC Endpoint ] ---> [ Private API Gateway ] ---> [ Lambda (Employee Directory) ]
```

Only resources inside the VPC can reach the API through the interface endpoint.

## 🔧 Step-by-Step Deployment (Terraform)

### 1. Create VPC, Subnet, and Security Group

```go
resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true
}

resource "aws_subnet" "private" {
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.0.1.0/24"
  availability_zone       = "us-east-1a"
  map_public_ip_on_launch = false
}

resource "aws_security_group" "vpce_sg" {
  name        = "vpce-sg"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [aws_vpc.main.cidr_block]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
```

### 2. Deploy the Lambda Function

Use the Python code below and package it:

```bash
mkdir employee-directory && cd employee-directory
vi index.py  # lets use our mini lambda below
zip employee_directory.zip index.py
```

```python
import json

EMPLOYEES = {
    "1001": {"id": "1001", "name": "Qais N", "role": "DevOps Engineer"},
    "1002": {"id": "1002", "name": "Abz Ab", "role": "Backend Engineer"},
    "1003": {"id": "1003", "name": "James John", "role": "Product Manager"},
}

def lambda_handler(event, context):
    path = event.get("path", "")
    method = event.get("httpMethod", "")

    if method == "GET" and path.startswith("/employee/"):
        emp_id = path.split("/")[-1]
        emp = EMPLOYEES.get(emp_id)
        if emp:
            return {"statusCode": 200, "body": json.dumps(emp)}
        return {"statusCode": 404, "body": json.dumps({"error": "Employee not found"})}

    if method == "GET" and path == "/employee":
        return {"statusCode": 200, "body": json.dumps(list(EMPLOYEES.values()))}

    return {"statusCode": 400, "body": json.dumps({"error": "Unsupported operation"})}
```

```go
resource "aws_iam_role" "lambda_exec" {
  name = "lambda_exec_role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Action = "sts:AssumeRole",
      Principal = { Service = "lambda.amazonaws.com" },
      Effect = "Allow",
    }],
  })
}

resource "aws_lambda_function" "employee_directory" {
  filename         = "employee_directory.zip"
  function_name    = "employee_directory"
  role             = aws_iam_role.lambda_exec.arn
  handler          = "index.lambda_handler"
  runtime          = "python3.9"
  source_code_hash = filebase64sha256("employee_directory.zip")
}
```

### 3. Create Private API Gateway and Integration

```go
resource "aws_api_gateway_rest_api" "private_api" {
  name        = "employee-directory-api"
  endpoint_configuration { types = ["PRIVATE"] }
}

resource "aws_api_gateway_resource" "employee" {
  rest_api_id = aws_api_gateway_rest_api.private_api.id
  parent_id   = aws_api_gateway_rest_api.private_api.root_resource_id
  path_part   = "employee"
}

resource "aws_api_gateway_method" "get" {
  rest_api_id   = aws_api_gateway_rest_api.private_api.id
  resource_id   = aws_api_gateway_resource.employee.id
  http_method   = "GET"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "lambda_integration" {
  rest_api_id             = aws_api_gateway_rest_api.private_api.id
  resource_id             = aws_api_gateway_resource.employee.id
  http_method             = aws_api_gateway_method.get.http_method
  type                    = "AWS_PROXY"
  integration_http_method = "POST"
  uri                     = aws_lambda_function.employee_directory.invoke_arn
}
```

### 4. Configure VPC Endpoint and Policy

```go
resource "aws_vpc_endpoint" "api_gw" {
  vpc_id             = aws_vpc.main.id
  service_name       = "com.amazonaws.${var.region}.execute-api"
  vpc_endpoint_type  = "Interface"
  subnet_ids         = [aws_subnet.private.id]
  security_group_ids = [aws_security_group.vpce_sg.id]
}

resource "aws_api_gateway_deployment" "deployment" {
  rest_api_id = aws_api_gateway_rest_api.private_api.id
  stage_name  = "prod"
  depends_on  = [aws_api_gateway_integration.lambda_integration]
}

resource "aws_api_gateway_rest_api_policy" "restrict_to_vpce" {
  rest_api_id = aws_api_gateway_rest_api.private_api.id
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Deny",
        Principal = "*",
        Action = "execute-api:Invoke",
        Resource = "*",
        Condition = {
          StringNotEquals = {
            "aws:SourceVpce" = aws_vpc_endpoint.api_gw.id
          }
        }
      }
    ]
  })
}
```

## 🧪 Test the Setup

1. SSH into an EC2 inside the VPC
2. Run:

```bash
curl https://<rest-api-id>.execute-api.<region>.amazonaws.com/prod/employee
curl https://<rest-api-id>.execute-api.<region>.amazonaws.com/prod/employee/1002
```

✅ Should return valid responses inside the VPC (inside EC2)
❌ Should be blocked from public internet

---

## 📌 Summary

- Built a fully private Employee Directory API
- Integrated it with Lambda + API Gateway
- Made it VPC-only using Interface Endpoints
- Secured access with SourceVpce IAM policy

Great for internal-only microservices, B2B APIs, or compliance-sensitive systems.

🧼 Destroy with:

```bash
terraform destroy
```

Part 2 coming out soon on: DynamoDB, PrivateLink cross-VPC, or IAM auth with SigV4!
---

Learn how to deploy a secure, private-only API Gateway inside your VPC using interface endpoints, resource policies, and VPC integration.

Securing APIs in AWS: Private API Gateway + VPC Endpoint Deep Dive


## Introduction

In this deep dive, we'll explore how to configure AWS Route 53 to route user traffic to the region with the lowest latency and automatically fail over to another region if the primary becomes unhealthy. This setup ensures high availability and optimal performance for global applications.

Why Should You Care?
Implementing latency-based routing with health checks in Route 53 allows your application to:

- Serve users from the nearest healthy region, reducing latency.
- Automatically reroute traffic during regional outages, enhancing resilience.
- Maintain high availability without manual intervention.

## Prerequisites

- Basic understanding of AWS Route 53 and DNS concepts.
- An AWS account with Route 53 and CloudWatch access.
- A domain name registered with Route 53 (if hosted on Cloudflare or elsewhere, make sure you delegate the domain to Route 53 or create a subdomain to subdelegate).
- Terraform installed on your local machine.

## Architecture Overview

We'll deploy identical applications in two AWS regions: us-east-1 and eu-west-1. Each region will have:

- An Application Load Balancer (ALB) fronting the application.
- A Route 53 health check monitoring the ALB's /health endpoint.
- A latency-based DNS record directing traffic to the region with the lowest latency.

If a health check fails, Route 53 will exclude that region from DNS responses, effectively failing over to the healthy region.

## Step-by-Step Guide

1. Set Up ALBs in Both Regions
Deploy your application behind an ALB in both us-east-1 (US Virginia) and eu-west-1 (EU Ireland). Ensure each ALB has a listener on port 80 and a target group with healthy targets.

2. Create Route 53 Health Checks
Define health checks for each ALB's /health endpoint.

```go
resource "aws_route53_health_check" "us_east" {
  fqdn              = "alb-us-east-1.example.com"
  port              = 80
  type              = "HTTP"
  resource_path     = "/health"
  failure_threshold = 3
  request_interval  = 30
}

resource "aws_route53_health_check" "eu_west" {
  fqdn              = "alb-eu-west-1.example.com"
  port              = 80
  type              = "HTTP"
  resource_path     = "/health"
  failure_threshold = 3
  request_interval  = 30
}
```

## 3. Configure Latency-Based DNS Records

Create latency-based DNS records pointing to each ALB, associating them with the respective health checks.

```go
resource "aws_route53_record" "us_east" {
  zone_id = aws_route53_zone.primary.zone_id
  name    = "r53-demo.moabukar.co.uk"
  type    = "A"

  alias {
    name                   = "alb-us-east-1.example.com"
    zone_id                = "Z35SXDOTRQ7X7K" // Replace with your ALB's zone ID
    evaluate_target_health = true
  }

  set_identifier = "us-east-1"
  region         = "us-east-1"
  latency_routing_policy {
    region = "us-east-1"
  }

  health_check_id = aws_route53_health_check.us_east.id
}

resource "aws_route53_record" "eu_west" {
  zone_id = aws_route53_zone.primary.zone_id
  name    = "r53-demo.moabukar.co.uk"
  type    = "A"

  alias {
    name                   = "alb-eu-west-1.example.com"
    zone_id                = "Z32O12XQLNTSW2" // Replace with your ALB's zone ID
    evaluate_target_health = true
  }

  set_identifier = "eu-west-1"
  region         = "eu-west-1"
  latency_routing_policy {
    region = "eu-west-1"
  }

  health_check_id = aws_route53_health_check.eu_west.id
}
```

Note: Replace zone_id values with the correct zone IDs for your ALBs. You can find these in the AWS documentation.

## Apply on TF

```bash
terraform init
terraform apply
```

Verify that the DNS records are created and health checks are active.

## Test the Setup

- Normal Operation: When both regions are healthy, Route 53 directs users to the region with the lowest latency based on their location.

- Simulate Failure: Stop the application in us-east-1 to trigger a health check failure.

- Failover: Route 53 detects the failure and stops including us-east-1 in DNS responses. Traffic is rerouted to eu-west-1.

- Recovery: Restart the application in us-east-1. Once the health check passes, Route 53 includes it again in DNS responses.

## Monitoring and Observability

- Route 53 Console: Monitor health check statuses and DNS records.

- CloudWatch: Set up alarms for health check failures to receive notifications.

- Logs: Enable query logging in Route 53 to analyze DNS queries.

## Conclusion

By configuring latency-based routing with health checks in Route 53, you ensure your application serves users from the nearest healthy region, providing low latency and high availability. This setup is crucial for global applications where performance and uptime are paramount.

References:

- [AWS Route 53 Latency-Based Routing](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy-latency.html)

- [AWS Route 53 Health Checks](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-failover.html)

- [Terraform AWS Provider Documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_health_check)


A hands-on guide to configuring AWS Route 53 for latency-based routing across multiple regions, incorporating health checks for automatic failover.

Route 53 Deep Dive: Multi-Region Latency Routing with Health-Based Failover


## 🔍 Overview

AWS PrivateLink enables secure, private connectivity between VPCs and services hosted on AWS without exposing traffic to the public internet. It's essential for secure service-to-service communication across accounts or VPCs.

In this post, we'll implement a working PrivateLink setup with Terraform: one VPC exposing a service (via NLB), and another VPC accessing it (via interface endpoint). We'll use two Terraform providers to simulate cross-account setup.

---

## 🛠️ Architecture

```
[ Service Provider VPC ]                  [ Service Consumer VPC ]
[ EC2 (httpd) ] -> [ NLB ] -> [ Endpoint Service ] <- [ VPC Endpoint ] <- [ EC2 ]
```

The service is hosted on a private EC2 behind a Network Load Balancer. An Endpoint Service is created from the NLB. The consumer VPC accesses it through an Interface VPC Endpoint.

---

## 🧱 Prerequisites

- Terraform ≥ 1.0
- Two AWS profiles or roles simulating service provider and consumer
- Basic networking knowledge (subnets, NLB, SGs)

---

## 🧩 Step 1: Service Provider VPC with NLB and Endpoint Service

```go
provider "aws" {
  alias  = "provider"
  region = "us-east-1"
}

resource "aws_vpc" "provider" {
  provider   = aws.provider
  cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "provider_subnet" {
  provider          = aws.provider
  vpc_id            = aws_vpc.provider.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = "us-east-1a"
}

resource "aws_security_group" "provider_sg" {
  provider = aws.provider
  vpc_id   = aws_vpc.provider.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["10.0.0.0/16"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "provider_instance" {
  provider        = aws.provider
  ami             = "ami-0c94855ba95c71c99"
  instance_type   = "t2.micro"
  subnet_id       = aws_subnet.provider_subnet.id
  security_groups = [aws_security_group.provider_sg.id]
  user_data       = <<-EOF
                #!/bin/bash
                echo "Hello from PrivateLink Service Provider" > index.html
                nohup python -m SimpleHTTPServer 80 &
                EOF
}

resource "aws_lb" "nlb" {
  provider           = aws.provider
  name               = "privatelink-nlb"
  internal           = true
  load_balancer_type = "network"
  subnets            = [aws_subnet.provider_subnet.id]
}

resource "aws_lb_target_group" "tg" {
  provider    = aws.provider
  name        = "privatelink-tg"
  port        = 80
  protocol    = "TCP"
  vpc_id      = aws_vpc.provider.id
  target_type = "instance"
}

resource "aws_lb_target_group_attachment" "tg_attachment" {
  provider         = aws.provider
  target_group_arn = aws_lb_target_group.tg.arn
  target_id        = aws_instance.provider_instance.id
  port             = 80
}

resource "aws_lb_listener" "listener" {
  provider          = aws.provider
  load_balancer_arn = aws_lb.nlb.arn
  port              = 80
  protocol          = "TCP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.tg.arn
  }
}

resource "aws_vpc_endpoint_service" "service" {
  provider                    = aws.provider
  acceptance_required        = true
  network_load_balancer_arns = [aws_lb.nlb.arn]
}
```

---

## 🧩 Step 2: Service Consumer VPC with VPC Endpoint

```go
provider "aws" {
  alias  = "consumer"
  region = "us-east-1"
}

resource "aws_vpc" "consumer" {
  provider   = aws.consumer
  cidr_block = "10.1.0.0/16"
}

resource "aws_subnet" "consumer_subnet" {
  provider          = aws.consumer
  vpc_id            = aws_vpc.consumer.id
  cidr_block        = "10.1.1.0/24"
  availability_zone = "us-east-1a"
}

resource "aws_security_group" "consumer_sg" {
  provider = aws.consumer
  vpc_id   = aws_vpc.consumer.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["10.1.0.0/16"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_vpc_endpoint" "consumer_endpoint" {
  provider           = aws.consumer
  vpc_id             = aws_vpc.consumer.id
  service_name       = aws_vpc_endpoint_service.service.service_name
  vpc_endpoint_type  = "Interface"
  subnet_ids         = [aws_subnet.consumer_subnet.id]
  security_group_ids = [aws_security_group.consumer_sg.id]
}
```

---

## 🧪 Testing

- Accept the VPC Endpoint connection from the Provider side.
- SSH into an EC2 in the Consumer VPC and use:

```bash
curl http://<interface-endpoint-dns-name>
```

You should see:

```
Hello from PrivateLink Service Provider
```

---

## 🔐 Security Notes

- Ensure correct SGs between NLB and Endpoint
- Use IAM condition keys or resource policies if needed
- Enable VPC Flow Logs for visibility

---

## 📌 Summary

- We implemented AWS PrivateLink with real infra using Terraform
- Service is exposed internally via NLB and Interface Endpoint
- Setup is secure, scalable, and avoids internet exposure

Next steps? Add DNS, IAM auth, or cross-region routing!

---

A hands-on technical guide to implementing AWS PrivateLink between VPCs using Terraform.

AWS PrivateLink with Terraform


![Kafka on K8s screenshot](/images/example.png)

# Deploying Kafka on Kubernetes with Strimzi

In this article, we'll walk through deploying an Apache Kafka cluster on a local Kubernetes cluster using [Kind](https://kind.sigs.k8s.io/) (Kubernetes in Docker). We'll use the [Strimzi operator](https://strimzi.io/) for Kafka, which simplifies the deployment and management of Kafka on Kubernetes. We'll also touch on how you could incorporate Terraform to manage some of the setup or perform upgrades.

## Prerequisites

- **Docker Desktop** installed
- **Kind** installed
- **kubectl** installed

## 1. Create a Local Kubernetes Cluster with Kind

First, create a Kind cluster. For a simple default cluster:

```bash
kind create cluster --name kafka-demo
```

Verify your cluster:

```bash
kubectl cluster-info --context kind-kafka-demo
```


## 2. Install the Strimzi Operator

2.1 Using kubectl
Strimzi provides YAML manifests you can apply directly:

```bash
# Download the Strimzi installation YAML
curl -L "https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.34.0/strimzi-cluster-operator-0.34.0.yaml" -o strimzi.yaml
```

# Apply to your cluster

```bash
kubectl create namespace kafka
kubectl apply -f strimzi.yaml -n kafka
```

## 2.2 Using Helm (Optional)

```bash
helm repo add strimzi https://strimzi.io/charts/
helm repo update
helm install strimzi-kafka-operator strimzi/strimzi-kafka-operator --namespace kafka --create-namespace
```

## 1. Deploy a Kafka Cluster

Create a kafka-cluster.yaml file:

```yaml
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-kafka-cluster
  namespace: kafka
spec:
  kafka:
    replicas: 3
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
    storage:
      type: ephemeral
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}
```

```
kubectl apply -f kafka-cluster.yaml -n kafka
```

Strimzi will spin up your Kafka brokers and Zookeeper pods.


## 1. Verifying Kafka

Once the operator has created the Kafka cluster, check your pods:

```bash

kubectl get pods -n kafka

You should see:
```bash
                                   READY   STATUS    RESTARTS   AGE
my-kafka-cluster-kafka-0                1/1     Running   0          2m
my-kafka-cluster-kafka-1                1/1     Running   0          2m
my-kafka-cluster-kafka-2                1/1     Running   0          2m
my-kafka-cluster-zookeeper-0            1/1     Running   0          2m
my-kafka-cluster-zookeeper-1            1/1     Running   0          2m
my-kafka-cluster-zookeeper-2            1/1     Running   0          2m
strimzi-cluster-operator-...            1/1     Running   0          3m
```

## 6. Testing Kafka

You can create a Kafka topic and produce/consume messages:

```bash
# Create a topic

kubectl exec -it my-kafka-cluster-kafka-0 -n kafka -- \
  bin/kafka-topics.sh --create --topic test-topic --partitions 1 --replication-factor 1 --bootstrap-server my-kafka-cluster-kafka-bootstrap:9092

# Produce messages
kubectl exec -it my-kafka-cluster-kafka-0 -n kafka -- \
  bin/kafka-console-producer.sh --topic test-topic --bootstrap-server my-kafka-cluster-kafka-bootstrap:9092

# Consume messages
kubectl exec -it my-kafka-cluster-kafka-1 -n kafka -- \
  bin/kafka-console-consumer.sh --topic test-topic --from-beginning --bootstrap-server my-kafka-cluster-kafka-bootstrap:9092

Type a few messages in the producer terminal, and they should appear in the consumer terminal.
```

## 7. Clean Up

When you’re done, remove the cluster:

```bash
kind delete cluster --name kafka-demo
```

## Conclusion

With this setup, you have a quick local environment for developing or testing Kafka. The Strimzi operator abstracts away the complexity of Kafka management on Kubernetes. If you want to scale to a production environment, you’ll switch from ephemeral storage to persistent volumes, integrate with your cloud platform, and further tune your Terraform and Strimzi configs.


A step-by-step guide to setting up a Kafka cluster on a local Kind cluster using the Strimzi operator, with optional Terraform provisioning.

Posts tagged with: terraform