
## Introduction

In [Part 1](./ecs-fargate-deep-dive-part-1.md), we pulled back the curtain on **how ECS Fargate works**: Firecracker microVMs, ENIs, IAM, and the hidden host fleet.  

Now it’s time to get hands-on. We’ll spin up Firecracker locally, boot a microVM, and even simulate the ENI networking model.  

### Why Should You Care?

Running Firecracker yourself helps you:  

- See how AWS runs **each Fargate task inside its own VM**.  
- Understand **boot performance** and why Fargate cold starts take 30–60s.  
- Learn how ENIs map to **VPC-level isolation**.  
- Debug networking limits by replicating them locally.  

### Scenario Overview

We’ll simulate a Fargate task locally:  

- Launch a Firecracker microVM.  
- Attach a minimal Linux root filesystem.  
- Configure a TAP interface for networking (like an ENI).  
- Map this setup back to ECS Fargate’s architecture.  

Goal: see how isolation and networking work in practice.  

---

## Deep Dive: Step-by-Step Setup

### Prerequisites

Run this on a Linux host (Ubuntu VM, EC2, or Multipass).  

```bash
sudo apt-get update
sudo apt-get install -y git curl wget build-essential \
    unzip cmake pkg-config libseccomp-dev

# Download Firecracker

curl -LOJ https://github.com/firecracker-microvm/firecracker/releases/download/v1.8.0/firecracker-v1.8.0-x86_64.tgz
tar -xvf firecracker-v1.8.0-x86_64.tgz
sudo mv release-v1.8.0-x86_64/firecracker /usr/local/bin/

### Kernel & RootFS

curl -LO https://s3.amazonaws.com/spec.ccfc.min/img/hello/kernel/vmlinux.bin
curl -LO https://s3.amazonaws.com/spec.ccfc.min/img/hello/fsfiles/x86_64/rootfs.ext4


### Launching a MicroVM

rm -f /tmp/firecracker.socket
firecracker --api-sock /tmp/firecracker.socket &

# VM config + boot source
curl --unix-socket /tmp/firecracker.socket -i \
  -X PUT 'http://localhost/machine-config' \
  -H 'Content-Type: application/json' \
  -d '{"vcpu_count":1,"mem_size_mib":512,"ht_enabled":false}'

# Kernel
curl --unix-socket /tmp/firecracker.socket -i \
  -X PUT 'http://localhost/boot-source' \
  -H 'Content-Type: application/json' \
  -d '{"kernel_image_path":"./vmlinux.bin","boot_args":"console=ttyS0 reboot=k panic=1 pci=off"}'

# RootFS
curl --unix-socket /tmp/firecracker.socket -i \
  -X PUT 'http://localhost/drives/rootfs' \
  -H 'Content-Type: application/json' \
  -d '{"drive_id":"rootfs","path_on_host":"./rootfs.ext4","is_root_device":true,"is_read_only":false}'

## Start the VM

curl --unix-socket /tmp/firecracker.socket -i \
  -X PUT 'http://localhost/actions' \
  -H 'Content-Type: application/json' \
  -d '{"action_type":"InstanceStart"}'

## You’ll see a minimal Linux VM booting instantly.

###Networking Simulation

### Fargate gives each task its own ENI. We’ll mimic this with a TAP device:

sudo ip tuntap add tap0 mode tap
sudo ip addr add 172.16.0.1/24 dev tap0
sudo ip link set tap0 up


### Attach it to Firecracker:

curl --unix-socket /tmp/firecracker.socket -i \
  -X PUT 'http://localhost/network-interfaces/eth0' \
  -H 'Content-Type: application/json' \
  -d '{"iface_id":"eth0","host_dev_name":"tap0","guest_mac":"02:FC:00:00:00:01"}'


### Inside the VM, bring up eth0 and assign an IP. Now the microVM talks to the host — exactly how a Fargate task talks to your VPC.

```

### Mapping Back to Fargate

- Firecracker VM → Your Fargate task’s isolated runtime.

- RootFS → Base container environment + ECS runtime + your container image.

- TAP device → Equivalent of a VPC ENI.

- API config → ECS Scheduler telling Fargate what to run.

### Architecture

```mermaid
flowchart TD
    subgraph LocalLab["Local Firecracker Lab"]
        subgraph Firecracker["MicroVM"]
            Kernel[Linux Kernel]
            RootFS[RootFS + Container Runtime]
            IF1[eth0 (TAP)]
        end
        Host["Linux Host"] --> Tap[tap0]
        Tap --> IF1
    end

    subgraph AWSFargate["AWS Fargate Analogy"]
        subgraph Task1["Fargate Task"]
            MVM1[Firecracker microVM]
            ENI1[ENI (VPC IP)]
        end
        VPC1[VPC Subnet + Security Group]
    end

    Tap -. maps to .-> ENI1
    Firecracker -. maps to .-> MVM1
    Host -. maps to .-> VPC1

```

### Takeaways

- Each Fargate task is a microVM, not just a container.

- ENI-per-task makes tasks first-class citizens in your VPC.

- Running Firecracker locally shows why Fargate cold starts exist.

- Debugging ENIs locally helps anticipate VPC IP exhaustion issues.

---

### Conclusion

By running Firecracker locally, you’ve just recreated the core of ECS Fargate. Every time you launch a task in AWS, the same steps happen — at massive scale, across AWS’s hidden fleet.

In Part 3, we’ll dive into real-world gotchas: ENI scaling limits, cold start tuning, and how observability works when you don’t control the host.


In the second part of our ECS Fargate Deep Dive, we get hands-on with Firecracker — the lightweight VMM that powers Fargate — and simulate task isolation and networking locally.

ECS Fargate Deep Dive Part 2: Firecracker in Action


## Introduction

This is Part 1 of our ECS Fargate Deep Dive series. In these posts, we’ll strip away the abstraction and look at what’s really happening under the hood when you launch a Fargate task.  

We’ll cover internals like **Firecracker microVMs, ENI networking, IAM roles, storage and scheduling** - the things AWS doesn’t show you in the console. 

### ECS Fargate: Behind the Scenes of Serverless Containers

When you launch a task on ECS Fargate, it feels like magic — no EC2 instances, no cluster autoscaling, just containers running. But what’s actually happening under the hood?

Let’s peel back the layers.

### Why Should You Care?

Fargate feels “serverless for containers” — but knowing how it works helps you:  

- Debug **networking issues** (e.g. ENI/IP exhaustion).  
- Understand **cold starts and scaling behaviour**.  
- Make **cost vs flexibility trade-offs** between ECS on EC2 and Fargate.  
- See why Fargate is safer in multi-tenant workloads.  

### What Fargate Really Is

Fargate isn’t a special kind of ECS cluster. It’s an AWS-managed capacity layer that runs your containers in lightweight microVMs. You pick CPU, memory, networking, and IAM roles — AWS handles the hosts, patching, and kernel security. 

## Deep Dive: What Actually Happens

### Firecracker MicroVMs

- Each Fargate task = one Firecracker microVM.  
- Built on **KVM**, stripped down for speed.  
- Isolation is stronger than plain Docker namespaces.  
- Each has its own kernel, cgroups, and seccomp profile.  

This is why tasks don’t share kernels, unlike ECS on EC2.  

### Networking

When you start a Fargate task:

- AWS provisions an Elastic Network Interface (ENI) in your VPC.
- Every task gets its own **Elastic Network Interface (ENI)**.  
- The ENI is placed directly in your VPC subnet and directly to the microVM.  
- Security groups/NACLs apply directly to the task.  
- From a network perspective, tasks behave like first-class EC2 instances. This is why Fargate tasks look and feel like EC2 instances on the network.
 
### Storage & Security

- **Ephemeral storage**: 20 GiB by default (up to 200 GiB). Dies with the task.
- **Persistent storage**: use EFS mounts if needed.  
- **IAM Roles for Tasks**: AWS injects temporary credentials directly into the microVM.
- No SSH/SSM access — AWS fully manages the hosts.  

### Scheduling & Scaling

1. ECS control plane tells Fargate: “Run this task with X CPU, Y memory.”  
2. Fargate provisions a microVM, attaches an ENI, pulls the image and starts the task.
3. Scaling = AWS spinning up/destroying microVMs behind the scenes.

## Performance & Limits

- **Cold starts**: ~30–60s (microVM boot + ENI attach + image pull).  
- **Resource limits**: 0.25–16 vCPUs, 0.5–120 GiB RAM.  
- **Restrictions**: no privileged mode, no GPUs, no custom kernels. Can't run DinD (docker in docker) in a task. 
- **Cost model**: billed per vCPU-second + GiB-second.  

---

### ECS on EC2 vs ECS Fargate

| Feature       | ECS on EC2                         | ECS Fargate                          |
|---------------|------------------------------------|--------------------------------------|
| Host control  | You manage AMIs & patching         | AWS manages hosts                     |
| Isolation     | Shared kernel (namespaces)         | Firecracker microVM per task          |
| Networking    | Shared ENI or `awsvpc`             | Dedicated ENI per task                |
| Flexibility   | Privileged, GPU, custom AMI        | Limited, no host access               |
| Cost model    | Pay per instance (even idle)       | Pay per task-second                   |


---

## Architecture Diagram  

```mermaid
flowchart TD
    ECS[ECS Control Plane] -->|Schedules Task| AWS_Fargate["AWS Fargate Host Fleet"]

    subgraph AWS_Fargate
        subgraph Firecracker1["Task A (Firecracker microVM)"]
            C1[Containers]
            ENI1[ENI]
        end
        subgraph Firecracker2["Task B (Firecracker microVM)"]
            C2[Containers]
            ENI2[ENI]
        end
    end

    ENI1 --> VPC1[VPC Subnet + Security Group]
    ENI2 --> VPC2[VPC Subnet + Security Group]
```

### Takeaways

- Each Fargate task runs in its own microVM, not a shared host kernel.

- Networking is first-class: ENI per task, direct VPC integration.

- You give up host control in exchange for security + simplicity.

- Cold starts, ENI limits, and no privileged mode are the main gotchas.

### How to Think of It

Fargate is basically AWS Lambda for containers:

- Instead of functions, you run containers.

- Instead of ephemeral processes, you get microVMs.

- You trade host control for simplicity, security, and auto-scaling.



In the first part of our ECS Fargate Deep Dive, we break down what happens behind the scenes when you run a task on Fargate — Firecracker microVMs, ENIs, IAM and the hidden host fleet.

Posts tagged with: ecs