Mo AbukarMo Abukar

Deep Dive into EC2 Networking

2022-05-07

Deep Dive into EC2 Networking

When working with Amazon EC2, networking isn’t just a checkbox—it's the core of how instances connect, communicate, and scale. One of the most fundamental components in EC2 networking is the Elastic Network Interface (ENI).

Elastic Network Interface (ENI)

An ENI is essentially a virtual network card in the cloud. It acts as a bridge between your EC2 instance and your Virtual Private Cloud (VPC). Every EC2 instance must be launched with a primary network interface, which is automatically created unless you explicitly provide a custom one.

Key Characteristics:

  • Every ENI has:

    • A primary private IPv4 address (static)

    • Optionally one or more secondary private IPv4 addresses

    • One or more security groups

    • A MAC address

    • A source/destination check flag

  • ENIs are standalone resources and can be moved between EC2 instances.

  • EC2 instances can attach multiple ENIs (limits depend on instance type).

IP Addressing

  • Primary Private IPv4 Address:

    • Assigned to the ENI via DHCP

    • Static for the lifetime of the ENI

    • Persists even if the ENI is detached from an instance

  • Secondary Private IPv4 Addresses:

    • Useful for multi-tenant applications or apps needing multiple IPs

    • Also attached to the ENI, not the EC2 instance

  • Public IPv4 Addressing:

    • Automatically assigned only if the subnet has "auto-assign public IP" enabled or manually specified

    • These are ephemeral and released on stop/terminate

  • Elastic IPs:

    • Static public IPv4 address

    • Can be attached to a specific private IP on an ENI

    • Charged when not attached to a running instance

Advanced Use Cases with Multiple ENIs

  1. Dual-Homed Instances

  • Separate traffic by roles:

    • Web requests on one ENI

    • Backend/database traffic on another

  1. High Availability (HA)

  • Failover with secondary ENIs:

    • A secondary ENI with its own private IP is used as the access point

    • If Instance A fails, move the ENI to Instance B

    • Client continues using the same IP without needing DNS updates

  1. MAC Address Licensing

  • Some legacy software ties licenses to MAC addresses:

  • Attach ENI with known MAC to a new instance

  • Software continues working without re-licensing

  1. Security Appliances

  • Build your own virtual firewall or proxy:

  • One ENI receives traffic

  • Instance runs the firewall software

  • Another ENI routes traffic to backend

Security Control Points

ENIs are also where security groups and NACLs are applied:

  • Security Groups: Stateful, attached to ENIs

  • NACLs: Stateless, attached to subnets

By using multiple ENIs, you can apply different security profiles per interface.

Deployment Strategies: Bootstrapping vs. AMI Baking

Bootstrapping

  • Attach a user data script to EC2 at launch

  • Uses cloud-init to:

    • Install packages

    • Configure apps

    • Register with systems (e.g., load balancers, config mgmt)

Pros:

  • Flexible per environment

  • Config/data never baked in

Cons:

  • Slower provisioning ("ready for service lag")

AMI Baking

  • Install & configure app on a running instance

  • Create an AMI from that instance

  • Launch new instances from that AMI

Pros:

  • Fast provisioning

  • Predictable and repeatable

Cons:

  • Less flexibility for last-minute config

  • Must update and re-bake for changes

Combined Architecture (Best Practice)

  • Install base app stack, perform time-consuming setup, create an AMI

  • Launch instances from baked AMI, but pass custom config via user data

This gives you fast launch time with just-enough flexibility:

  • Baked AMI handles the heavy lifting (installations, dependencies)

  • Bootstrap config customizes per environment (e.g., ENV vars, credentials, instance-specific metadata)

Real-World Example:

Imagine a three-tier app:

  • Web Tier (public subnet)

  • App Tier (private subnet)

  • DB Tier (private subnet)

The App Tier EC2 instance can be multi-homed:

  • ENI1 in Web Subnet: Handles traffic from the Web Tier

  • ENI2 in DB Subnet: Handles traffic to the database

  • Different SGs/NACLs for each ENI to segregate access control

  • If the App Tier fails, detach ENI1 and ENI2 and reattach to a hot standby instance—instant failover with no DNS propagation needed.

Summary

ENIs are at the heart of EC2 networking:

  • Control IPs, MACs, and security

  • Enable flexible architectures like dual-homing, HA, firewalls, and licensing portability

  • Allow you to separate traffic types and security contexts

When it comes to instance provisioning:

  • Use AMI baking for speed

  • Use bootstrapping for flexibility

  • Use both for production-grade performance and maintainability

  • Start small: one interface, one IP. Then scale out with ENIs and Elastic IPs to build truly production-grade cloud infrastructure.

Further Reading:

Related Posts