
## Introduction

This is Part 1 of our Container Networking Deep Dive series. In this hands-on deep dives, we show how to simulate container-like networking using Linux primitives: ip netns, veth pairs, and routing tables.

We're building it all from scratch. Think of it like plugging an Ethernet cable between two interface cards — one inside a Linux network namespace (our "container") and the other in the host.

### Why Should You Care?

Understanding the low-level mechanics and first principles behind container networking — without relying on Docker or Kubernetes — gives you ultimate debugging power. This series is designed to demystify how containers, pods, bridges, overlays and CNI plugins actually work under the hood.

### Scenario Overview

Prerequisites:

- Make sure to have `multipass` installed (if on macOS) or have a VM setup via UTM/VirtualBox.

We’ll set up:

- A VM using Multipass
- A network namespace (what containers are under the hood)
- A veth pair connecting host and namespace
- IP addresses on both ends
- Routing so they can talk

Goal: ping the namespace from the host and vice versa.

## Deep Dive: Step-by-Step Setup

### Provisioning the VM

Create a Makefile to provision the VM and transfer the files to it.

```bash
NAME=netns-lab
IMAGE=22.04
SCRIPT=scenario1.sh

up:
	@if multipass info $(NAME) >/dev/null 2>&1; then \
		echo "$(NAME) already exists. Skipping launch."; \
	else \
		echo "Launching $(NAME) VM..."; \
		multipass launch --name $(NAME) --cloud-init cloud-init.yaml --memory 1G --disk 5G; \
	fi
	@chmod +x env.sh scenario1.sh test.sh
	@echo "Transferring files to VM..."
	@for file in env.sh scenario1.sh test.sh; do \
		multipass transfer $$file $(NAME):/home/ubuntu/ || echo "Failed to transfer $$file"; \
	done
```

env.sh is a simple script to set up the environment variables.

```bash
CON="netns1"
IP="10.200.1.1"
```

```bash
#!/bin/bash -e

. /home/ubuntu/env.sh

echo "[+] Creating the namespace"
sudo ip netns add $CON

echo "[+] Creating the veth pair"
sudo ip link add veth1 type veth peer name veth2

echo "[+] Moving one end to namespace"
sudo ip link set veth2 netns $CON

echo "[+] Assigning IP inside namespace"
sudo ip netns exec $CON ip addr add $IP dev veth2

echo "[+] Bringing up interfaces"
sudo ip netns exec $CON ip link set veth2 up
sudo ip netns exec $CON ip link set lo up
sudo ip link set veth1 up

echo "[+] Routing setup"
sudo ip route add $IP/32 dev veth1 || true
sudo ip netns exec $CON ip route add default via $IP dev veth2 || true
```

We setup:

- We create a namespace called `netns1`
- We create a veth pair called `veth1` and `veth2`
- We move one end to the namespace
- We assign an IP address to the interface in the namespace
- We bring up the interfaces
- We add a route to the namespace from the host
- We add a default route to the namespace from the host

### Test connectivity

```bash
#!/bin/bash

NS=netns1
IP=10.200.1.1

check() {
  echo "[TEST] $1"
  eval "$2"
  echo ""
}

fail_if_empty() {
  if [ -z "$($1)" ]; then
    echo "[FAIL] $2"
    exit 1
  else
    echo "[PASS] $2"
  fi
}

check "Namespace exists" "sudo ip netns list | grep -q $NS && echo 'Found namespace $NS'"
fail_if_empty "sudo ip netns list | grep $NS" "Namespace '$NS' exists"

check "Interface in namespace" "sudo ip netns exec $NS ip link show veth2"
check "Interface on host" "sudo ip link show veth1"
check "Route on host to $IP" "sudo ip route | grep $IP"
check "Ping from host to namespace" "sudo ping -c 3 $IP"
check "Ping from namespace to host IP" "sudo ip netns exec $NS ping -c 3 $IP"
check "Routes inside namespace" "sudo ip netns exec $NS ip route"
```

### Rest of makefile

```bash
run:
	@echo "Running scenario script..."
	@multipass exec $(NAME) -- bash /home/ubuntu/$(SCRIPT)

shell:
	@multipass shell $(NAME)

test:
	@multipass exec $(NAME) -- bash /home/ubuntu/test.sh

destroy:
	@echo "Destroying VM..."
	@multipass delete $(NAME) --purge || echo "$(NAME) does not exist."
```

## Takeaways & Lessons learned

- `ip netns` is the real container under the hood.
- A veth pair gives you point-to-point connectivity. Similar to what a virtual Ethernet cable does.
- You control the routing tables just like in real containers.
- Network namespaces give you isolation.
- veth pairs connect those isolated spaces.
- Routing needs to be explicit.
- You can debug container networking using raw Linux tools.

## Conclusion

We’ve now created our first standalone network namespace with basic connectivity from host to namespace. This is how most CNI plugins start under the hood.

In the next part of the series, we’ll look at creating two namespaces on the same node and wiring them together using a bridge — effectively building a virtual switch.

Stay tuned for Part 2!


In the first part of our Container Networking Deep Dive, we explore how to set up a single network namespace inside a VM and connect it to the host using a veth pair.

Container Networking Deep Dive Part 1: Single Network Namespace on a VM


## Introduction

### What Are We Building? 🧱

In this second part of the series, we connect two network namespaces (netns1, netns2) to the same virtual bridge on a single VM. This mirrors how Kubernetes pods share a flat L2 network on a single node.

### Why This Matters

This setup is the basis of many container networks. It replicates the behavior of a Linux bridge or a virtual switch (like what Docker and most CNIs do).

## Scenario Overview

- Two namespaces: `netns1`, `netns2`
- Each has a veth interface (`veth11`, `veth21`)
- The host side of the veth pairs (`veth10`, `veth20`) is connected to a Linux bridge `br0`
- The bridge has IP `172.16.0.1`
- `netns1` gets `172.16.0.2`, `netns2` gets `172.16.0.3`
- All interfaces are on the same /24 subnet

## Architecture

![cns2](/images/cns2.png)

## Setup

We'll use the same setup script as in the previous part.

```bash
NAME=scenario2
IMAGE=22.04

up:
	@if multipass info $(NAME) >/dev/null 2>&1; then \
		echo "$(NAME) already exists. Skipping launch."; \
	else \
		echo "Launching $(NAME)..."; \
		multipass launch --name $(NAME) --memory 1G --disk 5G; \
	fi
	@chmod +x env.sh setup.sh test.sh
	@echo "Transferring files..."
	@for f in env.sh setup.sh test.sh; do \
		multipass transfer $$f $(NAME):/home/ubuntu/; \
	done
```

Now run the VM setup.

```bash
make up
```

env.sh contains the variables for the scenario.

```bash
CON1="netns1"
CON2="netns2"
NODE_IP="10.0.0.10"
BRIDGE_IP="172.16.0.1"
IP1="172.16.0.2"
IP2="172.16.0.3"
```

### Creating the Namespaces & Interfaces

setup.sh creates the namespaces and interfaces.

```bash
#!/bin/bash -e
. /home/ubuntu/env.sh

sudo ip netns add $CON1
sudo ip netns add $CON2

sudo ip link add veth10 type veth peer name veth11
sudo ip link add veth20 type veth peer name veth21

sudo ip link set veth11 netns $CON1
sudo ip link set veth21 netns $CON2

sudo ip netns exec $CON1 ip addr add $IP1/24 dev veth11
sudo ip netns exec $CON2 ip addr add $IP2/24 dev veth21

sudo ip netns exec $CON1 ip link set veth11 up
sudo ip netns exec $CON2 ip link set veth21 up
```

### Connecting via Bridge

```bash
sudo ip link add name br0 type bridge
sudo ip link set veth10 master br0
sudo ip link set veth20 master br0
sudo ip addr add $BRIDGE_IP/24 dev br0
```

### Bridging everything together

```bash
sudo ip link set br0 up
sudo ip link set veth10 up
sudo ip link set veth20 up
sudo ip netns exec $CON1 ip link set lo up
sudo ip netns exec $CON2 ip link set lo up
```

### Routing configuration

```bash
sudo ip netns exec $CON1 ip route add default via $BRIDGE_IP dev veth11
sudo ip netns exec $CON2 ip route add default via $BRIDGE_IP dev veth21
```

## Manual testing

Conditions to test:

- Ping from `netns1` to `netns2`
- Ping from `netns2` to `netns1`
- Ping from host to `netns1` & host to `netns2`
- Ping from `netns1` to host & `netns2` to host. 

## AutomatedTesting

test.sh contains the tests for the scenario.

```bash
#!/bin/bash
. /home/ubuntu/env.sh

check() {
  echo "[TEST] $1"
  eval "$2"
  echo ""
}

fail_if_empty() {
  if [ -z "$($1)" ]; then
    echo "[FAIL] $2"
    exit 1
  else
    echo "[PASS] $2"
  fi
}

fail_if_empty "sudo ip netns list | grep $CON1" "Namespace $CON1 exists"
fail_if_empty "sudo ip netns list | grep $CON2" "Namespace $CON2 exists"

check "IP in $CON1" "sudo ip netns exec $CON1 ip a show dev veth11"
check "IP in $CON2" "sudo ip netns exec $CON2 ip a show dev veth21"
check "Ping $IP2 from $CON1" "sudo ip netns exec $CON1 ping -c 3 $IP2"
check "Ping $IP1 from $CON2" "sudo ip netns exec $CON2 ping -c 3 $IP1"
check "Ping bridge from $CON1" "sudo ip netns exec $CON1 ping -c 3 $BRIDGE_IP"
check "Ping bridge from $CON2" "sudo ip netns exec $CON2 ping -c 3 $BRIDGE_IP"
check "Bridge info" "ip a show br0"
```

Run with `make test`.

## Makefile workflow

```bash
run:
	multipass exec $(NAME) -- bash /home/ubuntu/setup.sh
test:
	multipass exec $(NAME) -- bash /home/ubuntu/test.sh
shell:
	multipass shell $(NAME)
destroy:
	multipass delete $(NAME) --purge || echo "Nothing to destroy."
```

```bash
make run # Run the setup script
make test # Run the test script
make shell # Drop into the VM
make destroy # Clean up
```

## Key Takeaways

- A Linux bridge behaves like a virtual switch
- All devices on the same bridge can talk L2 directly
- Namespaces need explicit routing (default route via bridge IP)
- Debugging tools like `ip a`, `ip r`, `ip link` help massively when it comes to networking & troubleshooting.

## Conclusion

You now have a working L2 network across multiple isolated namespaces. This is the basis for pod networking on a single node.

Next up in Part 3: we’ll span this setup across two VMs, and show how to keep everything on the same subnet using a shared L2 segment or VXLAN overlay.


In the second part of our Container Networking Deep Dive, we connect two network namespaces via a bridge on the same Linux host.

Posts tagged with: linux