Mo AbukarMo Abukar

Apache Pulsar Playground: Running Pulsar Locally on kind with Dashboards, Clients, and Admin Tools

2024-05-22

🎡 Introduction

In this blog, I’ll walk you through setting up a full-featured Apache Pulsar playground using kind (Kubernetes in Docker). Whether you’re testing Pulsar for learning or demoing a real pub/sub model with admin tools and monitoring, this setup gives you everything:

  • ✅ Pulsar cluster deployed via Helm
  • ✅ Prometheus + Grafana for monitoring
  • ✅ Pulsar Manager UI
  • ✅ Python producer/consumer using pulsar-client
  • ✅ Proxy + dashboard exposure
  • ✅ Hands-on with Pulsar admin CLI

🧰 Prerequisites

🔧 Step 1: Create a kind Cluster with Port Mappings

Create a local cluster that maps the UI and proxy ports:

cat <<EOF | kind create cluster --name pulsar --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 8080  # Pulsar REST Broker
    hostPort: 8080
  - containerPort: 6650  # Pulsar Binary Protocol
    hostPort: 6650
  - containerPort: 7750  # Pulsar Manager UI
    hostPort: 7750
  - containerPort: 3000  # Grafana
    hostPort: 3000
EOF

Install Pulsar with Helm

helm repo add apache https://pulsar.apache.org/charts
helm repo update

git clone https://github.com/apache/pulsar-helm-chart
cd pulsar-helm-chart

./scripts/pulsar/prepare_helm_release.sh -n pulsar -k pulsar-mini -c

helm install \
  --values examples/values-minikube.yaml \
  --namespace pulsar \
  pulsar-mini apache/pulsar

kubectl get pods -n pulsar

Produce and Consume Messages

Get Proxy info:

kubectl get svc -n pulsar | grep pulsar-mini-proxy

Run a simple message test:

kubectl exec -it -n pulsar pulsar-mini-toolset-0 -- /bin/bash

Create a tenant, namespace, and topic:

bin/pulsar-admin tenants create apache
bin/pulsar-admin namespaces create apache/pulsar
bin/pulsar-admin topics create-partitioned-topic apache/pulsar/test-topic -p 4

Produce messages:

bin/pulsar-client produce apache/pulsar/test-topic -m "Hello Apache Pulsar" -n 10

Consume messages:

bin/pulsar-client consume apache/pulsar/test-topic -s sub -n 0

Pulsar Manager UI

kubectl exec -it pulsar-mini-pulsar-manager-0 -n pulsar -- /bin/bash

# Create superuser
CSRF_TOKEN=$(curl http://localhost:7750/pulsar-manager/csrf-token)
curl \
  -H "X-XSRF-TOKEN: $CSRF_TOKEN" \
  -H "Cookie: XSRF-TOKEN=$CSRF_TOKEN;" \
  -H 'Content-Type: application/json' \
  -X PUT http://localhost:7750/pulsar-manager/users/superuser \
  -d '{"name": "pulsar", "password": "pulsar", "description": "test", "email": "user@test.org"}'

Password secrets:

kubectl get secret pulsar-mini-pulsar-manager-secret -n pulsar -o jsonpath="{.data}" | jq -r 'to_entries|map("\(.key)=\(.value|@base64d)")|.[]'

Access Pulsar Manager:

📍 http://localhost:7750 Login: pulsar / pulsar

Grafana Dashboard

kubectl port-forward svc/pulsar-mini-grafana -n pulsar 3000:80

Access Grafana:

📍 http://localhost:3000

Credentials:

kubectl get secret pulsar-mini-grafana -n pulsar -o jsonpath="{.data}" | jq -r 'to_entries|map("\(.key)=\(.value|@base64d)")|.[]'

Pulsar Python Client

pip install pulsar-client

Create producer.py:

import pulsar

client = pulsar.Client('pulsar://localhost:6650')
producer = client.create_producer('apache/pulsar/test-topic')

for i in range(10):
    producer.send(('msg-%d' % i).encode('utf-8'))

client.close()

Create consumer.py:

import pulsar

client = pulsar.Client('pulsar://localhost:6650')
consumer = client.subscribe('apache/pulsar/test-topic', 'sub')

for i in range(10):
    msg = consumer.receive()
    print(f"Received: {msg.data()}")
    consumer.acknowledge(msg)

client.close()

Topic Stats

curl http://localhost:8080/admin/v2/persistent/public/default/my-topic/stats | python -m json.tool

Cleanup

helm uninstall pulsar-mini -n pulsar
kind delete cluster --name pulsar

🧠 Final Thoughts

This Pulsar playground setup gives you a fully working Pulsar cluster on Kind, ready for real pub/sub workflows and admin use. It’s perfect for experimentation, workshops, or prototyping.

Related Posts