Progress · 0/7 sections
04 — WebSockets & Real-time at Scale (50M Users)
7 min read
04 — WebSockets & Real-time at Scale (50M Users)
Scenario: Live cricket scores, chat, order tracking, stock prices — yeh sab polling se nahi hota. 50M users × real-time = WebSocket at scale. Yeh easy problem nahi hai.
Real-time Protocols Comparison
Polling (setInterval every 2s):
Client: "Koi update hai?" → Server: "Nahi"
Client: "Koi update hai?" → Server: "Nahi"
Client: "Koi update hai?" → Server: "Haan, yeh raha"
❌ Wasteful (99% requests useless)
❌ High latency (up to 2s delay)
❌ Server load: 50M × 0.5 req/s = 25M req/s — impossible
Long Polling:
Client: "Koi update hai?" → Server: [holds connection until update]
Server: "Yeh raha update" → Client: immediately reconnects
✅ Lower latency than polling
❌ Still HTTP overhead per update
❌ Connection state complex
Server-Sent Events (SSE):
Client opens ONE HTTP connection.
Server PUSHES text events anytime.
Client cannot send data (server-to-client only).
✅ Simple, HTTP/2 multiplexed
✅ Auto-reconnect built into browser
✅ No special library needed
❌ Server-to-client only
❌ Text only (no binary)
WebSockets:
Full-duplex bidirectional TCP channel over HTTP upgrade.
Both client and server can send anytime.
✅ Bidirectional
✅ Binary support (protobuf, msgpack)
✅ Low latency (no HTTP headers per message)
❌ Complex at scale (sticky sessions, connection state)
❌ Firewall/proxy issues in some corporate networks
Choose:
Updates only → SSE (simpler)
Chat, gaming, collaborative → WebSocket
WebSocket Handshake
Client sends HTTP request with upgrade:
GET /ws/chat HTTP/1.1
Host: api.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
Server responds:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
After this: raw TCP frames, no more HTTP.
WebSocket Server in Go
import "github.com/gorilla/websocket"
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
CheckOrigin: func(r *http.Request) bool {
// CSRF protection: only allow your domain
origin := r.Header.Get("Origin")
return origin == "https://app.example.com"
},
}
type Client struct {
conn *websocket.Conn
userID string
send chan []byte // buffered channel for outgoing messages
}
func HandleWebSocket(w http.ResponseWriter, r *http.Request) {
// Authenticate before upgrade (JWT in query param or cookie)
userID, err := authenticateWS(r)
if err != nil {
http.Error(w, "Unauthorized", 401)
return
}
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
return
}
client := &Client{
conn: conn,
userID: userID,
send: make(chan []byte, 256), // buffer for slow clients
}
hub.Register(client)
defer hub.Unregister(client)
// Two goroutines per connection
go client.writePump() // sends messages to client
client.readPump() // reads messages from client (blocks)
}
func (c *Client) readPump() {
defer c.conn.Close()
c.conn.SetReadLimit(512 * 1024) // max message: 512KB
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
c.conn.SetPongHandler(func(string) error {
c.conn.SetReadDeadline(time.Now().Add(60 * time.Second))
return nil
})
for {
_, message, err := c.conn.ReadMessage()
if err != nil {
if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseAbnormalClosure) {
log.Println("ws error:", err)
}
return // exits, defer closes connection
}
hub.Broadcast(message)
}
}
func (c *Client) writePump() {
ticker := time.NewTicker(30 * time.Second) // ping every 30s
defer func() {
ticker.Stop()
c.conn.Close()
}()
for {
select {
case message, ok := <-c.send:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if !ok {
c.conn.WriteMessage(websocket.CloseMessage, []byte{})
return
}
if err := c.conn.WriteMessage(websocket.TextMessage, message); err != nil {
return
}
case <-ticker.C:
c.conn.SetWriteDeadline(time.Now().Add(10 * time.Second))
if err := c.conn.WriteMessage(websocket.PingMessage, nil); err != nil {
return // client gone
}
}
}
}Hub Pattern — Fan-out
type Hub struct {
clients map[*Client]bool
rooms map[string]map[*Client]bool // room → clients
broadcast chan []byte
register chan *Client
unregister chan *Client
mu sync.RWMutex
}
func (h *Hub) Run() {
for {
select {
case client := <-h.register:
h.mu.Lock()
h.clients[client] = true
h.mu.Unlock()
case client := <-h.unregister:
h.mu.Lock()
if _, ok := h.clients[client]; ok {
delete(h.clients, client)
close(client.send)
}
h.mu.Unlock()
case message := <-h.broadcast:
h.mu.RLock()
for client := range h.clients {
select {
case client.send <- message:
// sent
default:
// client's send buffer full (slow client)
// drop message OR disconnect client
close(client.send)
delete(h.clients, client)
}
}
h.mu.RUnlock()
}
}
}Scaling WebSockets — The Hard Problem
The Sticky Session Problem
Server A: Client 1, 2, 3, 4 connected (10K connections)
Server B: Client 5, 6, 7, 8 connected (10K connections)
Client 1 sends message to Client 5.
Client 1 is on Server A. Client 5 is on Server B.
Server A doesn't know about Client 5!
Solutions:
1. Sticky sessions (same user → same server always)
2. Pub/Sub backbone (Redis, Kafka) between servers
Solution: Redis Pub/Sub for Fan-out
┌─────────────┐
│ Redis │
│ Pub/Sub │
└──────┬──────┘
publish │ subscribe
┌─────────────────┤──────────────────┐
│ │ │
┌────▼────┐ ┌────▼────┐ ┌────▼────┐
│ WS │ │ WS │ │ WS │
│Server A │ │Server B │ │Server C │
│ │ │ │ │ │
│Client1 │ │Client5 │ │Client9 │
│Client2 │ │Client6 │ │Client10 │
└─────────┘ └─────────┘ └─────────┘
Client 1 (Server A) sends message:
1. Server A receives message from Client 1
2. Server A publishes to Redis channel "room:cricket_live"
3. ALL servers subscribed to "room:cricket_live" receive it
4. Each server delivers to its local connected clients in that room
Redis pub/sub is fast (~1ms) and handles this perfectly.
At huge scale: use Kafka instead (persistent, replay possible).
// Subscribe to Redis channel on server startup
func (h *Hub) SubscribeToRedis(ctx context.Context, rdb *redis.Client) {
pubsub := rdb.Subscribe(ctx, "ws:broadcast", "ws:room:*")
defer pubsub.Close()
for msg := range pubsub.Channel() {
var wsMsg WSMessage
json.Unmarshal([]byte(msg.Payload), &wsMsg)
// Deliver to local clients only
h.DeliverLocal(wsMsg)
}
}
// Send message from any service → publish to Redis → all WS servers receive
func (s *ChatService) SendMessage(ctx context.Context, roomID string, msg *Message) {
payload, _ := json.Marshal(WSMessage{Room: roomID, Data: msg})
rdb.Publish(ctx, "ws:room:"+roomID, payload)
}Connection Count Math
50M MAU. Peak concurrent: ~2M users online.
WebSocket server: can handle ~50K-100K connections (depending on RAM + goroutines)
2M connections ÷ 50K per server = 40 WebSocket servers minimum
Memory per connection:
2 goroutines × 8KB (stack) = 16KB
Send buffer: 256 × message_size
Connection overhead: ~8KB
Total: ~32KB per connection
40 servers × 50K connections × 32KB = ~64GB total memory
→ 40 × 4 core, 8GB RAM servers
→ Use spot instances (stateless after Redis pub/sub)
Backpressure — Slow Clients
Problem: Server generates 1000 messages/sec but client only consumes 10/sec.
Buffer fills up → OOM → server crashes.
Solutions:
1. Drop old messages (ring buffer):
If send buffer full, drop oldest message.
OK for: live scores (stale score not needed)
2. Drop new messages:
If buffer full, reject new.
OK for: non-critical notifications
3. Disconnect slow client:
Buffer full → close connection → client reconnects fresh.
OK for: high-frequency data streams (trading)
4. Apply back pressure (block producer):
Wait for client to consume before producing more.
OK for: reliable delivery required
Most real-time apps: drop old + send "you missed N messages, reload"
Server-Sent Events (SSE) — Simpler Option
// When bidirectional not needed (order tracking, notifications)
func OrderUpdatesSSE(w http.ResponseWriter, r *http.Request) {
// Check SSE support
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "SSE not supported", 500)
return
}
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
w.Header().Set("Access-Control-Allow-Origin", "https://app.example.com")
orderID := r.URL.Query().Get("order_id")
ctx := r.Context()
updateCh := orderService.Subscribe(ctx, orderID)
for {
select {
case update := <-updateCh:
// SSE format: "data: {json}\n\n"
fmt.Fprintf(w, "id: %d\n", update.ID)
fmt.Fprintf(w, "event: order_update\n")
fmt.Fprintf(w, "data: %s\n\n", mustJSON(update))
flusher.Flush() // send immediately, don't buffer
case <-ctx.Done():
return // client disconnected
}
}
}SSE features:
- Auto-reconnect: browser reconnects automatically
- Last-Event-ID: resume from last event after reconnect
- HTTP/2: multiple SSE streams multiplexed over one connection
When SSE > WebSocket:
Notifications, news feeds, order updates, dashboards
Any "server pushes, client just reads" scenario
Simpler to implement, no special proxies needed