Event-driven state management: lessons from Redis pub/sub in a React UI

Most frontend state management articles are about client-side state: what goes in Redux, when to use Context, how to avoid prop drilling. Those are real problems, but they're not the most interesting ones.

The interesting problem is: how do you keep the UI in sync with a backend that's doing work asynchronously, at variable speed, across multiple stages?

This is the problem we had with DocMind's document processing pipeline. A user uploads a document, and then — somewhere on a cluster — Textract OCR runs, chunks are generated, LLMs enrich the fields, confidence scores are computed. All of this takes 10–90 seconds. The user is waiting.

The naive answer is polling. The better answer taught us something about the line between backend and frontend concerns.

Why polling fails at this problem

We started with polling. Every 2 seconds, the frontend called a status endpoint. This was simple to implement and immediately revealed its problems:

• Stale intervals. At 2 seconds, users sometimes waited almost 2 full seconds to see a stage transition that had completed immediately. At 500ms, we were hammering the API for no reason most of the time.
• No backpressure. Multiple browser tabs meant multiple clients polling simultaneously. The endpoint had no coordination mechanism.
• False confidence. The endpoint returned the last known status. If the status update pipeline had a lag, the UI would show an outdated state confidently.

Redis pub/sub as the event source

The backend had Redis. Each document had a unique ID. The processing pipeline already published events to a Redis channel as stages completed. We just needed to get those events to the browser.

Architecture:

1. Document upload → returns document ID to client
2. Processing pipeline publishes stage events to doc:{documentId}:events Redis channel
3. Next.js API route subscribes to that Redis channel via SSE
4. Client connects to /api/documents/{id}/stream and receives events

// app/api/documents/[id]/stream/route.ts
export async function GET(req: Request, { params }: { params: { id: string } }) {
  const session = await getServerSession(req);
  if (!session) return new Response(null, { status: 401 });
 
  const stream = new ReadableStream({
    async start(controller) {
      const redis = new Redis(process.env.REDIS_URL);
      const channel = `doc:${params.id}:events`;
 
      redis.subscribe(channel, (message) => {
        const event = `data: ${message}\n\n`;
        controller.enqueue(new TextEncoder().encode(event));
      });
 
      // Clean up on disconnect
      req.signal.addEventListener("abort", () => {
        redis.unsubscribe(channel);
        redis.quit();
        controller.close();
      });
    },
  });
 
  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      Connection: "keep-alive",
    },
  });
}

The client side

The frontend connects to the SSE stream using EventSource, updates local state as events arrive, and shows a live progress stepper:

function useDocumentStream(documentId: string) {
  const [stages, setStages] = useState<PipelineStage[]>([]);
 
  useEffect(() => {
    const source = new EventSource(`/api/documents/${documentId}/stream`);
 
    source.onmessage = (e) => {
      const event = JSON.parse(e.data) as PipelineEvent;
      setStages((prev) => updateStages(prev, event));
    };
 
    return () => source.close();
  }, [documentId]);
 
  return stages;
}

updateStages is a pure function — takes the current state and an event, returns new state. No mutation, easy to test.

What we got wrong first

Not handling reconnection. SSE connections drop. The browser's EventSource auto-reconnects, but it sends a Last-Event-ID header that we weren't handling server-side. We added event IDs to every published event and a replay mechanism on reconnect that sends the last 20 events for that document.

Leaking Redis connections. The req.signal.abort cleanup worked for explicit disconnects but not for server restarts. We added a TTL on Redis subscriptions (auto-unsubscribe after 10 minutes of inactivity) as a safety valve.

One Redis connection per SSE stream. With concurrent users, this scaled poorly. We switched to a single Redis subscriber in the Next.js process that multiplexed events to active SSE streams by document ID.

What this taught us

The event-driven pattern solved the wrong-level problem of polling (the API didn't know when to serve fresh data) by moving the timing concern where it belonged: to the system that knows when things change.

This is the broader lesson: when frontend state feels hard to keep in sync with a backend process, the question to ask is not "how do I poll more efficiently?" but "where in the system does the event originate, and how do I route it to the UI?"

Sometimes the answer is WebSockets. Sometimes it's Realtime channels. For our use case — unidirectional, process-complete events — SSE was the right fit: simpler than WebSockets, lighter than a polling loop, and well-supported by EventSource.