Rebuilding Redis: What Happens Inside a Key-Value Store?

The Illusion of Instant Data

When you use Redis, it feels like magic. You send a SET command, and your data is stored; you send a GET, and it returns instantly. It feels like a simple dictionary, but underneath, it’s a high-performance network server handling thousands of requests per second.

What you see

You see a client library sending a command and receiving a response. To you, it’s just a function call: db.get('user:101'). The complexity of memory allocation, network buffers, and concurrency is hidden behind a clean API.

What actually happens

Under the hood, Redis is built on a few core pillars:

1. The Event Loop: Redis runs on a single thread. It uses an I/O multiplexing mechanism (like epoll or kqueue) to monitor thousands of network sockets simultaneously without needing a thread for each one.
2. The Command Dispatcher: Once the event loop detects data on a socket, it reads the request, parses the protocol (RESP), and dispatches it to the appropriate function.
3. Memory Storage: Redis stores everything in RAM using optimized data structures like hash tables, skip lists, and ziplists. This avoids the latency of disk I/O.
4. Persistence Layer: To ensure data isn’t lost, it periodically snapshots the memory to disk or logs every write command to an Append Only File (AOF).

Trace one example

Imagine you run SET name "Alice":

1. Connection: Your client opens a TCP socket to the Redis server.
2. Multiplexing: The server’s event loop notices the socket is readable.
3. Parsing: Redis reads the raw bytes, identifies the SET command, and extracts the key and value.
4. Execution: The server updates its internal hash table in memory.
5. Response: It writes an +OK string back to the socket.
6. Cleanup: The loop returns to waiting for the next event.

Why it feels instant

It feels instant because it avoids the two biggest bottlenecks in computing: disk access and context switching. By keeping data in RAM and using a single-threaded event loop, it eliminates the overhead of locking and thread synchronization. The code is essentially just a tight loop of ‘read, process, write’.

Takeaway

If you want to understand how databases work, stop looking at the SQL and start looking at the event loop. Try writing a basic socket server in C or Python that handles multiple clients at once—that’s the foundation of every modern high-performance system.