Session 37: One Pod, 10 Connections; 200 Pods, 2,000 Connections: Redis Connection Pool Arithmetic

1. Imagine If

A coffee shop has one very fast barista. She can only make one order at a time, but each order takes 2 seconds. In front of her counter are 10,000 waiting chairs.

Every office nearby sends couriers. One office sends 10 couriers "just in case", even though a single courier could carry all their orders. When the number of offices grows to 200, there are 2,000 couriers sitting in chairs. Most of them just sit there, ordering nothing.

Then management opens new office branches without closing the old ones (rolling update), every office runs 4 shifts at once (worker processes), and the office next door uses the same coffee shop. The chairs fill up. The next courier is turned away at the door, while the barista is barely working.

---

2. What Actually Happens

Redis executes commands on one main thread. Since Redis 6 there are optional I/O threads for reading and writing sockets, but command execution is still sequential. Redis doesn't need many connections to be fast. It needs short commands.

Every connection still has a cost:

1. Memory per client: query buffer and output buffer. The baseline is tens of KB per connection, and it can balloon when a client reads a large reply (for example SMEMBERS on a set with millions of members).
2. File descriptor: one socket = one fd. maxclients defaults to 10000, but is still capped by the Redis process's ulimit -n.
3. Hard rejection: connection number 10,001 is refused immediately with ERR max number of clients reached and counted in rejected_connections.

The arithmetic people forget:

total connections = pods x workers per pod x client instances per worker x pool per client

Memory counts too: 11,500 connections x ~30 KB = ~345 MB just for client buffers, before a single key is stored.

Client libraries use different models:

• Lettuce (Spring Data Redis default): multiplexed. One shared connection serves many threads through pipelining. A pool is usually unnecessary, except for blocking commands (BLPOP) or transactions (MULTI/EXEC).
• Jedis: one command per connection at a time, so a pool is required (maxTotal, maxIdle, minIdle, maxWait).
• redis-py: ConnectionPool(max_connections=...), and the pool is created per process, so multiply by the number of workers.
• ioredis: one connection per instance with auto-pipelining. Trouble starts when code calls new Redis() on every request.

How many do you actually need? Use Little's Law from Session 3 and Session 6:

concurrent connections = throughput x time per command
500 ops/s x 0.002 s = 1 busy connection on average

Most teams set 50 "just in case". The result is 49 idle connections per pod eating memory and fds on Redis.

Two opposite failures:

• Pool too small: requests wait for a connection. You see Could not get a resource from the pool (Jedis) or borrow timeouts. Application latency rises while Redis CPU stays low, because the time is spent queuing in the pool, not in Redis.
• Pool too large: thousands of idle connections, and when they all drop at once, you get a reconnect storm.

---

3. What If We Try...

"We're getting Redis timeouts. Just raise the pool to 100 per pod!"

Do the math first:

200 pods x 100 = 20,000 connections
maxclients     = 10,000

Half the pods get no connection at all and fail immediately with max number of clients reached. The lucky pods that do connect are not faster either, because Redis still processes one command at a time.

Worse, the root cause usually isn't pool size. When you dig in with SLOWLOG GET, there's a KEYS user:* run by a cron job every minute, taking 800 ms. For those 800 ms, every client in every pod waits. A bigger pool only adds more people to the queue behind that slow command.

Common culprits that hold connections and block Redis:

---

4. The Official Name

• Connection Pool: a set of reused connections so a new socket isn't opened per request.
• Connection Multiplexing / Pipelining: many commands sent over one connection without waiting for each reply (Lettuce, ioredis).
• maxclients: the number of clients Redis accepts, default 10000, capped by ulimit.
• Pool Exhaustion / Borrow Timeout: every pooled connection is in use, and new requests wait until maxWait expires.
• Reconnect Storm: many clients reopening connections at the same moment after a failover, deploy, or scale-out.
• Little's Law: L = λ x W, the basis for pool sizing.

---

5. In Our World

A sensible configuration for a cache with a command target under 5 ms:

# Spring Boot + Lettuce (multiplexed, no pool unless you need blocking commands)
spring:
  data:
    redis:
      host: redis.cache.svc
      timeout: 300ms          # command timeout
      connect-timeout: 1s
      lettuce:
        shutdown-timeout: 2s
        # enable the pool only if you use BLPOP / MULTI
        # pool:
        #   max-active: 8
        #   max-idle: 8
        #   min-idle: 0

// Jedis: a pool is required
JedisPoolConfig cfg = new JedisPoolConfig();
cfg.setMaxTotal(8);          // not 50
cfg.setMaxIdle(8);
cfg.setMinIdle(0);           // avoid mass pre-warming when 200 pods start together
cfg.setMaxWait(Duration.ofMillis(200)); // fail fast, don't hang threads
JedisPool pool = new JedisPool(cfg, "redis.cache.svc", 6379, 300 /* socket timeout ms */);

# redis-py: pool per process, remember to multiply by gunicorn workers
pool = redis.ConnectionPool(
    host="redis.cache.svc", port=6379,
    max_connections=8,
    socket_connect_timeout=1, socket_timeout=0.3,
    health_check_interval=30,
)
r = redis.Redis(connection_pool=pool)

// ioredis: one instance per process, not per request
const redis = new Redis({
  host: "redis.cache.svc",
  connectTimeout: 1000,
  commandTimeout: 300,
  enableAutoPipelining: true,
  retryStrategy: (times) => Math.min(times * 200, 3000) + Math.random() * 500, // jitter
});

What tends to show up on OpenShift:

• Connection storms on deploy and failover: 200 pods with minIdle: 10 starting together open 2,000 connections in a few seconds, plus a TLS handshake per connection. On a Redis failover, every client reconnects at once. Use lazy init, a small minIdle, and backoff with jitter (Session 26).
• Timeouts are mandatory: without a command timeout, a slow Redis hangs application threads, the pool drains, and the failure spreads upstream (Session 25). For a cache, 200–500 ms is already generous.
• Fail open carefully: if Redis goes down and every request is sent straight to the database, that's a thundering herd (Session 19) that moves the problem into Session 16.
• Graceful shutdown must close the pool (Session 36): without pool.close() in a shutdown hook, Redis keeps half-dead clients until timeout or tcp-keepalive cleans them up. During a rolling update, these zombie connections eat into the maxclients budget too.
• Redis Cluster: the client opens a pool per node. 6 shards x 200 pods x 8 = 9,600 connections in total, spread across nodes but still worth counting.
• Proxies for very large client counts: Twemproxy or the Envoy Redis proxy as a sidecar or gateway, similar to PgBouncer's role in Session 17.

---

6. The Performance Tester's Lens

Redis-side metrics:

redis-cli INFO clients
# connected_clients:8123
# blocked_clients:12
# maxclients:10000

redis-cli INFO stats | grep rejected_connections
# rejected_connections:347

redis-cli CLIENT LIST | awk '{print $2}' | cut -d= -f2 | cut -d: -f1 | sort | uniq -c | sort -rn | head
# count connections per pod IP

redis-cli SLOWLOG GET 10
redis-cli LATENCY LATEST

Test scenarios:

1. Scale to HPA max: raise replicas to the HPA limit, read connected_clients, and compare it with the formula in section 2. Do it during a rolling update so maxSurge is counted.
2. Redis failover under load: kill the primary, measure the error window, peak reconnect rate, and whether the backoff has jitter.
3. Latency injection: add 50–200 ms of delay to Redis (Toxiproxy or network chaos). Watch whether the command timeout fires, or whether application threads hang and the pool drains.
4. Slow command test: run KEYS * or a heavy Lua script in staging and watch the p99 of every service sharing that Redis.
5. Shutdown check: after a rolling restart, does the client count return to baseline, or are there zombie clients with high idle in CLIENT LIST?

---

7. Question for the Next Round

Say the pool is sized correctly, timeouts are set, and shutdown closes connections cleanly. 200 pods, 8 connections each, safely under maxclients.

But what if the OpenShift scheduler places 150 of those 200 pods on the same node, or in the same zone? One node dies, 75% of capacity disappears in a second, and 150 replacement pods reconnect to Redis at the same time.

The answer is in Session 38: All Eggs in One Basket, about topology spread constraints and maxSkew.