Redis Clustering Patterns — Cluster Mode, Sharding, and High-Availability Topologies

By Marcus Vance · Lead Systems & Backend Architect · November 2026

Redis Cluster Fundamentals

Architecture — Hash Slots, Not Consistent Hashing

Redis Cluster uses 16,384 fixed hash slots (not consistent hashing ring). Key → slot = CRC16(key) % 16384. Each master owns a contiguous slot range.

Key Hashing & Hash Tags

Default: CRC16(key) % 16384. Hash tags force related keys to same slot:

{user:1000}:profile  → slot 1234
{user:1000}:orders   → slot 1234  (same slot — {user:1000} is hash tag)
{user:1001}:profile  → slot 5678

Only the substring between { and } is hashed. Enables multi-key operations (MGET, transactions, Lua) on related data.

Client Redirection — MOVED / ASK

Smart clients (redis-py, lettuce, StackExchange.Redis, ioredis) maintain local slot map, handle redirection transparently.

Topology & High Availability

Minimum Production Topology

3 masters + 3 replicas (1 per master) = 6 nodes. Tolerates 1 master failure. Quorum = majority of masters (2/3).

# 6 nodes, 3 masters, 1 replica each
redis-cli --cluster create 
  10.0.0.1:7000 10.0.0.2:7001 10.0.0.3:7002 
  10.0.0.4:7000 10.0.0.5:7001 10.0.0.6:7002 
  --cluster-replicas 1

Recommended: 3 Masters, 2 Replicas Each (9 Nodes)

Tolerates 1 master + 1 replica failure simultaneously. Better read scaling (2 replicas per master).

Cross-AZ Deployment

# AZ-aware: rack awareness (Redis 7.0+)
redis-cli --cluster create 
  10.0.1.1:7000 10.0.2.1:7001 10.0.3.1:7002 
  10.0.1.2:7000 10.0.2.2:7001 10.0.3.2:7002 
  --cluster-replicas 1
# Nodes alternate AZ: master in AZ1, replica in AZ2, etc.

Failover Process

1. Master fails (misses heartbeats on cluster bus).
2. Replicas detect via gossip. Replica election — first to get majority of master votes wins.
3. Winner becomes master, claims slots. Sends PONG with new config.
4. Other replicas update config, replicate from new master.
5. Clients receive MOVED, update slot map.

Typical failover: < 5 seconds (configurable via cluster-node-timeout, default 15s).

Manual Failover (Maintenance)

# On replica — graceful failover
redis-cli -p 7001 CLUSTER FAILOVER

# Force (if master unreachable)
redis-cli -p 7001 CLUSTER FAILOVER FORCE

# Takeover (no data loss guarantee)
redis-cli -p 7001 CLUSTER FAILOVER TAKEOVER

Scaling — Resharding & Rebalancing

Online Resharding — Moving Slots

# Move slots 0-1000 from master A to master B
redis-cli --cluster reshard 10.0.0.1:7000 
  --cluster-from  
  --cluster-to  
  --cluster-slots 1000 
  --cluster-yes

Process: Target master imports slots (creates empty keys), source master migrates keys (SCAN + MIGRATE), slot ownership updates via gossip. Zero downtime.

Rebalancing — Even Slot Distribution

# Auto-rebalance (distribute slots evenly across masters)
redis-cli --cluster rebalance 10.0.0.1:7000 
  --cluster-use-empty-masters 
  --cluster-simulate  # Dry run first

Adding / Removing Nodes

# Add new master (empty slots)
redis-cli --cluster add-node 10.0.0.7:7000 10.0.0.1:7000

# Add replica to existing master
redis-cli --cluster add-node 10.0.0.8:7000 10.0.0.1:7000 
  --cluster-slave --cluster-master-id 

# Remove node (must be empty — reshard first)
redis-cli --cluster del-node 10.0.0.1:7000 

Redis 8.0 Features (2026)

Vector Sets — Native Vector Search

# Add vectors
VSADD docs 1 0.1 0.2 0.3 0.4  # 4-dim vector
VSADD docs 2 0.5 0.6 0.7 0.8

# Similarity search (cosine)
VSIM docs 3 1.0 0.0 0.0 0.0  # Returns (id, score)
# With filter
VSIM docs 3 1.0 0.0 0.0 0.0 FILTER category=tech

# Random sampling
VSRANDMEMBER docs 5

• HNSW index (configurable M, ef_construction).
• Distance: COSINE, L2, IP (dot product).
• Sub-10M vectors: competes with pgvector, Milvus. No separate service.

Redis Functions — Server-Side TypeScript

# Register function library
FUNCTION LOAD "#!
redis.register_function('myLib', 'getAndIncr', function(keys, args) {
  const val = redis.call('GET', keys[1]);
  if (!val) return nil;
  return redis.call('INCR', keys[1]);
})"

# Call
FCALL getAndIncr 1 mykey

• Replaces EVAL/EVALSHA. Versioned, namespaced, testable.
• Atomic execution. No script replication issues.
• Engine: QuickJS (fast, small).

Cluster Auto-Resharding

# Automatic slot rebalancing
CLUSTER RESHARD AUTO

# Policy config
cluster-reshard-threshold 0.1  # Rebalance if skew > 10%
cluster-reshard-interval 3600  # Check every hour

Client Patterns & Best Practices

Connection Pooling

# Python (redis-py)
pool = ConnectionPool.from_url(
  "redis://cluster:7000",
  max_connections=50,
  retry_on_timeout=True,
  health_check_interval=30
)
client = Redis(connection_pool=pool)

# Java (Lettuce)
RedisClusterClient client = RedisClusterClient.create(
  RedisURI.Builder.redis("cluster", 7000).build()
);
StatefulRedisClusterConnection conn = client.connect();
RedisAdvancedClusterCommands cmd = conn.sync();

Read Replicas — READONLY / READWRITE

# On replica connection
READONLY  # Allow reads (may be stale)
# Or
READWRITE # Disallow reads (default for replicas)

# Client-side: read_from_replicas=True (redis-py)
# Lettuce: ReadFrom.REPLICA

Pipelining & Batching

# Pipeline (non-transactional batch)
pipe = client.pipeline()
for i in range(1000):
  pipe.set(f"key:{i}", f"value:{i}")
pipe.execute()  # Single RTT

# Transaction (atomic) — ONLY if all keys in SAME SLOT
pipe = client.pipeline(transaction=True)
pipe.multi()
pipe.set("{user:1}:a", "1")
pipe.set("{user:1}:b", "2")
pipe.exec()

Lua Scripts — Atomic, Same Slot

# All keys MUST hash to same slot (use hash tags)
local function atomic_transfer(from, to, amount)
  local from_val = tonumber(redis.call('GET', from))
  if not from_val or from_val < amount then
    return {err = "insufficient funds"}
  end
  redis.call('DECRBY', from, amount)
  redis.call('INCRBY', to, amount)
  return {ok = true}
end

# Register
SCRIPT LOAD "..."  # Returns SHA
# Execute
EVALSHA  2 {user:1}:balance {user:1}:reserved 100

Persistence — AOF + RDB

Cluster Persistence Config

# /etc/redis/redis.conf (per node)
save 900 1      # RDB: 1 change in 15 min
save 300 10     # RDB: 10 changes in 5 min
save 60 10000   # RDB: 10000 changes in 1 min

appendonly yes
appendfsync everysec  # Default: fsync every second
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

Monitoring & Operations

Key Metrics

Critical Commands

# Cluster health
CLUSTER INFO
CLUSTER NODES
CLUSTER SLOTS

# Node info
INFO memory
INFO replication
INFO stats
INFO keyspace

# Slow log
SLOWLOG GET 10
SLOWLOG RESET

# Memory analysis
MEMORY STATS
MEMORY DOCTOR
LATENCY DOCTOR
LATENCY HISTOGRAM command

Backup Strategy

1. RDB copy: BGSAVE → copy dump.rdb to S3 (per node, staggered).
2. AOF copy: Copy appendonly.aof (continuous).
3. Cluster backup: redis-cli --cluster backup /backup/dir (coordinates across nodes).
4. Point-in-time: Use replica — BGSAVE on replica, copy RDB.

Common Architectural Patterns

Pattern 1: Session Store (High Read, Low Write)

• 3 masters, 2 replicas each.
• Read from replicas (READONLY).
• TTL-based expiry. maxmemory-policy allkeys-lru.
• Hash tag: {session:user123}:data.

Pattern 2: Rate Limiting (High Write, Atomic)

• Lua script for sliding window (atomic).
• Hash tag per client: {ratelimit:client456}:window.
• 3 masters, 1 replica. maxmemory-policy noeviction.

Pattern 3: Message Queue (Streams)

• Redis Streams (5.0+): XADD, XREAD, XGROUP.
• Consumer groups = exactly-once delivery (with XACK).
• Shard by stream key (hash tag).
• Trim: XTRIM MAXLEN ~ 10000.

Pattern 4: Vector Search / RAG (Redis 8.0)

• Vector Sets: VSADD, VSIM.
• Embeddings stored with metadata (JSON).
• Single Redis Cluster replaces separate vector DB.

Pattern 5: Distributed Lock (Redlock)

# Redlock algorithm (antirez)
# 1. Get time T1
# 2. Try acquire lock on N/2+1 masters (SET key value NX PX ttl)
# 3. If majority acquired, T2 - T1 < ttl → success
# 4. Else unlock all, retry
# 5. Lock validity = ttl - (T2 - T1) - clock_drift

# Use Redisson (Java), redis-py-redlock, or similar library

TL;DR — Key Takeaways

• Redis Cluster = 16,384 hash slots distributed across masters. Hash tags for multi-key ops.
• Min production: 3 masters + 1 replica each (6 nodes). Recommended: 3 masters + 2 replicas (9 nodes).
• Failover: Automatic (gossip-based), < 5s typical. Manual: CLUSTER FAILOVER.
• Scaling: Online resharding (--cluster reshard), rebalancing (--cluster rebalance). Zero downtime.
• Redis 8.0: Vector Sets (native HNSW), Functions (TypeScript), Auto-resharding.
• Clients: Smart client (slot cache), pipeline for batch, Lua for atomic same-slot ops.
• Persistence: RDB + AOF hybrid. Backup via coordinated cluster backup or replica RDB.
• Monitor: cluster_state, slots_assigned, replication_lag, used_memory.

Next in series: Kafka Partitioned Log — architecture, replication, and exactly-once semantics.