Linux RCU Synchronization — Read-Copy-Update Internals, APIs, and Usage Patterns

By Marcus Vance · Lead Systems & Backend Architect · November 2026

RCU Concept — Read-Copy-Update

The Core Idea

Readers never block, writers never block readers. RCU achieves this by:

1. Read side — No locks, no atomic ops, no cache-line bouncing. Just rcu_read_lock() / rcu_read_unlock() (often no-ops).
2. Write side — Copy data, modify copy, **publish** new version (pointer swap). Old version retained.
3. Reclamation — Wait for **grace period** (all pre-existing readers done) → free old version.

Why It Works

Readers execute in **RCU read-side critical sections**. A grace period waits for **all CPUs to pass through a quiescent state** (context switch, idle, user mode). Any reader that started before grace period must have ended — its critical section cannot span a quiescent state.

Performance Profile

RCU Flavors — Four Variants

Rule: Match flavor to the **read-side context**. Classic for process context. BH for softirq. Tasks for task_struct.

Grace Periods — The Synchronization Engine

What Is a Grace Period?

Time interval during which **every CPU passes through a quiescent state**. After grace period, all pre-existing readers have exited critical sections.

Quiescent States

• Classic: Context switch, idle loop, user-mode execution, cond_resched().
• BH: local_bh_disable() / local_bh_enable() pair.
• Tasks: Voluntary context switch (schedule()).

Grace Period Implementation — Tree Hierarchy

RCU tracks per-CPU state in a **tree of rcu_node structures** (one per 16-64 CPUs). Root → leaves. Grace period state machine:

1. Start: Root rcu_node enters GP_START. Records start time.
2. Propagate: State pushed down tree. Each leaf (CPU) notes new grace period.
3. Quiescent: CPU reports quiescent state → leaf clears bit.
4. Aggregate: When all leaves clear → parent clears. Up to root.
5. End: Root reaches GP_END. Callbacks invoked.

Expedited Grace Periods

synchronize_rcu_expedited() — Uses **IPI (Inter-Processor Interrupt)** to force quiescent state on all CPUs. ~10-50μs vs ~1-10ms normal. **Expensive** — use sparingly (boot, module unload).

Core RCU APIs

Read-Side

// Classic
void rcu_read_lock(void);           // Preempt disable (CONFIG_PREEMPT_RCU) or nothing
void rcu_read_unlock(void);

// BH
void rcu_read_lock_bh(void);        // local_bh_disable()
void rcu_read_unlock_bh(void);      // local_bh_enable()

// Tasks
void rcu_read_lock_tasks(void);
void rcu_read_unlock_tasks(void);

// Pointer access (compiler barrier + dependency ordering)
#define rcu_dereference(p)         // READ_ONCE + barrier
#define rcu_dereference_protected(p, c)  // With lockdep check
#define rcu_dereference_raw(p)     // No barrier (careful!)

Write-Side — Update & Publish

// Pointer replacement (atomic)
#define rcu_assign_pointer(p, v)   // WRITE_ONCE + smp_wmb()

// List updates
void list_add_rcu(struct list_head *new, struct list_head *head);
void list_del_rcu(struct list_head *entry);
void list_replace_rcu(struct list_head *old, struct list_head *new);

// Hash table
void hlist_add_head_rcu(struct hlist_node *n, struct hlist_head *h);
void hlist_del_rcu(struct hlist_node *n);

// Tree (RB-tree, interval tree) — manual pointer swap + synchronize_rcu()

Reclamation — Waiting for Grace Period

// Blocking (synchronous)
void synchronize_rcu(void);              // Wait for classic GP
void synchronize_rcu_bh(void);
void synchronize_rcu_tasks(void);
void synchronize_rcu_tasks_trace(void);

// Expedited
void synchronize_rcu_expedited(void);

// Asynchronous (callback-based) — PREFERRED for performance
void call_rcu(struct rcu_head *head, rcu_callback_t func);
void call_rcu_bh(struct rcu_head *head, rcu_callback_t func);
void call_rcu_tasks(struct rcu_head *head, rcu_callback_t func);

// Example
struct my_struct {
  int data;
  struct rcu_head rcu;
};

void my_free_callback(struct rcu_head *head) {
  struct my_struct *p = container_of(head, struct my_struct, rcu);
  kfree(p);
}

void update_struct(struct my_struct **ptr, int new_data) {
  struct my_struct *new = kmalloc(sizeof(*new), GFP_KERNEL);
  new->data = new_data;
  rcu_assign_pointer(*ptr, new);           // Publish
  call_rcu(&(*ptr)->rcu, my_free_callback); // Reclaim old after GP
}

kfree_rcu — Simplified Reclamation

// Since 4.13: no explicit callback needed
kfree_rcu(ptr, rcu_head_field);

// Or for struct with rcu_head as first member
kfree_rcu(ptr, rcu);

RCU-Protected Data Structures

Linked Lists — list_head with RCU

// Traversal (read-side)
rcu_read_lock();
list_for_each_entry_rcu(pos, head, member) {
  // Safe: pos won't be freed until grace period
  do_something(pos);
}
rcu_read_unlock();

// Deletion (write-side)
list_del_rcu(&entry->list);        // Remove from list
call_rcu(&entry->rcu, free_cb);    // Free after GP

Hash Tables — hlist_head

// Lookup
rcu_read_lock();
hlist_for_each_entry_rcu(pos, head, member) {
  if (key_match(pos->key)) { found = pos; break; }
}
rcu_read_unlock();

// Insert
hlist_add_head_rcu(&new->node, &head[hash]);

// Delete
hlist_del_rcu(&entry->node);
call_rcu(&entry->rcu, free_cb);

Radix Tree / XArray — xa_store / xa_load

// XArray (since 4.20) — RCU-safe by default
xa_store(&xa, index, entry, GFP_KERNEL);  // Publish
entry = xa_load(&xa);                      // Read (rcu_read_lock needed)
xa_erase(&xa, index);                      // Delete
call_rcu(&entry->rcu, free_cb);

Lockless Ring Buffer — ring_buffer (tracing)

Per-CPU buffers, RCU-protected metadata. Writers reserve space, commit. Readers consume via RCU iteration.

Advanced Patterns & Gotchas

RCU + Reference Counting (kref)

Combine RCU for lookup + kref for lifetime:

struct obj {
  struct kref ref;
  struct rcu_head rcu;
  // ...
};

struct obj *obj_get(struct obj *p) {
  if (p && kref_get_unless_zero(&p->ref)) return p;
  return NULL;
}

void obj_put(struct obj *p) {
  if (kref_put(&p->ref, obj_free)) {
    // Last ref dropped — but may have RCU readers!
    // Use kfree_rcu in obj_free
  }
}

void obj_free(struct kref *ref) {
  struct obj *p = container_of(ref, struct obj, ref);
  kfree_rcu(p, rcu);
}

// Lookup
rcu_read_lock();
p = obj_get(rcu_dereference(global_ptr));
rcu_read_unlock();
if (p) { use(p); obj_put(p); }

RCU + Locks — Common Pattern

Read-side: RCU (fast path). Write-side: mutex/spinlock + RCU update.

mutex_lock(&lock);
// ... modify ...
list_replace_rcu(&old->list, &new->list);
mutex_unlock(&lock);
call_rcu(&old->rcu, free_cb);

Nested RCU Read Sections

Allowed. rcu_read_lock() increments per-task counter. Grace period waits for counter to reach zero.

RCU in Interrupt Context

• Classic: OK in hardirq/softirq (preempt count tracks).
• BH: Use in softirq — rcu_read_lock_bh() disables softirq.
• Never sleep in RCU read section (classic: preempt disabled).

Common Bugs

Sleepable RCU (SRCU) — For Blocking Read Sections

When read-side must **sleep** (mutex, copy_to_user, filesystem):

struct srcu_struct my_srcu;
init_srcu_struct(&my_srcu);

// Read side
int idx = srcu_read_lock(&my_srcu);
// ... can sleep, call mutex, copy_to_user ...
srcu_read_unlock(&my_srcu, idx);

// Write side
synchronize_srcu(&my_srcu);  // Or call_srcu()

cleanup_srcu_struct(&my_srcu);

SRCU uses **per-CPU counters + grace period per srcu_struct**. Slower than classic RCU, but allows blocking.

Performance & Debugging

Key Metrics (/sys/kernel/debug/rcu/)

# Grace period stats
cat /sys/kernel/debug/rcu/rcu_preempt/gpnum
cat /sys/kernel/debug/rcu/rcu_preempt/rcu_gp_kthread_cpu

# Callback stats
cat /sys/kernel/debug/rcu/rcu_preempt/rcu_callbacks

# Stall detection (RCU stall warning)
dmesg | grep "rcu.*stall"

Lockdep + RCU

Enable CONFIG_PROVE_RCU + CONFIG_PROVE_RCU_REPEATEDLY. Catches: missing read lock, sleeping in RCU, wrong flavor.

RCU Torture Test

# Boot parameter
rcutorture.rcutorture=1 rcutorture.nreaders=4 rcutorture.nwriters=2

# Or module
modprobe rcutorture

Userspace RCU — liburcu

For user-space lockless data structures:

// Build: -lurcu
#include 
#include 

// Same APIs: rcu_read_lock(), rcu_dereference(), call_rcu(), etc.
// Grace period: user-space threads call rcu_quiescent_state() periodically
// or use signal-based (SIGUSR1) for automatic quiescent state

Used in: DPDK, Open vSwitch, Ceph, QEMU, custom high-performance services.

TL;DR — Key Takeaways

• RCU = **Lockless reads, deferred reclamation via grace periods**. Readers ~0 cost.
• Four flavors: Classic (general), BH (softirq), Tasks (task_struct), Tasks Trace (tracing).
• APIs: rcu_read_lock/unlock, rcu_dereference, rcu_assign_pointer, list_*_rcu, call_rcu/kfree_rcu.
• Grace period = all CPUs quiescent. Expedited = IPI (use sparingly).
• SRCU for blocking read sections. Userspace: liburcu.
• Debug: CONFIG_PROVE_RCU, /sys/kernel/debug/rcu/, stall warnings.

Next in series: Redis Clustering Patterns — Cluster mode, sharding, and high-availability topologies.