dm-pcache — Persistent Cache — The Linux Kernel documentation

# dm-pcache — Persistent Cache

Author: Dongsheng Yang <dongsheng.yang@linux.dev>

This document describes dm-pcache, a Device-Mapper target that lets a byte-addressable DAX (persistent-memory, “pmem”) region act as a high-performance, crash-persistent cache in front of a slower block device. The code lives in drivers/md/dm-pcache/.

## Quick feature summary

- Write-back caching (only mode currently supported).

- 16 MiB segments allocated on the pmem device.

- Data CRC32 verification (optional, per cache).

- Crash-safe: every metadata structure is duplicated (PCACHE_META_INDEX_MAX == 2) and protected with CRC+sequence numbers.

- Multi-tree indexing (indexing trees sharded by logical address) for high PMem parallelism

- Pure DAX path I/O – no extra BIO round-trips

- Log-structured write-back that preserves backend crash-consistency

## Constructor

    pcache <cache_dev> <backing_dev> [<number_of_optional_arguments> <cache_mode writeback> <data_crc true|false>]

`cache_dev` | Any DAX-capable block device (`/dev/pmem0`…). All metadata and cached blocks are stored here.
`backing_dev` | The slow block device to be cached.
`cache_mode` | Optional, Only `writeback` is accepted at the moment.
`data_crc` | Optional, default to `false` `true` – store CRC32 for every cached entry and verify on reads `false` – skip CRC (faster)

### Example

    dmsetup create pcache_sdb --table \
      "0 $(blockdev --getsz /dev/sdb) pcache /dev/pmem0 /dev/sdb 4 cache_mode writeback data_crc true"

The first time a pmem device is used, dm-pcache formats it automatically (super-block, cache_info, etc.).

## Status line

`dmsetup status <device>` (`STATUSTYPE_INFO`) prints:

    <sb_flags> <seg_total> <cache_segs> <segs_used> \
    <gc_percent> <cache_flags> \
    <key_head_seg>:<key_head_off> \
    <dirty_tail_seg>:<dirty_tail_off> \
    <key_tail_seg>:<key_tail_off>

### Field meanings

`sb_flags` | Super-block flags (e.g. endian marker).
`seg_total` | Number of physical pmem segments.
`cache_segs` | Number of segments used for cache.
`segs_used` | Segments currently allocated (bitmap weight).
`gc_percent` | Current GC high-water mark (0-90).
`cache_flags` | Bit 0 – DATA_CRC enabled Bit 1 – INIT_DONE (cache initialised) Bits 2-5 – cache mode (0 == WB).
`key_head` | Where new key-sets are being written.
`dirty_tail` | First dirty key-set that still needs write-back to the backing device.
`key_tail` | First key-set that may be reclaimed by GC.

## Messages

Change GC trigger

    dmsetup message <dev> 0 gc_percent <0-90>

## Theory of operation

### Sub-devices

backing_dev | Any block device (SSD/HDD/loop/LVM, etc.).
cache_dev | DAX device; must expose direct-access memory.

### Segments and key-sets

- The pmem space is divided into 16 MiB segments.

- Each write allocates space from a per-CPU data_head inside a segment.

- A cache-key records a logical range on the origin and where it lives inside pmem (segment + offset + generation).

- 128 keys form a key-set (kset); ksets are written sequentially in pmem and are themselves crash-safe (CRC).

- The pair (key_tail, dirty_tail) delimit clean/dirty and live/dead ksets.

### Write-back

Dirty keys are queued into a tree; a background worker copies data back to the backing_dev and advances dirty_tail. A FLUSH/FUA bio from the upper layers forces an immediate metadata commit.

### Garbage collection

GC starts when `segs_used >= seg_total * gc_percent / 100`. It walks from key_tail, frees segments whose every key has been invalidated, and advances key_tail.

### CRC verification

If `data_crc is enabled` dm-pcache computes a CRC32 over every cached data range when it is inserted and stores it in the on-media key. Reads validate the CRC before copying to the caller.

## Failure handling

- pmem media errors – all metadata copies are read with `copy_mc_to_kernel`; an uncorrectable error logs and aborts initialisation.

- Cache full – if no free segment can be found, writes return `-EBUSY`; dm-pcache retries internally (request deferral).

- System crash – on attach, the driver replays ksets from key_tail to rebuild the in-core trees; every segment’s generation guards against use-after-free keys.

## Limitations & TODO

- Only write-back mode; other modes planned.

- Only FIFO cache invalidate; other (LRU, ARC...) planned.

- Table reload is not supported currently.

- Discard planned.

## Example workflow

    # 1.  Create devices
    dmsetup create pcache_sdb --table \
      "0 $(blockdev --getsz /dev/sdb) pcache /dev/pmem0 /dev/sdb 4 cache_mode writeback data_crc true"

    # 2.  Put a filesystem on top
    mkfs.ext4 /dev/mapper/pcache_sdb
    mount /dev/mapper/pcache_sdb /mnt

    # 3.  Tune GC threshold to 80 %
    dmsetup message pcache_sdb 0 gc_percent 80

    # 4.  Observe status
    watch -n1 'dmsetup status pcache_sdb'

    # 5.  Shutdown
    umount /mnt
    dmsetup remove pcache_sdb

`dm-pcache` is under active development; feedback, bug reports and patches are very welcome!