ID Allocation — The Linux Kernel 5.18.5-gnu-1 documentation

# ID Allocation

Author:

  Matthew Wilcox

## Overview

A common problem to solve is allocating identifiers (IDs); generally small numbers which identify a thing. Examples include file descriptors, process IDs, packet identifiers in networking protocols, SCSI tags and device instance numbers. The IDR and the IDA provide a reasonable solution to the problem to avoid everybody inventing their own. The IDR provides the ability to map an ID to a pointer, while the IDA provides only ID allocation, and as a result is much more memory-efficient.

## IDR usage

Start by initialising an IDR, either with `DEFINE_IDR()` for statically allocated IDRs or `idr_init()` for dynamically allocated IDRs.

You can call `idr_alloc()` to allocate an unused ID. Look up the pointer you associated with the ID by calling `idr_find()` and free the ID by calling `idr_remove()`.

If you need to change the pointer associated with an ID, you can call `idr_replace()`. One common reason to do this is to reserve an ID by passing a `NULL` pointer to the allocation function; initialise the object with the reserved ID and finally insert the initialised object into the IDR.

Some users need to allocate IDs larger than `INT_MAX`. So far all of these users have been content with a `UINT_MAX` limit, and they use `idr_alloc_u32()`. If you need IDs that will not fit in a u32, we will work with you to address your needs.

If you need to allocate IDs sequentially, you can use `idr_alloc_cyclic()`. The IDR becomes less efficient when dealing with larger IDs, so using this function comes at a slight cost.

To perform an action on all pointers used by the IDR, you can either use the callback-based `idr_for_each()` or the iterator-style `idr_for_each_entry()`. You may need to use `idr_for_each_entry_continue()` to continue an iteration. You can also use `idr_get_next()` if the iterator doesn’t fit your needs.

When you have finished using an IDR, you can call idr_destroy() to release the memory used by the IDR. This will not free the objects pointed to from the IDR; if you want to do that, use one of the iterators to do it.

You can use `idr_is_empty()` to find out whether there are any IDs currently allocated.

If you need to take a lock while allocating a new ID from the IDR, you may need to pass a restrictive set of GFP flags, which can lead to the IDR being unable to allocate memory. To work around this, you can call idr_preload() before taking the lock, and then `idr_preload_end()` after the allocation.

idr synchronization (stolen from radix-tree.h)

`idr_find()` is able to be called locklessly, using RCU. The caller must ensure calls to this function are made within `rcu_read_lock()` regions. Other readers (lock-free or otherwise) and modifications may be running concurrently.

It is still required that the caller manage the synchronization and lifetimes of the items. So if RCU lock-free lookups are used, typically this would mean that the items have their own locks, or are amenable to lock-free access; and that the items are freed by RCU (or only freed after having been deleted from the idr tree and a `synchronize_rcu()` grace period).

## IDA usage

The IDA is an ID allocator which does not provide the ability to associate an ID with a pointer. As such, it only needs to store one bit per ID, and so is more space efficient than an IDR. To use an IDA, define it using DEFINE_IDA() (or embed a `struct ida` in a data structure, then initialise it using ida_init()). To allocate a new ID, call `ida_alloc()`, `ida_alloc_min()`, `ida_alloc_max()` or `ida_alloc_range()`. To free an ID, call `ida_free()`.

`ida_destroy()` can be used to dispose of an IDA without needing to free the individual IDs in it. You can use ida_is_empty() to find out whether the IDA has any IDs currently allocated.

The IDA handles its own locking. It is safe to call any of the IDA functions without synchronisation in your code.

IDs are currently limited to the range [0-INT_MAX]. If this is an awkward limitation, it should be quite straightforward to raise the maximum.

## Functions and structures

IDR_INIT

`IDR_INIT (name)`

> Initialise an IDR.

Parameters

`name`

  Name of IDR.

Description

A freshly-initialised IDR contains no IDs.

DEFINE_IDR

`DEFINE_IDR (name)`

> Define a statically-allocated IDR.

Parameters

`name`

  Name of IDR.

Description

An IDR defined using this macro is ready for use with no additional initialisation required. It contains no IDs.

unsigned int idr_get_cursor(const struct idr *idr)

  Return the current position of the cyclic allocator

Parameters

`const struct idr *idr`

  idr handle

Description

The value returned is the value that will be next returned from `idr_alloc_cyclic()` if it is free (otherwise the search will start from this position).

void idr_set_cursor(struct idr *idr, unsigned int val)

  Set the current position of the cyclic allocator

Parameters

`struct idr *idr`

  idr handle

`unsigned int val`

  new position

Description

The next call to `idr_alloc_cyclic()` will return val if it is free (otherwise the search will start from this position).

void idr_init_base(struct idr *idr, int base)

  Initialise an IDR.

Parameters

`struct idr *idr`

  IDR handle.

`int base`

  The base value for the IDR.

Description

This variation of `idr_init()` creates an IDR which will allocate IDs starting at `base`.

void idr_init(struct idr *idr)

  Initialise an IDR.

Parameters

`struct idr *idr`

  IDR handle.

Description

Initialise a dynamically allocated IDR. To initialise a statically allocated IDR, use `DEFINE_IDR()`.

bool idr_is_empty(const struct idr *idr)

  Are there any IDs allocated?

Parameters

`const struct idr *idr`

  IDR handle.

Return

`true` if any IDs have been allocated from this IDR.

void idr_preload_end(void)

  end preload section started with idr_preload()

Parameters

`void`

  no arguments

Description

Each idr_preload() should be matched with an invocation of this function. See idr_preload() for details.

idr_for_each_entry

`idr_for_each_entry (idr, entry, id)`

> Iterate over an IDR’s elements of a given type.

Parameters

`idr`

  IDR handle.

`entry`

  The type * to use as cursor

`id`

  Entry ID.

Description

entry and id do not need to be initialized before the loop, and after normal termination entry is left with the value NULL. This is convenient for a “not found” value.

idr_for_each_entry_ul

`idr_for_each_entry_ul (idr, entry, tmp, id)`

> Iterate over an IDR’s elements of a given type.

Parameters

`idr`

  IDR handle.

`entry`

  The type * to use as cursor.

`tmp`

  A temporary placeholder for ID.

`id`

  Entry ID.

Description

entry and id do not need to be initialized before the loop, and after normal termination entry is left with the value NULL. This is convenient for a “not found” value.

idr_for_each_entry_continue

`idr_for_each_entry_continue (idr, entry, id)`

> Continue iteration over an IDR’s elements of a given type

Parameters

`idr`

  IDR handle.

`entry`

  The type * to use as a cursor.

`id`

  Entry ID.

Description

Continue to iterate over entries, continuing after the current position.

idr_for_each_entry_continue_ul

`idr_for_each_entry_continue_ul (idr, entry, tmp, id)`

> Continue iteration over an IDR’s elements of a given type

Parameters

`idr`

  IDR handle.

`entry`

  The type * to use as a cursor.

`tmp`

  A temporary placeholder for ID.

`id`

  Entry ID.

Description

Continue to iterate over entries, continuing after the current position.

int ida_alloc(struct ida *ida, gfp_t gfp)

  Allocate an unused ID.

Parameters

`struct ida *ida`

  IDA handle.

`gfp_t gfp`

  Memory allocation flags.

Description

Allocate an ID between 0 and `INT_MAX`, inclusive.

Context

Any context. It is safe to call this function without locking in your code.

Return

The allocated ID, or `-ENOMEM` if memory could not be allocated, or `-ENOSPC` if there are no free IDs.

int ida_alloc_min(struct ida *ida, unsigned int min, gfp_t gfp)

  Allocate an unused ID.

Parameters

`struct ida *ida`

  IDA handle.

`unsigned int min`

  Lowest ID to allocate.

`gfp_t gfp`

  Memory allocation flags.

Description

Allocate an ID between min and `INT_MAX`, inclusive.

Context

Any context. It is safe to call this function without locking in your code.

Return

The allocated ID, or `-ENOMEM` if memory could not be allocated, or `-ENOSPC` if there are no free IDs.

int ida_alloc_max(struct ida *ida, unsigned int max, gfp_t gfp)

  Allocate an unused ID.

Parameters

`struct ida *ida`

  IDA handle.

`unsigned int max`

  Highest ID to allocate.

`gfp_t gfp`

  Memory allocation flags.

Description

Allocate an ID between 0 and max, inclusive.

Context

Any context. It is safe to call this function without locking in your code.

Return

The allocated ID, or `-ENOMEM` if memory could not be allocated, or `-ENOSPC` if there are no free IDs.

int idr_alloc_u32(struct idr *idr, void *ptr, u32 *nextid, unsigned long max, gfp_t gfp)

  Allocate an ID.

Parameters

`struct idr *idr`

  IDR handle.

`void *ptr`

  Pointer to be associated with the new ID.

`u32 *nextid`

  Pointer to an ID.

`unsigned long max`

  The maximum ID to allocate (inclusive).

`gfp_t gfp`

  Memory allocation flags.

Description

Allocates an unused ID in the range specified by nextid and max. Note that max is inclusive whereas the end parameter to `idr_alloc()` is exclusive. The new ID is assigned to nextid before the pointer is inserted into the IDR, so if nextid points into the object pointed to by ptr, a concurrent lookup will not find an uninitialised ID.

The caller should provide their own locking to ensure that two concurrent modifications to the IDR are not possible. Read-only accesses to the IDR may be done under the RCU read lock or may exclude simultaneous writers.

Return

0 if an ID was allocated, -ENOMEM if memory allocation failed, or -ENOSPC if no free IDs could be found. If an error occurred, nextid is unchanged.

int idr_alloc(struct idr *idr, void *ptr, int start, int end, gfp_t gfp)

  Allocate an ID.

Parameters

`struct idr *idr`

  IDR handle.

`void *ptr`

  Pointer to be associated with the new ID.

`int start`

  The minimum ID (inclusive).

`int end`

  The maximum ID (exclusive).

`gfp_t gfp`

  Memory allocation flags.

Description

Allocates an unused ID in the range specified by start and end. If end is <= 0, it is treated as one larger than `INT_MAX`. This allows callers to use start + N as end as long as N is within integer range.

The caller should provide their own locking to ensure that two concurrent modifications to the IDR are not possible. Read-only accesses to the IDR may be done under the RCU read lock or may exclude simultaneous writers.

Return

The newly allocated ID, -ENOMEM if memory allocation failed, or -ENOSPC if no free IDs could be found.

int idr_alloc_cyclic(struct idr *idr, void *ptr, int start, int end, gfp_t gfp)

  Allocate an ID cyclically.

Parameters

`struct idr *idr`

  IDR handle.

`void *ptr`

  Pointer to be associated with the new ID.

`int start`

  The minimum ID (inclusive).

`int end`

  The maximum ID (exclusive).

`gfp_t gfp`

  Memory allocation flags.

Description

Allocates an unused ID in the range specified by nextid and end. If end is <= 0, it is treated as one larger than `INT_MAX`. This allows callers to use start + N as end as long as N is within integer range. The search for an unused ID will start at the last ID allocated and will wrap around to start if no free IDs are found before reaching end.

The caller should provide their own locking to ensure that two concurrent modifications to the IDR are not possible. Read-only accesses to the IDR may be done under the RCU read lock or may exclude simultaneous writers.

Return

The newly allocated ID, -ENOMEM if memory allocation failed, or -ENOSPC if no free IDs could be found.

void *idr_remove(struct idr *idr, unsigned long id)

  Remove an ID from the IDR.

Parameters

`struct idr *idr`

  IDR handle.

`unsigned long id`

  Pointer ID.

Description

Removes this ID from the IDR. If the ID was not previously in the IDR, this function returns `NULL`.

Since this function modifies the IDR, the caller should provide their own locking to ensure that concurrent modification of the same IDR is not possible.

Return

The pointer formerly associated with this ID.

void *idr_find(const struct idr *idr, unsigned long id)

  Return pointer for given ID.

Parameters

`const struct idr *idr`

  IDR handle.

`unsigned long id`

  Pointer ID.

Description

Looks up the pointer associated with this ID. A `NULL` pointer may indicate that id is not allocated or that the `NULL` pointer was associated with this ID.

This function can be called under `rcu_read_lock()`, given that the leaf pointers lifetimes are correctly managed.

Return

The pointer associated with this ID.

int idr_for_each(const struct idr *idr, int (*fn)(int id, void *p, void *data), void *data)

  Iterate through all stored pointers.

Parameters

`const struct idr *idr`

  IDR handle.

`int (*fn)(int id, void *p, void *data)`

  Function to be called for each pointer.

`void *data`

  Data passed to callback function.

Description

The callback function will be called for each entry in idr, passing the ID, the entry and data.

If fn returns anything other than `0`, the iteration stops and that value is returned from this function.

`idr_for_each()` can be called concurrently with `idr_alloc()` and `idr_remove()` if protected by RCU. Newly added entries may not be seen and deleted entries may be seen, but adding and removing entries will not cause other entries to be skipped, nor spurious ones to be seen.

void *idr_get_next_ul(struct idr *idr, unsigned long *nextid)

  Find next populated entry.

Parameters

`struct idr *idr`

  IDR handle.

`unsigned long *nextid`

  Pointer to an ID.

Description

Returns the next populated entry in the tree with an ID greater than or equal to the value pointed to by nextid. On exit, nextid is updated to the ID of the found value. To use in a loop, the value pointed to by nextid must be incremented by the user.

void *idr_get_next(struct idr *idr, int *nextid)

  Find next populated entry.

Parameters

`struct idr *idr`

  IDR handle.

`int *nextid`

  Pointer to an ID.

Description

Returns the next populated entry in the tree with an ID greater than or equal to the value pointed to by nextid. On exit, nextid is updated to the ID of the found value. To use in a loop, the value pointed to by nextid must be incremented by the user.

void *idr_replace(struct idr *idr, void *ptr, unsigned long id)

  replace pointer for given ID.

Parameters

`struct idr *idr`

  IDR handle.

`void *ptr`

  New pointer to associate with the ID.

`unsigned long id`

  ID to change.

Description

Replace the pointer registered with an ID and return the old value. This function can be called under the RCU read lock concurrently with `idr_alloc()` and `idr_remove()` (as long as the ID being removed is not the one being replaced!).

Return

the old value on success. `-ENOENT` indicates that id was not found. `-EINVAL` indicates that ptr was not valid.

int ida_alloc_range(struct ida *ida, unsigned int min, unsigned int max, gfp_t gfp)

  Allocate an unused ID.

Parameters

`struct ida *ida`

  IDA handle.

`unsigned int min`

  Lowest ID to allocate.

`unsigned int max`

  Highest ID to allocate.

`gfp_t gfp`

  Memory allocation flags.

Description

Allocate an ID between min and max, inclusive. The allocated ID will not exceed `INT_MAX`, even if max is larger.

Context

Any context. It is safe to call this function without locking in your code.

Return

The allocated ID, or `-ENOMEM` if memory could not be allocated, or `-ENOSPC` if there are no free IDs.

void ida_free(struct ida *ida, unsigned int id)

  Release an allocated ID.

Parameters

`struct ida *ida`

  IDA handle.

`unsigned int id`

  Previously allocated ID.

Context

Any context. It is safe to call this function without locking in your code.

void ida_destroy(struct ida *ida)

  Free all IDs.

Parameters

`struct ida *ida`

  IDA handle.

Description

Calling this function frees all IDs and releases all resources used by an IDA. When this call returns, the IDA is empty and can be reused or freed. If the IDA is already empty, there is no need to call this function.

Context

Any context. It is safe to call this function without locking in your code.