nuttx/libs
Marco Casaroli b7faaed58f libc, sched: Resolve FDPIC descriptors at module callback entry points.
A module's function pointer is the address of a two-word descriptor in its
writable segment, not a code address.  Firmware that accepts one, stores it,
and later branches to it therefore jumps into the module's RAM data.  There
is no diagnostic: the board takes a HardFault with a dead console.

qsort and bsearch have resolved the descriptor since FDPIC support landed.
Nothing else did, and nothing noticed, because qsort's comparison function
was the only callback anything exercised.  That left every other entry point
looking perfectly usable -- a module could call one, it would return success,
and the fault arrived later from somewhere else.  Resolution is opt-in per
call site via fdpic_callback(), and this covers the rest of them:
pthread_create, signal, task_create, pthread_once, task_spawn, scandir and
sigaction.

Resolution happens once, in the innermost routine the paths share, for the
reason qsort already documents: resolving twice would treat a code address as
a descriptor.  task_create_with_stack resolves and task_create forwards to
it; nxsig_action() resolves for both signal() and a direct sigaction(), on
the local copy it already makes, so the caller's const struct is untouched
and the SA_SIGINFO form is covered through the union.

Two entry points need care beyond that pattern:

signal() has to exclude the dispositions by hand.  SIG_IGN, SIG_DFL, SIG_HOLD
and SIG_ERR are the integers 0, 1, 2 and -1 rather than addresses, and
fdpic_callback() declines to dereference NULL and nothing else -- handing it
SIG_ERR would read through (void *)-1.

scandir takes two pointers.  Its filter is called by scandir itself and is
resolved here; its comparison function is handed to qsort(), whose own entry
point resolves it, so resolving it here as well would resolve twice.  The
test passes both at once, which is what exercises that distinction.

Resolving the code address is sufficient at all of these.  A thread or task
created from a module inherits its D-Space -- nxtask_dup_dspace() runs before
up_initial_state() installs it in the FDPIC register -- and a signal handler
or pthread_once init routine runs in a context that already holds the
module's data base.

Verified on an RP2350 with a module that hands its own function to each entry
point.  Each assertion was also checked against a deliberate breakage:
removing the scandir filter resolution, and adding a second resolution of the
comparison function, each take the board down with a HardFault rather than
printing a FAIL line -- which is how a broken callback resolution manifests on
Cortex-M, and confirms the assertions test what they name.

mq_notify and timer_create with SIGEV_THREAD remain, and are harder: their
callback runs on a work-queue worker that carries no module data base at all.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
2026-08-01 12:23:27 +02:00
..
libbuiltin libs/libbuiltin/compiler-rt: skip unsupported Arm VFP builtins asm 2026-07-31 15:37:33 -03:00
libc libc, sched: Resolve FDPIC descriptors at module callback entry points. 2026-08-01 12:23:27 +02:00
libdsp tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
libm tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
libnx tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
libxx tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
CMakeLists.txt libs: migrate to SPDX identifier 2024-10-01 12:25:52 +08:00