NuttX implemented fork() and vfork() as the same function. Both were libc
wrappers around a single up_fork() syscall; vfork() differed only by a
trailing waitpid(). Underneath, the child joined the parent's address
environment -- the same addrenv_join() that pthread_create() uses -- and got
a private copy of the stack. So the child shared .data, .bss and the heap
with its parent and ran concurrently with it.
That is not fork(). It is vfork()-with-a-private-stack under fork()'s name,
and the history says so: today's fork() is NuttX's old vfork(), renamed in
c33d1c9c97 (2023) without any change of behaviour. The failure was silent --
a program written against POSIX fork() compiled, ran, and had its child's
writes land in the parent's variables.
Separate them into two primitives, chosen by which function the caller
called rather than by what the hardware happens to be:
fork() child gets its own copy of the parent's memory at the same
virtual addresses; runs concurrently. Only where an address
environment can be duplicated -- elsewhere it is not declared at
all, so calling it is a build error naming the function.
vfork() child shares the parent's memory; parent suspended until the
child _exit()s or exec()s. Implementable everywhere.
Below libc there is still one syscall. up_fork() gains a bool saying which
primitive the caller used, since the per-architecture register snapshot is
the same for both, and passes it to nxtask_setup_fork(), which is the single
place the memory semantics are decided. The argument arrives in the first
argument register and is never touched: each architecture's snapshot takes
some other call-clobbered register for its scratch, so the flag is simply
still there when the C worker is called.
The vfork() parent suspension moves out of libc into nxtask_start_fork(),
released from nxsched_release_tcb() by nxtask_resume_vfork(). Two things
follow: the parent is resumed at exec(), since exec_swap() has already handed
the child's pid to the loaded program by the time the vfork stub exits, and
vfork() no longer depends on CONFIG_SCHED_WAITPID.
Releasing there requires one fix in nxtask_exit(). It raises rtcb->lockcount
directly rather than through sched_lock() while it tears the TCB down, so the
nxsem_post() that wakes the vfork() parent leaves it queued where a blocked
task collects while pre-emption is off -- g_pendingtasks, or g_readytorun on
SMP -- and the matching raw lockcount-- does not publish it the way
sched_unlock() would, leaving the parent stranded with nothing to move it on.
The fix mirrors sched_unlock() for each case: nxsched_merge_pending(), or
nxsched_deliver_task() under CONFIG_SMP. Both are no-ops while pre-emption is
still disabled, and up_exit() re-reads this_task() afterwards, so a change of
the ready-to-run head is honoured. Without it vfork() deadlocks wherever no
other task happens to call sched_unlock() afterwards -- rv-virt:nsh64 and
rv-virt:pnsh64, where NSH is blocked in waitpid() holding the lock, and
qemu-armv8a:citest_smp, which hangs the moment the vfork() test runs.
fork() is built on a new addrenv_fork(), backed by an up_addrenv_fork() hook
that duplicates an address environment into freshly allocated pages mapped at
the same virtual addresses -- unlike up_addrenv_clone(), which copies only
the representation and leaves both pointing at the same page tables. The
child then adopts the parent's stack geometry rather than being given a
relocated copy: a pointer to a stack local taken before fork() must name the
same object in the child that it named in the parent, and the parent's stack
is already in the duplicate, with its contents, at the parent's address.
No architecture implements up_addrenv_fork() yet, so this commit leaves
fork() unavailable everywhere. That is the intended state. It withdraws
fork() from ARCH_ARM, flat ARCH_ARM64, ARCH_RISCV, ARCH_SIM and ARCH_X86_64,
where until now it named the sharing primitive; per-architecture patches
restore it, with POSIX semantics, as up_addrenv_fork() lands. In the
meantime the sharing primitive is still there under the name that describes
it: vfork() for a child that runs a program, pthread_create() for a second
flow of control that shares memory, posix_spawn() for both at once.
Kconfig: ARCH_HAVE_VFORK inherits ARCH_HAVE_FORK's select lines, conditions
included, so no configuration gains machinery; ARCH_HAVE_FORK is redefined to
mean "can provide POSIX fork() semantics" and now depends on ARCH_ADDRENV.
There is one deliberate departure from "verbatim". ARCH_ARM selected the
fork family unconditionally, BUILD_KERNEL included, and that has never
worked: on a kernel build the architecture's fork entry point sees the
kernel's return address and stack pointer rather than the caller's, so the
child resumes at a kernel address. On qemu-armv7a:knsh master faults in
ostest's fork case with "Child did not run" and then a data abort; without
the condition this change faults the same way through vfork(). ARCH_ARM64
and ARCH_X86_64 already carried "if !BUILD_KERNEL" for exactly this reason --
ARM was the outlier. Conditioning it turns a runtime fault into an honest
absence, which is the whole point of the change; arch/arm takes the condition
off again in the patch that adds its saved-syscall-frame path. Only the
MMU-capable ARM ports are affected, since Cortex-M cannot build BUILD_KERNEL
at all.
Also fixes two latent syntax errors found on the way: a missing comma in
riscv_fork.c and mips_fork.c, both in *_FRAMEPOINTER && !SAVE_GP branches
that are never compiled today.
BREAKING CHANGE: fork() is withdrawn from every architecture. It is no
longer declared in unistd.h, so code that calls it fails to build with an error
naming the function, and the sharing behaviour it used to have is gone rather
than renamed. CONFIG_ARCH_HAVE_FORK no longer means "fork() exists"; it means
"this configuration can provide POSIX fork() semantics", and no architecture
selects it yet.
Quick fix, chosen by why the call was made:
to run a program vfork() + exec*(), or better posix_spawn()
a second flow of control that pthread_create()
shares the caller's memory
a genuinely independent copy keep fork(), and wait for the per-arch patch
of the process that implements up_addrenv_fork() and selects
CONFIG_ARCH_HAVE_FORK
Out-of-tree code that tests CONFIG_ARCH_HAVE_FORK to decide whether a
fork-then-exec path is available wants CONFIG_ARCH_HAVE_VFORK instead, which is
selected in exactly the places CONFIG_ARCH_HAVE_FORK used to be. The full
migration guide is Documentation/guides/fork_vfork_migration.rst.
Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
70c2ef5911