nuttx/arch
Marco Casaroli 067e30e14f arch/x86_64: Build fork() children from the caller's syscall frame.
In a kernel build vfork() is reached through a system call, so the return
address and stack pointer the architecture's entry point can see for itself
are the kernel's, not the caller's.  A child built from those resumes at a
kernel address, which is why x86_64 selected the fork family only for the
flat build.

x86_64_syscall() now publishes the caller's frame in xcp.sregs for the
duration of the stub call, and x86_64_fork() builds the child from it:

  x86_64_fork_syscall()  when xcp.sregs is non-NULL, so that the child
                         returns from the very same `syscall' instruction as
                         the parent, in user mode, on its own stack;
  x86_64_fork_direct()   otherwise, which is the flat build and any kernel
                         thread that calls the entry point as a plain
                         function.

The discriminator is xcp.sregs rather than TCB_FLAG_SYSCALL, which arm64 and
RISC-V use:  that flag also defers signal actions, x86_64 has never raised it,
and its kernel-build signal path does not survive being made to -- a
pre-existing problem that does not belong to this work.

Two properties of SYSCALL/SYSRET shape the child's frame.  The instruction
leaves the caller's RIP and RFLAGS in RCX and R11 rather than on a stack, so
they are moved into the RIP and RFLAGS slots of the interrupt frame the child
is resumed from; and the hardware never records the caller's CS and SS at all,
SYSRETQ reconstructing them from IA32_STAR, so the child's are filled in with
the user code and data selectors at RPL 3.  The frame is therefore not copied
wholesale:  the extended state and the general registers are inherited, while
the segment registers and the thread pointer stay as up_initial_state() left
them, the child's stack being a fresh allocation the parent's FS base does not
describe.

x86_64_fork_relocfp() is new and is not optional here.  A function returns
with `leave', which feeds the frame pointer into the stack pointer, so
relocating only the RBP the child resumes with gets it exactly one frame:
the next return loads a saved RBP still pointing into the parent's stack.

With that in place ARCH_X86_64 can select ARCH_HAVE_VFORK unconditionally.

Build-verified on qemu-intel64:knsh_romfs and qemu-intel64:ostest.  NuttX on
qemu-intel64 requires tsc-deadline and pcid, which TCG does not implement, so
it cannot be run on this host.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
2026-08-10 08:57:30 -03:00
..
arm arch/arm: Build fork() children from the caller's syscall frame on armv7-a. 2026-08-10 08:57:30 -03:00
arm64 arch/arm64: Build fork() children from the caller's syscall frame. 2026-08-10 08:57:30 -03:00
avr tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
ceva !sched/arch/libc: Give fork() and vfork() their real, separate semantics. 2026-08-10 08:57:30 -03:00
dummy
hc tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
mips !sched/arch/libc: Give fork() and vfork() their real, separate semantics. 2026-08-10 08:57:30 -03:00
misoc tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
or1k tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
renesas tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
risc-v !sched/arch/libc: Give fork() and vfork() their real, separate semantics. 2026-08-10 08:57:30 -03:00
sim !sched/arch/libc: Give fork() and vfork() their real, separate semantics. 2026-08-10 08:57:30 -03:00
sparc tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
tricore cmake: Use NUTTX(_DIR/_BIN_DIR) instead CMAKE(_SRC_DIR/_BIN_DIR) 2026-08-09 11:13:08 -03:00
x86 tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
x86_64 arch/x86_64: Build fork() children from the caller's syscall frame. 2026-08-10 08:57:30 -03:00
xtensa cmake: Use NUTTX(_DIR/_BIN_DIR) instead CMAKE(_SRC_DIR/_BIN_DIR) 2026-08-09 11:13:08 -03:00
z16 tools: fix stale archive members surviving a Kconfig-driven CSRCS change 2026-07-28 21:26:03 -03:00
z80 style: fix checkpatch issues after debug.h move 2026-04-07 07:50:06 -03:00
CMakeLists.txt cmake: Use NUTTX(_DIR/_BIN_DIR) instead CMAKE(_SRC_DIR/_BIN_DIR) 2026-08-09 11:13:08 -03:00
Kconfig arch/x86_64: Build fork() children from the caller's syscall frame. 2026-08-10 08:57:30 -03:00