Aksal/nuttx-apps
1
0
Fork 1
mirror of https://github.com/apache/nuttx-apps.git synced 2026-08-16 18:03:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
nuttx-apps/interpreters/python/Makefile
Marco Casaroli 860815a33a testing/ostest: Split the fork test into vfork and fork. 
ostest's "vfork" test was never testing vfork().  It has the child write a
global and the parent observe the write -- the defining property of *sharing*,
not of vfork(), whose defining property is that the parent is suspended and
whose contract forbids the child to write anything at all.  It passed because
NuttX implemented fork() and vfork() as the same sharing primitive, which
apache/nuttx#19562 separates.

vfork.c is rewritten to test what vfork() promises.  The child does only what
POSIX permits -- it calls _exit(42) and nothing else, not even exit(), which
would run atexit handlers and flush stdio in the parent's address space.  Since
the child may not write memory and the parent cannot run while the child lives,
the observable is the child's exit status:  had the parent not been suspended,
it would have reached waitpid() while the child was still alive.  Where child
status is not retained -- ostest_main() sets SA_NOCLDWAIT for the whole run,
deliberately -- ECHILD is accepted as equally good evidence, since it says the
child was already gone when the parent asked.

fork.c is new and tests POSIX fork():  the child's writes to .data, .bss and
the heap are invisible to the parent and vice versa, a pointer to a stack local
taken before the fork names the same object in both, and the child does
everything a vfork() child may not -- calls malloc() and printf(), and returns
from the function that called fork().

Both run at the top of user_main().  They exercise the lowest-level machinery
in the suite -- address environments, stack setup, the architecture's register
context -- so a fault in one takes the process down instead of reporting a
failure.  Learning that in seconds rather than after everything else has passed
matters when a port is being brought up.

Each test gates on the one primitive it tests, ARCH_HAVE_VFORK and
ARCH_HAVE_FORK respectively.  There is no compatibility layer and no mapping
between symbols.  vfork.c no longer requires SCHED_WAITPID:  the suspension is
in the kernel primitive now, so the test's core assertion holds without it and
only the status check is conditional.

The simulator is the one exception.  It selects ARCH_HAVE_VFORK, but ostest
takes the sim down as soon as the test runs there, so the call keeps the
!ARCH_SIM guard that apps ee7642793 put on the old test in 2024.  The old gate
hid this:  ARCH_HAVE_FORK is not set on the sim, so the test was not built
there at all.

The other in-tree callers are audited for which primitive they actually meant:

* interpreters/python's _posixsubprocess and netutils/libwebsockets'
  LWS_HAVE_WORKING_VFORK want the fork-then-exec path -- ARCH_HAVE_VFORK.
* python's os.fork() and libwebsockets' LWS_HAVE_FORK mean real fork() and stay
  on ARCH_HAVE_FORK, so they become *absent* rather than silently wrong.
* testing/fs/fdsantest's vfork case follows ARCH_HAVE_VFORK.

interpreters/bas is deliberately left alone.  Its SHELL and EDIT statements
reach for vfork() under an ARCH_HAVE_FORK guard and want the same treatment,
but checkpatch.sh checks the whole of any file a patch touches and
bas_statement.c produces 1681 pre-existing findings against master, so a
one-line change there fails CI on its own.  The consequence is small:
EXAMPLES_BAS_SHELL is EXPERIMENTAL and already depends on ARCH_HAVE_FORK, so it
becomes unselectable rather than misbehaving.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
2026-08-14 07:46:54 -03:00

296 lines
13 KiB
Makefile
Raw Blame History

############################################################################
# apps/interpreters/python/Makefile
#
# SPDX-License-Identifier: Apache-2.0
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership. The
# ASF licenses this file to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance with the
# License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
############################################################################
include $(APPDIR)/Make.defs
CPYTHON_URL ?= "https://github.com/python/cpython/archive"
CPYTHON_VERSION = $(patsubst "%",%,$(strip $(CONFIG_INTERPRETERS_CPYTHON_VERSION)))
CPYTHON_VERSION_MINOR=$(basename $(CPYTHON_VERSION))
CPYTHON_ZIP = v$(CPYTHON_VERSION).zip
CPYTHON_UNPACKNAME = Python
UNPACK ?= unzip -q -o
MACHDEP=nuttx
CONFIG_SITE=${CURDIR}/config.site
SETUP_LOCAL=${CURDIR}/Setup.local
CPYTHON_PATH=$(CURDIR)/$(CPYTHON_UNPACKNAME)
BUILDIR=$(CURDIR)/build
INSTALLDIR=$(CURDIR)/install
HOSTBUILD=$(BUILDIR)/host
HOSTINSTALL=$(INSTALLDIR)/host
HOSTPYTHON=$(HOSTINSTALL)/bin/python3
TARGETBUILD=$(BUILDIR)/target
TARGETINSTALL=$(INSTALLDIR)/target
TARGETLIBPYTHON=$(TARGETINSTALL)/libpython$(CPYTHON_VERSION_MINOR).a
TARGETMODULESPACK=$(TARGETBUILD)/lib/python$(shell echo $(CPYTHON_VERSION_MINOR) | tr -d .).zip
TARGETMODULES=$(TARGETINSTALL)/lib/
CFLAGS += ${INCDIR_PREFIX}$(CPYTHON_PATH)$(DELIM)Include
CFLAGS += ${INCDIR_PREFIX}$(CPYTHON_PATH)$(DELIM)Test
CFLAGS += ${INCDIR_PREFIX}$(CPYTHON_PATH)$(DELIM)Include$(DELIM)internal
CFLAGS += ${INCDIR_PREFIX}$(APPDIR)$(DELIM)system
CFLAGS += ${INCDIR_PREFIX}$(APPDIR)$(DELIM)system$(DELIM)zlib$(DELIM)zlib
CFLAGS += ${INCDIR_PREFIX}$(TARGETBUILD)
CFLAGS += -Wno-shadow -Wno-undef -Wno-format -Wno-builtin-macro-redefined
CFLAGS += -Wno-type-limits -Wno-implicit-fallthrough -Wno-char-subscripts
CFLAGS += -Wno-sign-compare -Wno-unused-const-variable -Wno-unused-function
CFLAGS += -Wno-unused-variable -Wno-overflow -Wno-unused-but-set-variable
CFLAGS += -Wno-strict-prototypes -Wno-maybe-uninitialized -nostdlib
DEPPATH += --dep-path $(CPYTHON_UNPACKNAME)$(DELIM)Programs
VPATH += :$(CPYTHON_UNPACKNAME)$(DELIM)Programs
$(CPYTHON_ZIP):
@echo "Downloading: $(CPYTHON_URL)/$(CPYTHON_ZIP)"
$(Q) $(call DOWNLOAD,$(CPYTHON_URL),$(CPYTHON_ZIP))
$(CPYTHON_UNPACKNAME): $(CPYTHON_ZIP)
@echo "Unpacking: $(CPYTHON_ZIP) -> $(CPYTHON_UNPACKNAME)"
$(Q) $(UNPACK) $(CPYTHON_ZIP)
$(Q) mv cpython-$(CPYTHON_VERSION) $(CPYTHON_UNPACKNAME)
@echo "Patching $(CPYTHON_UNPACKNAME)"
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0001-workaround-newlib-resource.h-limitations.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0002-fix-various-uint32_t-unsigned-int-type-mismatch-issu.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0003-reuse-wasm_assets.py-for-generating-an-archive-of-py.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0004-recognize-nuttx-as-a-supported-OS.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0005-gh-122907-Fix-Builds-Without-HAVE_DYNAMIC_LOADING-Se.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0006-change-var-name-to-avoid-conflict-with-nuttx-unused_.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0007-undef-atexit_register.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0008-declare-struct-timeval.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0009-include-nuttx-sys-select-header-to-define-FD_SETSIZE.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0010-check-for-the-d_ino-member-of-the-structure-dirent.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0011-avoid-redefinition-warning-if-UNUSED-is-already-defi.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0012-hack-place-_PyRuntime-structure-into-PSRAM-bss-regio.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0013-transform-functions-used-by-NuttX-to-lowercase.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0014-insert-prefix-to-list_length-to-avoid-symbol-collisi.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0015-keep-ensurepip-in-stdlib-archive.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0016-fix-timezone-offset-check-when-time-t-is-unsigned.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0017-stdlib-zip-keep-pydecimal-and-trim-tooling-extras.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0018-ignore-chmod-on-nuttx-like-wasi.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0019-undef-get_errno-set_errno-to-avoid-nuttx-macro-coll.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0020-ctypes-skip-pythonapi-on-nuttx.patch
$(Q) patch -p1 -d $(CPYTHON_UNPACKNAME) < patch$(DELIM)0021-posixmodule-ignore-utime-enosys-on-nuttx.patch
$(HOSTPYTHON):
mkdir -p $(HOSTBUILD)
mkdir -p $(HOSTINSTALL)
$(Q) ( \
cd $(HOSTBUILD) && $(CPYTHON_PATH)/configure \
--with-pydebug \
--prefix=$(HOSTINSTALL) \
--disable-test-modules \
)
$(MAKE) -C $(HOSTBUILD) install
# The `config.site` file contains settings that override the configuration
# settings provided by the `configure` script. Depending on the features
# enabled on NuttX, this file may need to be adjusted.
$(CONFIG_SITE):
$(Q) ( cp $(CONFIG_SITE).in $(CONFIG_SITE))
ifeq ($(CONFIG_ARCH_HAVE_FORK),y)
@echo "export ac_cv_func_fork=\"yes\"" >> $@
else
@echo "export ac_cv_func_fork=\"no\"" >> $@
endif
ifneq ($(CONFIG_ARCH_HAVE_VFORK),)
@echo "export ac_cv_func_vfork=\"yes\"" >> $@
else
@echo "export ac_cv_func_vfork=\"no\"" >> $@
endif
ifeq ($(CONFIG_SYSTEM_SYSTEM),y)
@echo "export ac_cv_func_system=\"yes\"" >> $@
else
@echo "export ac_cv_func_system=\"no\"" >> $@
endif
ifeq ($(CONFIG_NET),y)
@echo "export ac_cv_func_getaddrinfo=\"yes\"" >> $@
@echo "export ac_cv_func_getnameinfo=\"yes\"" >> $@
else
@echo "export ac_cv_func_getaddrinfo=\"no\"" >> $@
@echo "export ac_cv_func_getnameinfo=\"no\"" >> $@
endif
# The `Setup.local` file enables or disables Python modules.
# Depending on the features enabled on NuttX, this file may need to be
# adjusted. Please note that the base `Setup.local.in` file only contains
# a section to disable Python modules. Inserting lines to it will disable
# such modules.
$(SETUP_LOCAL):
$(Q) ( cp $(SETUP_LOCAL).in $(SETUP_LOCAL))
# _posixsubprocess is the fork-then-exec path, so vfork() is enough for it;
# os.fork() itself needs a real fork() and is governed by ac_cv_func_fork
# above.
ifeq ($(CONFIG_ARCH_HAVE_FORK)$(CONFIG_ARCH_HAVE_VFORK),)
@echo "_posixsubprocess" >> $@
endif
ifneq ($(CONFIG_LIBC_DLFCN),y)
@echo "_ctypes" >> $@
endif
ifneq ($(CONFIG_NET),y)
@echo "_socket" >> $@
endif
# For the Python's `configure` script, please consider the following
# when building for NuttX:
#
# Use sed to remove optimization flags from NuttX's CFLAGS because
# Python's configure script requires them in OPT. Having the flags in
# both places causes a conflict.
#
# Also, use -O0 for OPT because -Os is causing problems in
# Python/Modules/getpath.c (issue will be filed soon to track this
# problem).
ifneq ($(CONFIG_NET),y)
PYTHON_CONFIGURE_EXTRAS = --disable-ipv6
else
PYTHON_CONFIGURE_EXTRAS =
endif
$(TARGETBUILD)/Makefile: $(HOSTPYTHON) $(CONFIG_SITE) $(SETUP_LOCAL)
$(Q) mkdir -p $(TARGETBUILD)/Modules
$(Q) mkdir -p $(TARGETMODULES)/python$(CPYTHON_VERSION_MINOR)
$(Q) ( cp Setup.local $(TARGETBUILD)/Modules/Setup.local )
$(Q) ( \
cd $(TARGETBUILD); \
CFLAGS="$(CFLAGS)"; \
ARCH=$(CONFIG_ARCH); \
ARCH_CHIP=$(CONFIG_ARCH_CHIP); \
ARCH="$${ARCH//-/}"; \
ARCH_CHIP="$${ARCH_CHIP//-/}"; \
CFLAGS="$$(echo "$${CFLAGS}" | sed 's/-Os //')" \
CC="$(CC)" \
CXX="$(CXX)" \
AR="$(AR)" \
ARFLAGS=" " \
MACHDEP="$(MACHDEP)" \
OPT="-Os" \
CONFIG_SITE="$(CONFIG_SITE)" \
$(CPYTHON_PATH)/configure \
--prefix=${TARGETINSTALL} \
--disable-shared \
--host=$${ARCH}-$${ARCH_CHIP}-nuttx \
--build=$(shell $(CPYTHON_PATH)/config.guess) \
--with-build-python=${HOSTPYTHON} \
--without-mimalloc \
--without-pymalloc \
--disable-test-modules \
--with-ensurepip=no \
$(PYTHON_CONFIGURE_EXTRAS) \
)
$(Q) sed -i 's/^#define HAVE_LIBB2 1/\/* #undef HAVE_LIBB2 *\//' $(TARGETBUILD)/pyconfig.h
$(Q) sed -i 's/-lb2//g' $(TARGETBUILD)/Makefile
BUNDLED_WHEELS_DIR = $(CPYTHON_PATH)/Lib/ensurepip/_bundled
# PyPI downloads use the host system pip, not $(HOSTPYTHON)'s ensurepip bundle:
# pip 24.x vendors urllib3 that breaks on Python 3.13 (zlib.Decompress.unused_data).
HOSTPIP = python3 -m pip
$(TARGETLIBPYTHON): $(TARGETBUILD)/Makefile
ifneq ($(or $(filter y,$(CONFIG_INTERPRETERS_CPYTHON_ENABLE_PIP)),$(filter y,$(CONFIG_INTERPRETERS_CPYTHON_INSTALL_NUTTX_PACKAGE))),)
$(Q) mkdir -p $(BUNDLED_WHEELS_DIR)
endif
ifeq ($(CONFIG_INTERPRETERS_CPYTHON_ENABLE_PIP),y)
$(Q) ( \
PIP_WHEEL_VERSION=$$($(HOSTPYTHON) -c "import ensurepip; print(ensurepip._PIP_VERSION)"); \
PIP_WHEEL=$(BUNDLED_WHEELS_DIR)/pip-$${PIP_WHEEL_VERSION}-py3-none-any.whl; \
if [ ! -f "$${PIP_WHEEL}" ]; then \
echo "Fetching pip wheel $${PIP_WHEEL_VERSION} for ensurepip bundle"; \
$(HOSTPIP) download --only-binary=:all: --no-deps --dest $(BUNDLED_WHEELS_DIR) pip==$${PIP_WHEEL_VERSION}; \
fi; \
echo "Pre-compiling pip wheel with build Python (must match embedded CPython version)"; \
$(HOSTPYTHON) $(CURDIR)/repack_wheel_add_pyc.py --package pip "$${PIP_WHEEL}"; \
)
endif
ifeq ($(CONFIG_INTERPRETERS_CPYTHON_INSTALL_NUTTX_PACKAGE),y)
$(Q) ( \
set -e; \
NUTTX_WHEEL=$$(ls $(BUNDLED_WHEELS_DIR)/nuttx_periphery-*-py3-none-any.whl 2>/dev/null | head -1 || true); \
if [ -z "$${NUTTX_WHEEL}" ]; then \
echo "Fetching latest nuttx-periphery wheel for ensurepip bundle"; \
$(HOSTPIP) download --only-binary=:all: --no-deps --dest $(BUNDLED_WHEELS_DIR) nuttx-periphery; \
NUTTX_WHEEL=$$(ls $(BUNDLED_WHEELS_DIR)/nuttx_periphery-*-py3-none-any.whl); \
fi; \
echo "Pre-compiling nuttx-periphery wheel with build Python (must match embedded CPython version)"; \
$(HOSTPYTHON) $(CURDIR)/repack_wheel_add_pyc.py --package nuttx_periphery "$${NUTTX_WHEEL}"; \
)
endif
$(MAKE) -C $(TARGETBUILD) regen-frozen
$(MAKE) -C $(TARGETBUILD) libpython$(CPYTHON_VERSION_MINOR).a wasm_stdlib
$(Q) ( cp $(TARGETBUILD)/libpython$(CPYTHON_VERSION_MINOR).a $(TARGETLIBPYTHON) )
$(Q) $(UNPACK) $(TARGETMODULESPACK) -d $(TARGETMODULES)/python$(CPYTHON_VERSION_MINOR)
ifneq ($(or $(filter y,$(CONFIG_INTERPRETERS_CPYTHON_ENABLE_PIP)),$(filter y,$(CONFIG_INTERPRETERS_CPYTHON_INSTALL_NUTTX_PACKAGE))),)
$(Q) mkdir -p $(TARGETMODULES)/python$(CPYTHON_VERSION_MINOR)/site-packages
$(Q) ( \
set -e; \
BUNDLED_DIR=$(TARGETMODULES)/python$(CPYTHON_VERSION_MINOR)/ensurepip/_bundled; \
SITE_PACKAGES=$(TARGETMODULES)/python$(CPYTHON_VERSION_MINOR)/site-packages; \
: > "$${SITE_PACKAGES}/bundled_wheels.pth"; \
for wheel in $${BUNDLED_DIR}/*.whl; do \
[ -f "$${wheel}" ] || continue; \
whl_name=$$(basename "$${wheel}"); \
echo "Pre-installing wheel via zipimport into target sys.path: $${whl_name}"; \
echo "../ensurepip/_bundled/$${whl_name}" >> "$${SITE_PACKAGES}/bundled_wheels.pth"; \
done; \
)
endif
MODULE = $(CONFIG_INTERPRETERS_CPYTHON)
PROGNAME += $(CONFIG_INTERPRETERS_CPYTHON_PROGNAME)
PRIORITY += $(CONFIG_INTERPRETERS_CPYTHON_PRIORITY)
STACKSIZE += $(CONFIG_INTERPRETERS_CPYTHON_STACKSIZE)
MAINSRC += python_wrapper.c
checkgenromfs:
@genromfs -h 1>/dev/null 2>&1 || { \
echo "Host executable genromfs not available in PATH"; \
echo "You may need to download in from https://romfs.sourceforge.net/"; \
exit 1; \
}
romfs_cpython_modules.img : $(TARGETLIBPYTHON) checkgenromfs
@genromfs -f $@ -d $(TARGETMODULES) -V "ROMFS_Test" || { echo "genromfs failed" ; exit 1 ; }
romfs_cpython_modules.h : romfs_cpython_modules.img
@xxd -i $< | sed -e "s/^unsigned/static const unsigned/g" >$@ || { echo "xxd of $< failed" ; exit 1 ; }
context:: $(CPYTHON_UNPACKNAME)
depend:: romfs_cpython_modules.h
distclean::
$(call DELDIR, $(BUILDIR))
$(call DELDIR, $(INSTALLDIR))
$(call DELDIR, $(CPYTHON_UNPACKNAME))
$(call DELFILE, $(CPYTHON_ZIP))
$(call DELFILE, romfs_cpython_modules.img)
$(call DELFILE, romfs_cpython_modules.h)
$(call DELFILE, config.site)
$(call DELFILE, Setup.local)
include $(APPDIR)/Application.mk
Reference in a new issue View git blame Copy permalink