The new CONFIG_FS_AIO_LISTIO_MAX option defaults to 10 and lio_listio()
now rejects nent > {AIO_LISTIO_MAX} with EINVAL. The LTP release pinned
by apps/testing/ltp (20230516) submits 256 requests in a single batch from
conformance/interfaces/lio_listio/2-1.c, so ltp_interfaces_lio_listio_2_1
now fails on every configuration that enables CONFIG_TESTING_LTP
(sim:citest, rv-virt:citest, sim:posix_test):
lio_listio/2-1.c Error at lio_listio() 22: Invalid argument
The EINVAL check itself is required by POSIX, so keep it and raise the
default instead; the limit no longer costs memory because the requests are
linked through the aiocb's own lio_link.
While here, keep _POSIX_AIO_LISTIO_MAX at its POSIX-mandated value of 2
and let AIO_LISTIO_MAX carry the configurable implementation limit.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
aio_fsync()/aio_read()/aio_write()/lio_listio() initialized
aiocbp->lio_link with list_initialize(), which makes the node
self-referential (prev = next = &node). aio_signal() tests
list_in_list(&lio_link) to detect lio_listio batches, so it wrongly
entered the lio_listio completion path for every standalone AIO
operation and notified through the uninitialized
lio_sigevent/lio_sigwork.
With CONFIG_SIG_EVTHREAD=y, garbage lio_sigevent.sigev_notify ==
SIGEV_THREAD caused nxsig_notification() to queue &lio_sigwork.work
onto the low-priority work queue with garbage func/value. After the
aiocb was freed, the dangling work_s was dispatched with worker=NULL,
crashing in work_dispatch().
Fix: initialize lio_link with list_clear_node() (prev = next = NULL)
so list_in_list() returns false for non-lio_listio operations and
aio_signal() skips the lio_listio path.
While there, reject a NULL aiocbp in aio_fsync(): POSIX Issue 6 no
longer defines a NULL special case, and the old DEBUGASSERT() panicked
debug builds.
Co-developed-by: dengwenqi <dengwenqi@xiaomi.com>
Co-developed-by: fangxinyong <fangxinyong@xiaomi.com>
Signed-off-by: fangxinyong <fangxinyong@xiaomi.com>
Signed-off-by: dengwenqi <dengwenqi@xiaomi.com>
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
lio_listio() links each aiocbp->lio_link into its batch list before
submitting the I/O, but submitted the operations through the public
aio_read()/aio_write(), which re-initialized lio_link and destroyed
the list membership. With an aiocb pre-filled with garbage (as in
ostest), the completion path then walked an invalid list.
Extract aio_read_internal()/aio_write_internal() that skip the
lio_link setup; aio_read()/aio_write() initialize lio_link (and
reject a NULL aiocbp) before calling the internal functions, while
lio_listio() calls the internal functions directly to preserve its
own lio_link setup. For entries that are not part of a batch,
lio_listio() self-initializes lio_link instead.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
aio_fsync() never initialized aiocbp->lio_link, but the reworked
aio_signal() tests list_in_list(&lio_link) on every completion. With
an uninitialized (or zero-filled) lio_link the behavior was
unpredictable; initialize the node so standalone fsync operations are
self-consistent.
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Per POSIX, aio_read() and aio_write() must return -1 and set errno to
EINVAL when the request cannot be queued (aio_reqprio < 0,
aio_offset < 0), and the error must also be retrievable via
aio_error(). Conversely, when queuing fails with a bad file
descriptor, the error belongs to the asynchronous operation: the
functions must return 0 and report EBADF through aio_error().
- Merge the offset/reqprio checks and return ERROR with errno set,
after storing the result in aio_result for aio_error().
- Drop the aio_fildes < 0 early return: a closed descriptor is now
caught by fcntl()/aio_queue() and reported through aio_result with
the function returning OK.
- aio_error(): report -EINVAL (failed validation) through errno
instead of returning it as an error value.
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
aioc_decant() frees the AIO container and detaches the aiocbp. The
I/O workers (aio_read_worker, aio_write_worker, aio_fsync_worker)
called it before signaling completion, so aio_signal() and any code
touching the container afterwards ran on freed memory. Additionally,
if the caller closed the file early the detached container could be
reused with a stale file reference. Move aioc_decant() to after
aio_signal() and use aioc->aioc_aiocbp directly in the workers.
aio_cancel() also had two problems: with no aiocbp it looped over
g_aio_pending with a do/while that skipped the list re-entry check, so
a failed work_cancel() on an already running I/O caused an endless
loop; and an invalid fildes only checked 'fildes < 0' instead of
validating the descriptor, so a closed fd was not reported as EBADF.
Use a for-loop that always advances and validate the descriptor with
file_get()/file_put().
Co-developed-by: wushenhui <wushenhui@xiaomi.com>
Signed-off-by: wushenhui <wushenhui@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
lio_listio() never validated 'nent' against {AIO_LISTIO_MAX}, so a
batch larger than the documented limit was silently accepted, and the
hard-coded _POSIX_AIO_LISTIO_MAX value of 2 was too small for real
workloads (LTP uses 10 entries per call).
Add the FS_AIO_LISTIO_MAX Kconfig option (default 10), use it for
_POSIX_AIO_LISTIO_MAX in include/limits.h, validate 'nent' in
lio_listio(), and report the limit through sysconf(_SC_AIO_LISTIO_MAX).
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
POSIX declares lio_listio() as:
int lio_listio(int, struct aiocb *restrict const [restrict], int,
struct sigevent *restrict);
Update the prototype in include/aio.h (and the implementation and
libc.csv entry) accordingly, and drop the parameter names from the
other aio_* prototypes for consistency.
Signed-off-by: guoshichao <guoshichao@xiaomi.com>
When a queued operation fails immediately (bad fd, EINVAL, or a failed
aio_read/aio_write submission), lio_listio() unconditionally deleted
the aiocbp from the request list. In LIO_WAIT mode (or when no sig was
requested) the lio_link nodes were never linked into the list, so
list_delete() corrupted memory and crashed.
Only unlink the node when it was actually linked, i.e. when
mode == LIO_NOWAIT and a sigevent was provided.
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
When lio_listio() is called with LIO_NOWAIT and a non-NULL sig, and no
I/O could be queued (or all entries are LIO_NOP/NULL), the completion
notification dereferences a NULL aiocbp picked from an empty iteration,
crashing nxsig_notification().
Scan the list for any non-NULL entry before delivering the
notification, and skip it entirely when the list contains only NULL
entries.
Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Previously, lio_listio() called aio_read()/aio_write() to submit the
I/O and only then initialized the per-request notification state
(aio_priv based), so a worker thread could complete an operation before
that state was set up (thread-unsafe), and the completion notification
hijacked the per-request sigevent machinery.
Rework the implementation: lio_listio() now links every aiocb of the
batch into a list (lio_link) before any I/O is submitted. When an
operation completes, aio_signal() removes its node from the list under
aio_lock() and delivers the lio_listio completion notification only
when the list becomes empty. The unused aio_priv field is replaced by
the lio_link/lio_sigevent/lio_sigwork fields in struct aiocb.
Co-developed-by: wushenhui <wushenhui@xiaomi.com>
Signed-off-by: wushenhui <wushenhui@xiaomi.com>
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
lio_listio() submits I/O through the internal aio_read/aio_write
helpers and is only built when CONFIG_FS_AIO is enabled. Keeping it in
libs/libc splits one subsystem across two directories and forces fs/aio
to export internal interfaces to the libc build.
Move the file (and its two build system entries) from libs/libc/aio to
fs/aio so that the whole AIO implementation lives in one place.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
Align the NuttX open(2) flag constants with the Linux asm-generic
values so that the FUSE wire protocol and other cross-platform
interfaces work without conversion.
All code that used '(flags & O_RDONLY)' as a bitmask check (always 0
now that O_RDONLY=0) has been updated to use '(flags & O_ACCMODE)'
comparisons.
The NUTTX_O_* constants in include/nuttx/fs/hostfs.h are updated to
match, and the sim hostfs open flag mapping is fixed.
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
debug.h is a NuttX-specific, non-POSIX header. Placing it in the
top-level include/ directory creates naming conflicts with external
projects that define their own debug.h.
This commit moves the canonical header to include/nuttx/debug.h,
following the NuttX convention for non-POSIX/non-standard headers,
and updates all in-tree references.
A backward-compatibility shim is left at include/debug.h that
emits a deprecation #warning and re-includes <nuttx/debug.h>,
allowing out-of-tree code to continue building while migrating.
Signed-off-by: Piyush Patle <piyushpatle228@gmail.com>
Most tools used for compliance and SBOM generation use SPDX identifiers
This change brings us a step closer to an easy SBOM generation.
Signed-off-by: Alin Jerpelea <alin.jerpelea@sony.com>
1. Update all CMakeLists.txt to adapt to new layout
2. Fix cmake build break
3. Update all new file license
4. Fully compatible with current compilation environment(use configure.sh or cmake as you choose)
------------------
How to test
From within nuttx/. Configure:
cmake -B build -DBOARD_CONFIG=sim/nsh -GNinja
cmake -B build -DBOARD_CONFIG=sim:nsh -GNinja
cmake -B build -DBOARD_CONFIG=sabre-6quad/smp -GNinja
cmake -B build -DBOARD_CONFIG=lm3s6965-ek/qemu-flat -GNinja
(or full path in custom board) :
cmake -B build -DBOARD_CONFIG=$PWD/boards/sim/sim/sim/configs/nsh -GNinja
This uses ninja generator (install with sudo apt install ninja-build). To build:
$ cmake --build build
menuconfig:
$ cmake --build build -t menuconfig
--------------------------
2. cmake/build: reformat the cmake style by cmake-format
https://github.com/cheshirekow/cmake_format
$ pip install cmakelang
$ for i in `find -name CMakeLists.txt`;do cmake-format $i -o $i;done
$ for i in `find -name *\.cmake`;do cmake-format $i -o $i;done
Co-authored-by: Matias N <matias@protobits.dev>
Signed-off-by: chao an <anchao@xiaomi.com>
NuttX kernel should not use the syscall functions, especially after
enabling CONFIG_SCHED_INSTRUMENTATION_SYSCALL, all system functions
will be traced to backend, which will impact system performance.
Signed-off-by: chao an <anchao@xiaomi.com>
since the code could map the unsupported work to the
supported one and remove select SCHED_WORKQUEUE from
Kconfig since SCHED_[L|H]PWORK already do the selection
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
aio/lio_listio.c:227:7: warning: implicit declaration of function ‘ferr’ [-Wimplicit-function-declaration]
227 | ferr("ERROR: lib_zalloc failed\n");
| ^~~~
aio/lio_listio.c:275:3: warning: implicit declaration of function ‘finfo’ [-Wimplicit-function-declaration]
275 | finfo("Registering signal handler\n");
aio/aio_read.c: In function ‘aio_read_worker’:
aio/aio_read.c:90:11: warning: implicit declaration of function ‘file_pread’; did you mean ‘aio_read’? [-Wimplicit-function-declaration]
90 | nread = file_pread(aioc->aioc_filep, (FAR void *)aiocbp->aio_buf,
| ^~~~~~~~~~
| aio_read
aio/aio_write.c: In function ‘aio_write_worker’:
aio/aio_write.c:85:12: warning: implicit declaration of function ‘file_fcntl’ [-Wimplicit-function-declaration]
85 | oflags = file_fcntl(aioc->aioc_filep, F_GETFL);
| ^~~~~~~~~~
CC: mmap/fs_mmap.c
CC: pthread/pthread_condclockwait.c
aio/aio_write.c:107:18: warning: implicit declaration of function ‘file_write’; did you mean ‘aio_write’? [-Wimplicit-function-declaration]
107 | nwritten = file_write(aioc->aioc_filep,
| ^~~~~~~~~~
| aio_write
aio/aio_write.c:113:18: warning: implicit declaration of function ‘file_pwrite’; did you mean ‘aio_write’? [-Wimplicit-function-declaration]
113 | nwritten = file_pwrite(aioc->aioc_filep,
| ^~~~~~~~~~~
| aio_write
In file included from aio/aio_write.c:34:
aio/aio_write.c:121:12: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘ssize_t’ {aka ‘long int’} [-Wformat=]
121 | ferr("ERROR: write/pwrite/send failed: %d\n", nwritten);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~
| |
| ssize_t {aka long int}
aio/aio_write.c:121:47: note: format string is defined here
121 | ferr("ERROR: write/pwrite/send failed: %d\n", nwritten);
| ~^
| |
| int
| %ld
Signed-off-by: Xiang Xiao <xiaoxiang@xiaomi.com>
All complaints fixed except for those that were not possible to fix:
- Used of Mixed case identifier in ESP32 files. These are references to Expressif ROM functions which are outside of the scope of NuttX.
This commit resolves issue #620:
Remove CONFIG_CAN_PASS_STRUCTS #620
The configuration option CONFIG_CAN_PASS_STRUCTS was added many years ago to support an old version of the SDCC compiler. That compiler is currently used only with the Z80 and Z180 targets. The limitation of that old compiler was that it could not pass structures or unions as either inputs or outputs. For example:
#ifdef CONFIG_CAN_PASS_STRUCTS
struct mallinfo mallinfo(void);
#else
int mallinfo(FAR struct mallinfo *info);
#endif
And even leads to violation of a few POSIX interfaces like:
#ifdef CONFIG_CAN_PASS_STRUCTS
int sigqueue(int pid, int signo, union sigval value);
#else
int sigqueue(int pid, int signo, FAR void *sival_ptr);
#endif
This breaks the 1st INVIOLABLES rule:
Strict POSIX compliance
-----------------------
o Strict conformance to the portable standard OS interface as defined at
OpenGroup.org.
o A deeply embedded system requires some special support. Special
support must be minimized.
o The portable interface must never be compromised only for the sake of
expediency.
o Expediency or even improved performance are not justifications for
violation of the strict POSIX interface
Also, it appears that the current SDCC compilers have resolve this issue and so, perhaps, this is no longer a problem: z88dk/z88dk#1132
NOTE: This commit cannot pass the PR checks because it depends on matching changes to the apps/ directory.
Resolution of Issue 619 will require multiple steps, this part of the first step in that resolution: Every call to nxsem_wait_uninterruptible() must handle the return value from nxsem_wait_uninterruptible properly. This commit is only for those files under fs/driver, fs/aio, fs/nfs, crypto/, and boards/.
Please note: The modified file under fs/nfs generates several " Mixed case identifier found" errors. Please ignore these. These cannot be fixed without changes to numerous other files. They also follow a non-standard convention that is used many files: Using lower case structure names in custom SIZEOF_ definitions.
* Simplify EINTR/ECANCEL error handling
1. Add semaphore uninterruptible wait function
2 .Replace semaphore wait loop with a single uninterruptible wait
3. Replace all sem_xxx to nxsem_xxx
* Unify the void cast usage
1. Remove void cast for function because many place ignore the returned value witout cast
2. Replace void cast for variable with UNUSED macro
