Rewrite arch_memcpy.S and arch_memset.S to be register-width aware on
both RV32 and RV64 using REG_L/REG_S/SZREG macros from asm.h.
memcpy gains:
- 16xSZREG unrolled main loop (128B/iter on RV64, 64B on RV32).
- Shift-merge path for misaligned src: reads two aligned words
straddling each output word and shifts them together, so no load
or store is ever misaligned.
- Single SZREG and byte loops for remainder and small copies.
memset gains:
- 32xSZREG unrolled main loop (256B/iter on RV64, 128B on RV32)
using Duff's device for non-power-of-two remainders.
- .option norvc ensures fixed 4-byte instruction width for correct
jump offset calculation in the Duff's device entry.
- Zero-length input handled correctly (branch to guarded tail).
The old memcpy always used lw/sw even on RV64, wasting half the
memory bandwidth. The old memset unrolled only 16 bytes per iteration.
Signed-off-by: ganjing <ganjing@xiaomi.com>
Add word-at-a-time strlcpy using DETECTNULL for both the copy phase
and the strlen tail when truncated. The copy loop aligns src and
processes a register at a time, falling to bytewise for the last word
containing the terminator. When truncated, the remaining src length
is measured with a second word-at-a-time loop.
strlcpy has 46 call sites in a typical kernel image (more than strcpy)
and is not covered by newlib OPTSPEED, making it a high-value target.
Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: ganjing <ganjing@xiaomi.com>
Reduce branch overhead in the memcmp main loop by comparing four
words per iteration: XOR each pair, OR the four differences together,
and branch once. On a mismatch the single-word loop locates the
exact differing word within four words of the fault.
Add a beqz guard at .Lbyte_cmp entry to handle the case where the
4-word loop consumes all remaining bytes exactly.
Measured on QEMU RV32: memcmp(128) 313 -> 271 cycles (13% faster).
Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: ganjing <ganjing@xiaomi.com>
Add assembly-optimized implementations for 14 string/memory functions
using word-at-a-time techniques (DETECTNULL, broadcast+XOR) and
XLEN-adaptive macros for both RV32 and RV64:
- memmove: direction check + forward tail to memcpy, reverse path
with 16xSZREG unroll and shift-merge for misaligned src.
- memcmp: word-granularity compare when both pointers share alignment,
bytewise fallback for mismatched pointers.
- memchr: broadcast target byte, XOR with each word, DETECTNULL to
find matches. Counter-based bounds (no pointer overflow).
- strlen: DETECTNULL word loop, constants loaded from .srodata.
- strnlen: strlen with counter-based length limit.
- strcpy/strncpy: word loop with DETECTNULL, zero-fill remainder
for strncpy. strncpy reuses strcpy via #define USE_AS_STRNCPY.
- stpcpy/stpncpy: reuse strcpy/strncpy via #define USE_AS_STPCPY.
- strchr/strchrnul: broadcast+XOR detecting both target char and
null simultaneously. strchrnul reuses strchr via #define.
- strrchr: forward scan recording last match position.
- strncmp: word-at-a-time compare with null detection and counter.
- strcat: strlen(dst) then strcpy(dst_end, src) word-at-a-time.
Each function is independently selectable via CONFIG_RISCV_<FUNC>,
or all enabled together with CONFIG_RISCV_STRING_FUNCTION=y.
Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: ganjing <ganjing@xiaomi.com>
Implement architecture-specific ELF header definitions and relocation handling
for the MIPS architecture to enable loadable modules.
Fixes#19178.
Changes include:
- Add `arch/mips/include/elf.h` with MIPS ELF relocation types and
architecture-specific ELF data structures (`arch_elfdata_s`).
- Implement `libs/libc/machine/mips/arch_elf.c` containing `up_checkarch`,
`up_relocate`, and `up_relocateadd` functions handling `R_MIPS_NONE`,
`R_MIPS_32`, `R_MIPS_26`, `R_MIPS_HI16`, and `R_MIPS_LO16` relocations.
- Integrate MIPS machine-specific C library support in
`libs/libc/machine/mips/Make.defs`.
- Update `LDMODULEFLAGS` in `arch/mips/src/mips32/Toolchain.defs` to include the
little-endian (`-EL`) flag.
- Update `up_coherent_dcache` for proper cache synchronization on JZ4780.
Signed-off-by: Lwazi Dube <lwazeh@gmail.com>
1. The original method of restoring the register context relies on
the ret instruction, which does not conform to the semantics of
longjmp not returning.
2. There are csa leak during longjmp and need recycle.
Signed-off-by: zhangyu117 <zhangyu117@xiaomi.com>
Add setjmp/longjmp support for TriCore architecture using iLLD
intrinsics to save and restore the CSA (Context Save Area) chain.
This implementation saves upper/lower context registers and walks
the CSA linked list to restore the full call context on longjmp.
Signed-off-by: zhangyuan29 <zhangyuan29@xiaomi.com>
Add dedicated NEON implementations for mutually aligned medium and long memcpy copies when building with __ARM_NEON__. These paths use NEON multi-register loads and stores while preserving the existing VFP implementation for non-NEON VFP configurations.
NEON builds also define USE_VFP, so select the NEON implementation explicitly before falling back to VFP. Apply the same aligned-copy optimization to the armv7-a, armv7-r, and armv8-r implementations.
Signed-off-by: yaojiaqi <yaojiaqi@lixiang.com>
debug.h is a NuttX-specific, non-POSIX header. Placing it in the
top-level include/ directory creates naming conflicts with external
projects that define their own debug.h.
This commit moves the canonical header to include/nuttx/debug.h,
following the NuttX convention for non-POSIX/non-standard headers,
and updates all in-tree references.
A backward-compatibility shim is left at include/debug.h that
emits a deprecation #warning and re-includes <nuttx/debug.h>,
allowing out-of-tree code to continue building while migrating.
Signed-off-by: Piyush Patle <piyushpatle228@gmail.com>
Add support for ARMv6-M and ARMv8-M Baseline architectures
(Cortex-M0/M0+/M23) in the mcount profiling function. These cores
only support limited Thumb-1 instruction set and require different
assembly instructions compared to ARMv7-M and higher.
Changes:
- Use MOVS+BICS instead of BIC for bit clearing on M0/M23
- Separate register restore for limited push/pop instructions
- Use BX instead of direct POP to PC on M0/M23
Signed-off-by: yinshengkai <yinshengkai@bytedance.com>
when we build with greenhills compiler, if the exception_common function
is put into a separate section: ".text.exception_common", then the
address of exception_common function is an even number, for example "0x16a6c",
in this case, it will trigger the UsageFaults:INVSTATE error.
The reason for this error is that in the GHS compiler, a section declared
through the ".section" directive must specify at least the "a" attribute.
Otherwise, this section will not be linked into the final file, and this
is why the address of exception_common function is invalid.
The following is the official explanation in the GHS compiler documentation:
"The reason for this error is that in the GHS compiler, a section
declared through the .section directive must specify at least the "a"
attribute. Otherwise, this section will not be linked into the final
file. The following is the official explanation in the documentation:
"Sections that are intended be part of the final linked output should
have at least the `a` attribute"
and for GHS compiler, the .text section is recommended specify the
attribute of `ax`.
Signed-off-by: guoshichao <guoshichao@xiaomi.com>
This allows the compiler to automatically identify which string functions
can be compiled into libraries, and the compiler's internal implementation
is faster than libc functions.
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
(gdb) bt
at /home/mi/ssd/dev-system/nuttx/include/string.h:321
ppcmd=<synthetic pointer>, vtbl=0x80069498) at nsh_parse.c:1909
param=0x800692b8) at nsh_parse.c:2593
cmdline=cmdline@entry=0x80069750 "hello") at nsh_parse.c:3028
argc=argc@entry=1, argv=argv@entry=0x80068870) at nsh_session.c:246
at nsh_consolemain.c:75
at nsh_main.c:74
...
Signed-off-by: yangao1 <yangao1@xiaomi.com>
riscv-none-elf-ld: /home/ligd/platform/dev-system/nuttx/staging/libc.a(arch_libc.o): in function `memcpy':
/home/ligd/platform/dev-system/nuttx/libs/libc/machine/arch_libc.c:131:(.text.memcpy+0x4e): undefined reference to `arch_memcpy'
Signed-off-by: ligd <liguiding1@xiaomi.com>
Add back these functions since clang will use them.
This reverts "libs/libc/arm: use builtin routines instead of aliases of __aeabi_mem*"
and adds source files to cmake.
Signed-off-by: chenxiaoyi <chenxiaoyi@xiaomi.com>
Fix an issue on ARMv8-R where NEON is not supported.
When NEON is disabled, the assembly-optimized implementations
of memset and memcpy should not be used.
Signed-off-by: Chengdong Wang <wangchengdong@lixiang.com>
Prior to this commit, it wasn't possible to load ELF modules from
the external PSRAM. There were two main issues about it: 1) copying
data using the instruction bus was being used instead of the data
bus (this, per si, isn't a problem, but requires special attention
regarding data alignment), and 2) the cache was not being properly
cleaned and flushed to properly access the loaded data using the
instruction bus.
Signed-off-by: Tiago Medicci Serrano <tiago.medicci@espressif.com>
Optimize crc32 standard(poly:0x04C11DB7) and crc32
castagnoli(poly:0x1EDC6F41) with arm crc32 extension instructions.
For example, crc32 standard caculates(lookup crc32 table) 1812 bytes data,
reduced the time from 118 us to 14 us through optimization.
Performance improved ~700%
Signed-off-by: Jinliang Li <lijinliang1@lixiang.com>
Summary:
- Replace direct use of `fence.i` instruction with `__ISB()` macro for instruction synchronization
- This change improves portability while maintaining the same functionality
Impact:
- No functional changes - both `fence.i` and `__ISB()` ensure instruction
synchronization on RISC-V
- Makes the code more maintainable by using the architecture abstraction
layer
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
It will be used to distinguish between hardware KASan and software KASan. Hardware KASan does not need to use plug-in
Signed-off-by: wangmingrong1 <wangmingrong1@xiaomi.com>
Modify the kernel to use only atomic_xx and atomic64_xx interfaces,
avoiding the use of sizeof or typeof to determine the type of
atomic operations, thereby simplifying the kernel's atomic
interface operations.
Signed-off-by: zhangyuan29 <zhangyuan29@xiaomi.com>
