Aksal/nuttx-apps
1
0
Fork 1
mirror of https://github.com/apache/nuttx-apps.git synced 2026-09-01 06:33:07 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

system/nxpkg: network sync, install hardening, and CLI completion.

Browse source 
Fill in most of what the initial nxpkg slice deferred: network sync
and artifact acquisition over plain HTTP (verified via SHA-256), an
install rewrite that supports network sources and reclaims state left
by an interrupted previous install, and wiring update/remove/rollback/
available into the CLI.

Harden the package path against untrusted input along the way: bounded
memory-safety helpers and explicit size limits throughout, storage
read/write hardened against partially-written files, path-traversal
rejection in manifest name/version before they reach the filesystem,
and a fix for a lost-update race on the shared installed-packages
database (two concurrent installs of different packages could
otherwise silently clobber each other's recorded state).

Add an optional manifest icon field for the nxstore GUI frontend to
consume, raise PKG_INDEX_MAX now that the in-memory index is
heap-allocated, and document the repository layout and local server
setup in system/nxpkg/README.txt.

Also fixes a real build break: pkg_runtime_compat() unconditionally
referenced CONFIG_ARCH_BOARD, a Kconfig string symbol with no default
clause under ARCH_BOARD_CUSTOM, so it is left entirely undefined
rather than defined-but-empty on a custom board - falls back to
CONFIG_ARCH_BOARD_CUSTOM_NAME instead. Routes pkg_error()/pkg_info()
through syslog rather than stdio, since neither is visible to a
supervisor with no attached console.

Assisted-by: Claude:claude-sonnet-5
Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
This commit is contained in:
aviralgarg05 2026-07-16 13:21:47 +05:30
parent 094b988be3
commit 81b88922f7
13 changed files with 2263 additions and 187 deletions
Download patch file Download diff file Expand all files Collapse all files

1
system/nxpkg/CMakeLists.txt
View file

@ -38,6 +38,7 @@ if(CONFIG_SYSTEM_NXPKG)
pkg_log.c
pkg_manifest.c
pkg_metadata.c
pkg_repo.c
pkg_store.c
pkg_txn.c)
endif()

9
system/nxpkg/Kconfig
View file

@ -29,4 +29,13 @@ config SYSTEM_NXPKG_STACKSIZE
int "'nxpkg' stack size"
default 16384
config SYSTEM_NXPKG_ROOT
string "'nxpkg' storage root"
default "/tmp/nxpkg"
---help---
Base directory used by nxpkg for its local index, installed
metadata, temporary downloads, and package payload storage.
Boards with external storage can point this to a persistent
location such as /mnt/sdcard/nxpkg.
endif

1
system/nxpkg/Makefile
View file

@ -29,6 +29,7 @@ MODULE = $(CONFIG_SYSTEM_NXPKG)
CSRCS = pkg_compat.c pkg_hash.c pkg_install.c pkg_log.c pkg_manifest.c
CSRCS += pkg_metadata.c pkg_store.c pkg_txn.c
CSRCS += pkg_repo.c
MAINSRC = pkg_main.c
include $(APPDIR)/Application.mk

175
system/nxpkg/README.txt Normal file
View file

@ -0,0 +1,175 @@
nxpkg - a package manager for NuttX application images
========================================================
nxpkg installs, updates, uninstalls, and rolls back standalone loadable
ELF applications (MODULE=m in an app's Makefile - see games/NXDoom or
examples/calculator for real ports) from a package repository served
over HTTP, onto local storage (an SD card on this board's config).
system/nxstore is an LVGL touchscreen front-end for nxpkg; this
document covers the repository/server side, which nxstore and the
nxpkg CLI both consume identically.
Repository layout
==================
A repository is nothing more than a directory of static files:
<repo-root>/
index.json - the package catalog
artifacts/<arch>/<chip>/<compat>/<name>/<version>/<artifact-file>
icons/<arch>/<chip>/<compat>/<name>.bin - optional, see below
index.json is a single JSON object with one "packages" array. Each
entry is one installable version of one package:
{
"packages": [
{
"name": "calc",
"version": "6",
"arch": "xtensa",
"compat": "esp32s3-touch-lcd-7",
"artifact": "artifacts/xtensa/esp32s3/esp32s3-touch-lcd-7/calc/6/calc",
"sha256": "<64 hex chars>",
"type": "elf",
"description": "Simple 4-function calculator",
"category": "Utilities",
"icon": "icons/xtensa/esp32s3/esp32s3-touch-lcd-7/calc.bin"
}
]
}
"artifact" and "icon" (both optional except "artifact") are resolved
relative to the repository's own base URL (the URL index.json itself
was fetched from, with the filename stripped) unless they're already
a full http(s):// URL - see pkg_resolve_artifact_source()/
pkg_resolve_icon_source() in pkg_repo.c. That means the whole
directory tree above can be moved, mirrored, or served from a
completely different host without editing a single path in
index.json, as long as the *relative* structure between index.json
and artifacts/icons/ stays the same.
"arch"/"compat" must match this board's CONFIG_ARCH ("xtensa") and
CONFIG_ARCH_BOARD ("esp32s3-touch-lcd-7") exactly - see
pkg_compat_check() in pkg_compat.c - so one repository can host
packages for several different boards side by side; each board only
ever installs the entries that match its own identity.
What actually serves this - any static file host works
========================================================
pkg_repo_fetch_url() (pkg_repo.c) does a plain HTTP GET with no auth,
no custom headers, and no server-side logic required - it just needs
a 2xx response. This means literally any static file server works:
nginx or Apache with a DocumentRoot pointed at the repo directory, a
one-line Python http.server, GitHub Pages, S3/R2/Cloudflare Pages, or
a NAS's built-in web server. There is nothing nxpkg-specific to
install on the server side.
For local development/testing (what this project actually used while
building and testing every package in this series on real hardware):
cd /path/to/repo-root
python3 -m http.server 8000
That's the entire "server setup." Confirm it's reachable from your
own machine first:
curl -sI http://<your-machine-ip>:8000/index.json
sha256 verification (pkg_hash_file_sha256(), checked against the
manifest's "sha256" field before an artifact is ever activated)
already protects payload integrity in transit even over plain HTTP.
Only confidentiality/availability would need HTTPS on top of this if
that matters for a given deployment - nothing in the protocol assumes
HTTP specifically, an https:// URL works exactly the same way.
Populating a repository: tools/export_pkg_repo.py
==================================================
Building the JSON by hand is error-prone (sha256 digests, exact
relative paths); tools/export_pkg_repo.py does it for you from a
built binary:
python3 tools/export_pkg_repo.py \
--arch xtensa --chip esp32s3 --compat esp32s3-touch-lcd-7 \
--package "calc:6:elf:/path/to/apps/bin/calc:Simple 4-function calculator:Utilities" \
/path/to/repo-root
--package can be repeated to export several packages/versions in one
call. The colon-separated spec is:
<name>:<version>:<elf|shared-lib>:<source>[:<description>:<category>[:<icon>]]
<source> is the built artifact on your machine (e.g. apps/bin/calc
after a normal `make`). <icon> is optional: any image Pillow can open
(PNG, JPEG, ...) - the tool resizes it to a fixed 48x48 and encodes it
into the small raw RGB565 format system/nxstore's icon rendering
expects (see nxstore_load_icon() in nxstore_main.c for the exact
on-wire layout). Re-running the script is idempotent: it loads
whatever index.json already exists in the target directory, merges in
the packages you just passed (matched on name+version+arch+compat+
type), and rewrites the file - existing unrelated entries are left
alone.
This board's FAT-formatted SD card only supports short (8.3) file/
directory names with no long-name extension - any path component
(package name, artifact leaf filename) over 8 characters before its
extension fails to install with errno 22, not a clear error message.
Keep names short (this is why this series' packages are named "calc",
"brick", "cgol" rather than "calculator", "brickmatch", "conway") -
this is a board/filesystem constraint, not an export_pkg_repo.py or
nxpkg one, and would not apply to a board with a different storage
filesystem.
Pointing the board at your repository
======================================
Two ways to get an index synced onto the board's local storage,
useful in different situations:
1. One-shot from NSH, useful for iterating on a package during
development (this is what building and testing every package in
this series actually looked like):
nxpkg sync http://<your-machine-ip>:8000/index.json
nxpkg install <name>
nxpkg update <name> # re-check for/install a newer version
nxpkg list # show what's installed, current vs previous
nxpkg rollback <name> # swap back to the previous version
nxpkg remove <name>
2. Automatically at boot, via system/nxstore: the repo URL nxstore
syncs from at startup is currently a single hardcoded string in
board_nxstore_autostart() (boards/xtensa/esp32s3/
esp32s3-touch-lcd-7/src/esp32s3_bringup.c) - change that one line
to point at a different host, or a public one, and every boot
re-syncs the catalog automatically (falling back to whatever was
last synced to local storage if the network/server is unreachable
at boot - nxstore never blocks the app list on a failed sync).
Development-network note: if your development machine's IP address
changes (switching Wi-Fi networks, a new DHCP lease, a hotspot),
re-run `nxpkg sync` with the new address - the board and the machine
serving the repository just need to be able to reach each other over
IP; nothing else about the setup changes.
Concurrency note
=================
nxpkg protects a single package's own install/update/rollback against
running twice at once (a per-package lock file), and separately
protects the shared installed-packages database against being
corrupted by two *different* packages' installs racing each other
(see pkg_install_acquire_installed_lock() in pkg_install.c) - so it is
safe to have, for example, system/nxstore's own background install
worker and a directly-invoked `nxpkg install` running at the same
time for different packages. Two operations on the *same* package
name at the same time will have the second one fail with -EBUSY
(surfaced as "package has an install/update in progress") rather than
either one silently clobbering the other's work.

154
system/nxpkg/pkg.h
View file

@ -27,30 +27,150 @@
* Included Files
****************************************************************************/
#include <nuttx/config.h>
#include <limits.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <nuttx/kmalloc.h>
/****************************************************************************
* Pre-processor Definitions
****************************************************************************/
#define PKG_REPO_DIR "/etc/nxpkg"
#define PKG_REPO_INDEX "/etc/nxpkg/index.json"
#define PKG_REPO_INSTALLED "/var/lib/nxpkg/installed.json"
#define PKG_STORE_DIR "/var/lib/nxpkg/pkgs"
#define PKG_TMP_DIR "/var/cache/nxpkg"
#define PKG_TMP_PKG_DIR "/var/cache/nxpkg/pkg"
#define PKG_ROOT_DIR CONFIG_SYSTEM_NXPKG_ROOT
#define PKG_REPO_DIR PKG_ROOT_DIR
#define PKG_REPO_INDEX PKG_ROOT_DIR "/index.jsn"
#define PKG_REPO_SOURCE PKG_ROOT_DIR "/repo.url"
#define PKG_REPO_INSTALLED PKG_ROOT_DIR "/instpkg.jsn"
#define PKG_STORE_DIR PKG_ROOT_DIR "/pkgs"
#define PKG_TMP_DIR PKG_ROOT_DIR "/tmp"
#define PKG_TMP_PKG_DIR PKG_ROOT_DIR "/tmp/pkg"
#define PKG_NAME_MAX 63
#define PKG_VERSION_MAX 31
#define PKG_ARCH_MAX 31
#define PKG_COMPAT_MAX 63
#define PKG_DESCRIPTION_MAX 127
#define PKG_CATEGORY_MAX 31
#define PKG_HASH_HEX_LEN 64
#define PKG_INDEX_MAX 32
/* Each manifest slot is ~1.7KB (dominated by PKG_LAUNCH_ARGS_MAX slots).
* This used to be 6 to fit inside a static internal-DRAM array, but the
* catalog outgrew that once it held every real GSoC package (calc, the
* four games, NXDoom, and the original examples) - system/nxstore/
* nxstore_main.c now heap-allocates its struct pkg_index_s g_index via
* pkg_zalloc(), which draws from the PSRAM-backed user heap on this board
* (CONFIG_ESP32S3_SPIRAM_USER_HEAP) instead of internal DRAM, so this can
* grow without the same pressure. Still not unbounded: keep it sized for
* "a real catalog", not arbitrary attacker-controlled growth.
*/
#define PKG_INDEX_MAX 16
#define PKG_INSTALLED_MAX 16
#define PKG_INSTALLED_VERSIONS_MAX 8
#define PKG_LAUNCH_ARGS_MAX 8
#define PKG_LAUNCH_ARG_MAX 127
/* Caps against a malicious/compromised HTTP server: without these, an
* oversized response can exhaust SD-card space (downloads) or force an
* unbounded single heap allocation sized directly off attacker-controlled
* content (pkg_store_read_text). Text/metadata files (index.jsn,
* instpkg.jsn) are always small; artifact downloads cover the largest
* real payloads seen in practice (a multi-MB WAD, a ~1MB game ELF) with
* generous headroom.
*/
#define PKG_TEXT_MAX_SIZE (256 * 1024)
#define PKG_DOWNLOAD_MAX_SIZE (32 * 1024 * 1024)
/* nxpkg is a one-shot CLI, not a daemon, so a lock file older than this
* cannot belong to a still-running install under normal use (even a full
* multi-MB artifact over a slow link finishes well within this window) -
* it can only be left over from a process that was killed or a device
* that lost power mid-install. Reclaiming it is what makes install/
* update/rollback usable again after the crash/power-loss scenarios this
* target is prone to, instead of failing with EBUSY forever.
*/
#define PKG_LOCK_STALE_SECONDS (600)
static inline void *pkg_malloc(size_t size)
{
void *ptr = malloc(size);
if (ptr == NULL)
{
ptr = kmm_malloc(size);
}
return ptr;
}
static inline void *pkg_zalloc(size_t size)
{
void *ptr = calloc(1, size);
if (ptr == NULL)
{
ptr = kmm_zalloc(size);
}
return ptr;
}
static inline void *pkg_realloc(void *ptr, size_t size)
{
if (ptr == NULL)
{
return pkg_malloc(size);
}
if (size == 0)
{
if (kmm_heapmember(ptr))
{
kmm_free(ptr);
}
else
{
free(ptr);
}
return NULL;
}
if (kmm_heapmember(ptr))
{
return kmm_realloc(ptr, size);
}
return realloc(ptr, size);
}
static inline void pkg_free(void *ptr)
{
if (ptr == NULL)
{
return;
}
if (kmm_heapmember(ptr))
{
kmm_free(ptr);
}
else
{
free(ptr);
}
}
static inline FAR char *pkg_path_alloc(void)
{
return pkg_malloc(PATH_MAX);
}
/****************************************************************************
* Public Types
@ -83,7 +203,12 @@ struct pkg_manifest_s
char compat[PKG_COMPAT_MAX + 1];
char artifact[PATH_MAX];
char sha256[PKG_HASH_HEX_LEN + 1];
char launch_args[PKG_LAUNCH_ARGS_MAX][PKG_LAUNCH_ARG_MAX + 1];
char description[PKG_DESCRIPTION_MAX + 1];
char category[PKG_CATEGORY_MAX + 1];
char icon[PATH_MAX];
enum pkg_payload_type_e type;
size_t launch_argc;
};
struct pkg_index_s
@ -124,6 +249,7 @@ int pkg_store_ensure_package_root(FAR const char *name);
int pkg_store_ensure_version_dir(FAR const char *name,
FAR const char *version);
int pkg_store_format_index_path(FAR char *buffer, size_t size);
int pkg_store_format_repo_source_path(FAR char *buffer, size_t size);
int pkg_store_format_installed_path(FAR char *buffer, size_t size);
int pkg_store_format_package_root(FAR char *buffer, size_t size,
FAR const char *name);
@ -152,6 +278,8 @@ int pkg_store_read_text(FAR const char *path, FAR char **buffer);
int pkg_store_write_text_atomic(FAR const char *path, FAR const char *text);
int pkg_store_copy_file(FAR const char *src, FAR const char *dest);
int pkg_store_remove_file(FAR const char *path);
int pkg_store_remove_version_dir(FAR const char *name,
FAR const char *version);
const char *pkg_runtime_arch(void);
const char *pkg_runtime_compat(void);
@ -160,6 +288,8 @@ int pkg_compat_check(FAR const struct pkg_manifest_s *manifest);
int pkg_hash_file_sha256(FAR const char *path,
FAR char digest[PKG_HASH_HEX_LEN + 1]);
int pkg_metadata_load_index_path(FAR const char *path,
FAR struct pkg_index_s *index);
int pkg_metadata_load_index(FAR struct pkg_index_s *index);
FAR const struct pkg_manifest_s *
pkg_metadata_find_latest(FAR const struct pkg_index_s *index,
@ -178,7 +308,17 @@ const char *pkg_txn_state_str(enum pkg_txn_state_e state);
int pkg_txn_write_state(FAR const char *name, enum pkg_txn_state_e state);
int pkg_txn_clear_state(FAR const char *name);
bool pkg_source_is_url(FAR const char *source);
int pkg_resolve_artifact_source(FAR char *buffer, size_t size,
FAR const struct pkg_manifest_s *manifest);
int pkg_resolve_icon_source(FAR char *buffer, size_t size,
FAR const struct pkg_manifest_s *manifest);
int pkg_acquire_source(FAR const char *source, FAR const char *dest);
int pkg_sync(FAR const char *source);
int pkg_install(FAR const char *name);
int pkg_uninstall(FAR const char *name);
int pkg_rollback(FAR const char *name);
int pkg_available(FAR FILE *stream);
int pkg_list(FAR FILE *stream);
void pkg_error(FAR const char *fmt, ...);

9
system/nxpkg/pkg_compat.c
View file

@ -40,7 +40,16 @@ const char *pkg_runtime_arch(void)
const char *pkg_runtime_compat(void)
{
#ifdef CONFIG_ARCH_BOARD
return CONFIG_ARCH_BOARD;
#elif defined(CONFIG_ARCH_BOARD_CUSTOM_NAME)
if (CONFIG_ARCH_BOARD_CUSTOM_NAME[0] != '\0')
{
return CONFIG_ARCH_BOARD_CUSTOM_NAME;
}
#endif
return "";
}
int pkg_compat_check(FAR const struct pkg_manifest_s *manifest)

730
system/nxpkg/pkg_install.c
View file

File diff suppressed because it is too large Load diff

24
system/nxpkg/pkg_log.c
View file

@ -26,6 +26,8 @@
#include <stdarg.h>
#include <stdio.h>
#include <string.h>
#include <syslog.h>
#include "pkg.h"
@ -33,13 +35,19 @@
* Private Functions
****************************************************************************/
static void pkg_vlog(FAR FILE *stream, FAR const char *level,
FAR const char *fmt, va_list ap)
static void pkg_vlog(FAR const char *level, FAR const char *fmt, va_list ap)
{
fprintf(stream, "nxpkg: %s: ", level);
vfprintf(stream, fmt, ap);
fputc('\n', stream);
fflush(stream);
char message[256];
int ret;
ret = vsnprintf(message, sizeof(message), fmt, ap);
if (ret < 0)
{
return;
}
syslog(strcmp(level, "error") == 0 ? LOG_ERR : LOG_INFO,
"nxpkg: %s: %s", level, message);
}
/****************************************************************************
@ -51,7 +59,7 @@ void pkg_error(FAR const char *fmt, ...)
va_list ap;
va_start(ap, fmt);
pkg_vlog(stderr, "error", fmt, ap);
pkg_vlog("error", fmt, ap);
va_end(ap);
}
@ -60,6 +68,6 @@ void pkg_info(FAR const char *fmt, ...)
va_list ap;
va_start(ap, fmt);
pkg_vlog(stdout, "info", fmt, ap);
pkg_vlog("info", fmt, ap);
va_end(ap);
}

107
system/nxpkg/pkg_main.c
View file

@ -29,8 +29,18 @@
#include <stdlib.h>
#include <string.h>
#include <netutils/cJSON.h>
#include "pkg.h"
/****************************************************************************
* Pre-processor Definitions
****************************************************************************/
#define PKG_USAGE \
"Usage: %s <install|update|remove|rollback|list|available|sync|help> " \
"[args]\n"
/****************************************************************************
* Public Functions
****************************************************************************/
@ -38,12 +48,15 @@
int main(int argc, FAR char *argv[])
{
FAR const char *cmd;
cJSON_Hooks hooks;
hooks.malloc_fn = malloc;
hooks.free_fn = free;
cJSON_InitHooks(&hooks);
if (argc < 2)
{
fprintf(stderr,
"Usage: %s <install|update|list|rollback|help> [args]\n",
argv[0]);
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
@ -52,9 +65,7 @@ int main(int argc, FAR char *argv[])
if (strcmp(cmd, "help") == 0 || strcmp(cmd, "--help") == 0 ||
strcmp(cmd, "-h") == 0)
{
fprintf(stdout,
"Usage: %s <install|update|list|rollback|help> [args]\n",
argv[0]);
fprintf(stdout, PKG_USAGE, argv[0]);
return EXIT_SUCCESS;
}
@ -63,19 +74,59 @@ int main(int argc, FAR char *argv[])
if (argc != 3)
{
pkg_error("install expects exactly one package name");
fprintf(stderr,
"Usage: %s <install|update|list|rollback|help> [args]\n",
argv[0]);
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
return pkg_install(argv[2]);
/* pkg_install() returns a real negative errno on the meaningful
* pipeline failures (nxstore uses that directly), not just
* EXIT_SUCCESS/EXIT_FAILURE - normalize to a plain 0/1 shell exit
* status here.
*/
return pkg_install(argv[2]) == 0 ? EXIT_SUCCESS : EXIT_FAILURE;
}
/* "update" is a package name resolving to whatever version is latest
* in the local index - pkg_install() already handles the "already
* installed at a different version" transition transparently via
* pkg_install_update_installed(), so no separate code path is needed.
*/
if (strcmp(cmd, "update") == 0)
{
pkg_error("'update' is not implemented yet in the current unit");
return EXIT_FAILURE;
if (argc != 3)
{
pkg_error("update expects exactly one package name");
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
return pkg_install(argv[2]) == 0 ? EXIT_SUCCESS : EXIT_FAILURE;
}
if (strcmp(cmd, "remove") == 0 || strcmp(cmd, "uninstall") == 0)
{
if (argc != 3)
{
pkg_error("remove expects exactly one package name");
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
return pkg_uninstall(argv[2]);
}
if (strcmp(cmd, "rollback") == 0)
{
if (argc != 3)
{
pkg_error("rollback expects exactly one package name");
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
return pkg_rollback(argv[2]);
}
if (strcmp(cmd, "list") == 0)
@ -83,24 +134,38 @@ int main(int argc, FAR char *argv[])
if (argc != 2)
{
pkg_error("list does not take additional arguments");
fprintf(stderr,
"Usage: %s <install|update|list|rollback|help> [args]\n",
argv[0]);
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
return pkg_list(stdout);
}
if (strcmp(cmd, "rollback") == 0)
if (strcmp(cmd, "available") == 0)
{
pkg_error("'rollback' is not implemented yet in the current unit");
return EXIT_FAILURE;
if (argc != 2)
{
pkg_error("available does not take additional arguments");
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
return pkg_available(stdout);
}
if (strcmp(cmd, "sync") == 0)
{
if (argc != 3)
{
pkg_error("sync expects exactly one index source");
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}
return pkg_sync(argv[2]);
}
fprintf(stderr, "ERROR: Unknown subcommand '%s'\n", cmd);
fprintf(stderr,
"Usage: %s <install|update|list|rollback|help> [args]\n",
argv[0]);
fprintf(stderr, PKG_USAGE, argv[0]);
return EXIT_FAILURE;
}

71
system/nxpkg/pkg_manifest.c
View file

@ -74,6 +74,49 @@ static bool pkg_validate_hex(FAR const char *value)
return true;
}
/****************************************************************************
* Name: pkg_validate_path_component
*
* Description:
* Reject any value that could escape the intended directory when spliced
* into a filesystem path (pkg_store.c's PKG_STORE_DIR "/%s/%s/..."
* formatters). This is required for "name" and "version" specifically,
* since both come straight from an untrusted, network-fetched
* index.json and are used unsanitized to build install paths - a
* version of "../../evil" would otherwise let a malicious index write
* or delete files outside the package store entirely.
*
****************************************************************************/
static bool pkg_validate_path_component(FAR const char *value)
{
FAR const char *p;
if (pkg_validate_required(value) < 0)
{
return false;
}
/* Reject a leading '.' outright: blocks ".", "..", and any
* "../"-prefixed traversal in one check.
*/
if (value[0] == '.')
{
return false;
}
for (p = value; *p != '\0'; p++)
{
if (*p == '/' || *p == '\\')
{
return false;
}
}
return true;
}
/****************************************************************************
* Public Functions
****************************************************************************/
@ -95,6 +138,8 @@ const char *pkg_manifest_type_str(enum pkg_payload_type_e type)
int pkg_manifest_validate(FAR const struct pkg_manifest_s *manifest)
{
size_t i;
if (manifest == NULL)
{
return -EINVAL;
@ -110,6 +155,19 @@ int pkg_manifest_validate(FAR const struct pkg_manifest_s *manifest)
return -EINVAL;
}
/* "name" and "version" get spliced unsanitized into on-disk paths
* (pkg_store.c) - they must not contain path separators or traversal
* sequences. "artifact" is validated separately in pkg_repo.c, where
* it's legitimately allowed to be a relative repo path (just not an
* absolute one or one that escapes the repo root).
*/
if (!pkg_validate_path_component(manifest->name) ||
!pkg_validate_path_component(manifest->version))
{
return -EINVAL;
}
if (strlen(manifest->sha256) != PKG_HASH_HEX_LEN)
{
return -EINVAL;
@ -126,6 +184,19 @@ int pkg_manifest_validate(FAR const struct pkg_manifest_s *manifest)
return -EINVAL;
}
if (manifest->launch_argc > PKG_LAUNCH_ARGS_MAX)
{
return -EINVAL;
}
for (i = 0; i < manifest->launch_argc; i++)
{
if (pkg_validate_required(manifest->launch_args[i]) < 0)
{
return -EINVAL;
}
}
return 0;
}

239
system/nxpkg/pkg_metadata.c
View file

@ -100,6 +100,54 @@ static FAR cJSON *pkg_metadata_packages_array(FAR cJSON *root)
return cJSON_GetObjectItemCaseSensitive(root, "packages");
}
static int pkg_metadata_parse_launch_args(
FAR cJSON *item, FAR struct pkg_manifest_s *manifest)
{
FAR cJSON *field;
FAR cJSON *arg;
size_t argc = 0;
FAR const char *value;
int ret;
field = cJSON_GetObjectItemCaseSensitive(item, "launch_args");
if (field == NULL)
{
manifest->launch_argc = 0;
return 0;
}
if (!cJSON_IsArray(field))
{
return -EINVAL;
}
cJSON_ArrayForEach(arg, field)
{
if (argc >= PKG_LAUNCH_ARGS_MAX)
{
return -E2BIG;
}
value = cJSON_GetStringValue(arg);
if (value == NULL)
{
return -EINVAL;
}
ret = pkg_copy_string(manifest->launch_args[argc],
sizeof(manifest->launch_args[argc]), value);
if (ret < 0)
{
return ret;
}
argc++;
}
manifest->launch_argc = argc;
return 0;
}
static int pkg_metadata_parse_manifest(FAR cJSON *item,
FAR struct pkg_manifest_s *manifest)
{
@ -170,6 +218,39 @@ static int pkg_metadata_parse_manifest(FAR cJSON *item,
return -EINVAL;
}
/* description/category/icon are optional and purely for UI display;
* missing fields just leave the manifest's copy empty.
*/
field = cJSON_GetObjectItemCaseSensitive(item, "description");
value = cJSON_GetStringValue(field);
if (value != NULL)
{
pkg_copy_string(manifest->description, sizeof(manifest->description),
value);
}
field = cJSON_GetObjectItemCaseSensitive(item, "category");
value = cJSON_GetStringValue(field);
if (value != NULL)
{
pkg_copy_string(manifest->category, sizeof(manifest->category),
value);
}
field = cJSON_GetObjectItemCaseSensitive(item, "icon");
value = cJSON_GetStringValue(field);
if (value != NULL)
{
pkg_copy_string(manifest->icon, sizeof(manifest->icon), value);
}
ret = pkg_metadata_parse_launch_args(item, manifest);
if (ret < 0)
{
return ret;
}
return pkg_manifest_validate(manifest);
}
@ -395,6 +476,8 @@ static FAR cJSON *pkg_metadata_manifest_to_json(
FAR const struct pkg_manifest_s *manifest)
{
FAR cJSON *root;
FAR cJSON *launch_args;
size_t i;
root = cJSON_CreateObject();
if (root == NULL)
@ -410,51 +493,55 @@ static FAR cJSON *pkg_metadata_manifest_to_json(
cJSON_AddStringToObject(root, "sha256", manifest->sha256);
cJSON_AddStringToObject(root, "type",
pkg_manifest_type_str(manifest->type));
if (manifest->description[0] != '\0')
{
cJSON_AddStringToObject(root, "description", manifest->description);
}
if (manifest->category[0] != '\0')
{
cJSON_AddStringToObject(root, "category", manifest->category);
}
if (manifest->launch_argc > 0)
{
launch_args = cJSON_AddArrayToObject(root, "launch_args");
if (launch_args == NULL)
{
cJSON_Delete(root);
return NULL;
}
for (i = 0; i < manifest->launch_argc; i++)
{
FAR cJSON *arg;
arg = cJSON_CreateString(manifest->launch_args[i]);
if (arg == NULL)
{
cJSON_Delete(root);
return NULL;
}
cJSON_AddItemToArray(launch_args, arg);
}
}
return root;
}
/****************************************************************************
* Public Functions
****************************************************************************/
int pkg_metadata_load_index(FAR struct pkg_index_s *index)
static int pkg_metadata_parse_index_text(FAR const char *text,
FAR struct pkg_index_s *index)
{
FAR cJSON *root;
FAR cJSON *packages;
FAR cJSON *item;
FAR char *text;
char path[PATH_MAX];
size_t count = 0;
size_t textlen;
int ret;
if (index == NULL)
{
return -EINVAL;
}
memset(index, 0, sizeof(*index));
ret = pkg_store_format_index_path(path, sizeof(path));
if (ret < 0)
{
return ret;
}
pkg_info("loading index from %s", path);
ret = pkg_store_read_text(path, &text);
if (ret < 0)
{
return ret;
}
textlen = strlen(text);
pkg_info("index read complete (%zu bytes)", textlen);
root = cJSON_Parse(text);
pkg_info("cJSON_Parse returned %s", root != NULL ? "success" : "failure");
free(text);
if (root == NULL)
{
return -EINVAL;
@ -471,15 +558,27 @@ int pkg_metadata_load_index(FAR struct pkg_index_s *index)
{
if (count >= PKG_INDEX_MAX)
{
cJSON_Delete(root);
return -E2BIG;
/* Keep what's already parsed rather than discarding the whole
* index: a catalog that's grown past PKG_INDEX_MAX shouldn't
* make every other package unavailable too.
*/
pkg_error("index has more than %d packages, truncating",
PKG_INDEX_MAX);
break;
}
ret = pkg_metadata_parse_manifest(item, &index->manifests[count]);
if (ret < 0)
{
cJSON_Delete(root);
return ret;
/* Skip a malformed entry instead of discarding the entire
* index: one bad/malicious package definition shouldn't make
* every other, otherwise-valid package unavailable too.
*/
pkg_error("skipping malformed package entry %zu: %d", count,
ret);
continue;
}
pkg_info("parsed manifest %s %s",
@ -493,6 +592,54 @@ int pkg_metadata_load_index(FAR struct pkg_index_s *index)
return 0;
}
/****************************************************************************
* Public Functions
****************************************************************************/
int pkg_metadata_load_index_path(FAR const char *path,
FAR struct pkg_index_s *index)
{
FAR char *text;
size_t textlen;
int ret;
if (path == NULL || index == NULL)
{
return -EINVAL;
}
memset(index, 0, sizeof(*index));
pkg_info("loading index from %s", path);
ret = pkg_store_read_text(path, &text);
if (ret < 0)
{
return ret;
}
textlen = strlen(text);
pkg_info("index read complete (%zu bytes)", textlen);
ret = pkg_metadata_parse_index_text(text, index);
pkg_free(text);
return ret;
}
int pkg_metadata_load_index(FAR struct pkg_index_s *index)
{
char path[PATH_MAX];
int ret;
ret = pkg_store_format_index_path(path, sizeof(path));
if (ret < 0)
{
return ret;
}
return pkg_metadata_load_index_path(path, index);
}
FAR const struct pkg_manifest_s *
pkg_metadata_find_latest(FAR const struct pkg_index_s *index,
FAR const char *name)
@ -573,7 +720,7 @@ int pkg_metadata_load_installed(FAR struct pkg_installed_db_s *db)
}
root = cJSON_Parse(text);
free(text);
pkg_free(text);
if (root == NULL)
{
return -EINVAL;
@ -590,15 +737,23 @@ int pkg_metadata_load_installed(FAR struct pkg_installed_db_s *db)
{
if (count >= PKG_INSTALLED_MAX)
{
cJSON_Delete(root);
return -E2BIG;
pkg_error("installed db has more than %d entries, truncating",
PKG_INSTALLED_MAX);
break;
}
ret = pkg_metadata_parse_installed_entry(item, &db->entries[count]);
if (ret < 0)
{
cJSON_Delete(root);
return ret;
/* A single corrupted entry (plausible after a crash mid-write,
* despite the atomic-write mechanism) must not make every
* other installed package look uninstalled - that would drive
* needless reinstalls for everything else.
*/
pkg_error("skipping malformed installed entry %zu: %d", count,
ret);
continue;
}
count++;

612
system/nxpkg/pkg_repo.c Normal file
View file

@ -0,0 +1,612 @@
/****************************************************************************
* apps/system/nxpkg/pkg_repo.c
*
* SPDX-License-Identifier: Apache-2.0
*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership. The
* ASF licenses this file to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the
* License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*
****************************************************************************/
/****************************************************************************
* Included Files
****************************************************************************/
#include <ctype.h>
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include <unistd.h>
#include <nuttx/config.h>
#ifdef CONFIG_NETUTILS_WEBCLIENT
# include "netutils/webclient.h"
#endif
#include "pkg.h"
/****************************************************************************
* Pre-processor Definitions
****************************************************************************/
/* Each webclient_perform() read/sink cycle costs a TCP receive plus an
* SD-card write call; at the old 512-byte size, a sub-1MB file like
* nxdoom's ~940KB ELF took ~1900 round trips and, in practice, close
* to two minutes to install - easily read as "stuck" with only a bare
* spinner for feedback. 4KB cuts that to ~230 round trips and lines
* up with typical SD card erase-block granularity, which also reduces
* write amplification. Still small enough to be a safe stack-local
* buffer against the 16KB (CLI) / 16KB (nxstore install worker) task
* stacks that call into this.
*/
#define PKG_REPO_FETCH_BUFFER_SIZE 4096
/****************************************************************************
* Private Types
****************************************************************************/
#ifdef CONFIG_NETUTILS_WEBCLIENT
struct pkg_fetch_context_s
{
int fd;
size_t total;
};
#endif
/****************************************************************************
* Private Functions
****************************************************************************/
static int pkg_repo_copy_string(FAR char *buffer, size_t size,
FAR const char *value)
{
int ret;
ret = snprintf(buffer, size, "%s", value);
if (ret < 0)
{
return ret;
}
return (size_t)ret >= size ? -ENAMETOOLONG : 0;
}
static int pkg_repo_source_base(FAR char *buffer, size_t size,
FAR const char *source)
{
FAR const char *slash;
size_t length;
slash = strrchr(source, '/');
if (slash == NULL)
{
return -EINVAL;
}
length = (size_t)(slash - source);
if (length == 0)
{
length = 1;
}
if (length >= size)
{
return -ENAMETOOLONG;
}
memcpy(buffer, source, length);
buffer[length] = '\0';
return 0;
}
/****************************************************************************
* Name: pkg_validate_artifact_relative
*
* Description:
* manifest->artifact is repo-relative content and gets spliced into a
* local filesystem path (or used to build a URL) unsanitized. Reject
* absolute paths outright - allowing them let a malicious index turn
* any local file with a known/predictable hash into an "installed"
* package, including making it executable - and reject any ".." path
* segment that would let the artifact escape the repo mirror directory.
*
****************************************************************************/
static bool pkg_validate_artifact_relative(FAR const char *value)
{
FAR const char *p;
if (value == NULL || value[0] == '\0' || value[0] == '/')
{
return false;
}
p = value;
while ((p = strstr(p, "..")) != NULL)
{
bool at_start = p == value || *(p - 1) == '/';
bool at_end = p[2] == '\0' || p[2] == '/';
if (at_start && at_end)
{
return false;
}
p++;
}
return true;
}
static int pkg_repo_read_source(FAR char *buffer, size_t size)
{
char path[PATH_MAX];
FAR char *text;
size_t length;
int ret;
ret = pkg_store_format_repo_source_path(path, sizeof(path));
if (ret < 0)
{
return ret;
}
ret = pkg_store_read_text(path, &text);
if (ret < 0)
{
return ret;
}
length = strlen(text);
while (length > 0 && isspace((unsigned char)text[length - 1]))
{
text[--length] = '\0';
}
ret = pkg_repo_copy_string(buffer, size, text);
pkg_free(text);
return ret;
}
#ifdef CONFIG_NETUTILS_WEBCLIENT
static int pkg_repo_sink(FAR char **buffer, int offset, int datend,
FAR int *buflen, FAR void *arg)
{
FAR struct pkg_fetch_context_s *ctx;
size_t remaining;
FAR char *cursor;
UNUSED(buffer);
UNUSED(buflen);
ctx = arg;
cursor = &(*buffer)[offset];
remaining = (size_t)(datend - offset);
/* Cap total downloaded bytes: an unbounded/malicious response could
* otherwise exhaust all SD-card space. Checked before writing more so
* the on-disk file never exceeds the cap even mid-chunk.
*/
if (remaining > 0 &&
(ctx->total > PKG_DOWNLOAD_MAX_SIZE ||
remaining > PKG_DOWNLOAD_MAX_SIZE - ctx->total))
{
return -EFBIG;
}
ctx->total += remaining;
while (remaining > 0)
{
ssize_t nwritten;
nwritten = write(ctx->fd, cursor, remaining);
if (nwritten < 0)
{
if (errno == EINTR)
{
continue;
}
return -errno;
}
cursor += nwritten;
remaining -= (size_t)nwritten;
}
return 0;
}
static int pkg_repo_fetch_url(FAR const char *url, FAR const char *dest)
{
struct pkg_fetch_context_s fetch;
struct webclient_context client;
char reason[64];
char buffer[PKG_REPO_FETCH_BUFFER_SIZE];
int ret;
fetch.fd = open(dest, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fetch.fd < 0)
{
return -errno;
}
fetch.total = 0;
webclient_set_defaults(&client);
client.method = "GET";
client.url = url;
client.buffer = buffer;
client.buflen = sizeof(buffer);
client.sink_callback = pkg_repo_sink;
client.sink_callback_arg = &fetch;
client.http_reason = reason;
client.http_reason_len = sizeof(reason);
ret = webclient_perform(&client);
if (ret < 0)
{
close(fetch.fd);
unlink(dest);
return ret;
}
if (client.http_status / 100 != 2)
{
close(fetch.fd);
unlink(dest);
return -EPROTO;
}
if (close(fetch.fd) < 0)
{
unlink(dest);
return -errno;
}
return 0;
}
#endif
static int pkg_resolve_relative_source(FAR char *buffer, size_t size,
FAR const char *relative)
{
char source[PATH_MAX];
char base[PATH_MAX];
int ret;
if (buffer == NULL || relative == NULL || relative[0] == '\0')
{
return -EINVAL;
}
if (pkg_source_is_url(relative))
{
return pkg_repo_copy_string(buffer, size, relative);
}
if (!pkg_validate_artifact_relative(relative))
{
return -EINVAL;
}
ret = pkg_repo_read_source(source, sizeof(source));
if (ret >= 0)
{
ret = pkg_repo_source_base(base, sizeof(base), source);
if (ret < 0)
{
return ret;
}
ret = snprintf(buffer, size, "%s/%s", base, relative);
if (ret < 0)
{
return ret;
}
return (size_t)ret >= size ? -ENAMETOOLONG : 0;
}
ret = snprintf(buffer, size, "%s/%s", PKG_REPO_DIR, relative);
if (ret < 0)
{
return ret;
}
return (size_t)ret >= size ? -ENAMETOOLONG : 0;
}
/****************************************************************************
* Public Functions
****************************************************************************/
bool pkg_source_is_url(FAR const char *source)
{
if (source == NULL)
{
return false;
}
return strncasecmp(source, "http://", 7) == 0 ||
strncasecmp(source, "https://", 8) == 0;
}
int pkg_resolve_artifact_source(FAR char *buffer, size_t size,
FAR const struct pkg_manifest_s *manifest)
{
if (manifest == NULL)
{
return -EINVAL;
}
return pkg_resolve_relative_source(buffer, size, manifest->artifact);
}
int pkg_resolve_icon_source(FAR char *buffer, size_t size,
FAR const struct pkg_manifest_s *manifest)
{
if (manifest == NULL)
{
return -EINVAL;
}
return pkg_resolve_relative_source(buffer, size, manifest->icon);
}
int pkg_acquire_source(FAR const char *source, FAR const char *dest)
{
if (source == NULL || dest == NULL)
{
return -EINVAL;
}
if (pkg_source_is_url(source))
{
#ifdef CONFIG_NETUTILS_WEBCLIENT
return pkg_repo_fetch_url(source, dest);
#else
return -ENOSYS;
#endif
}
return pkg_store_copy_file(source, dest);
}
int pkg_sync(FAR const char *source)
{
FAR struct pkg_index_s *index;
FAR char *text = NULL;
FAR char *tmp;
FAR char *index_path;
FAR char *source_path;
int ret;
if (source == NULL || source[0] == '\0')
{
pkg_error("sync requires a non-empty index source");
return EXIT_FAILURE;
}
index = pkg_zalloc(sizeof(*index));
tmp = pkg_path_alloc();
index_path = pkg_path_alloc();
source_path = pkg_path_alloc();
if (index == NULL || tmp == NULL || index_path == NULL ||
source_path == NULL)
{
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
pkg_error("unable to allocate index metadata buffer");
return EXIT_FAILURE;
}
ret = pkg_store_prepare_layout();
if (ret < 0)
{
pkg_error("unable to prepare package layout: %d", ret);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
return EXIT_FAILURE;
}
ret = snprintf(tmp, PATH_MAX, "%s/idxsync.jsn", PKG_TMP_DIR);
if (ret < 0 || (size_t)ret >= PATH_MAX)
{
pkg_error("temporary sync path is too long");
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
return EXIT_FAILURE;
}
ret = pkg_acquire_source(source, tmp);
if (ret < 0)
{
pkg_error("unable to fetch index source '%s': %d", source, ret);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
return EXIT_FAILURE;
}
ret = pkg_metadata_load_index_path(tmp, index);
if (ret < 0)
{
pkg_store_remove_file(tmp);
pkg_error("downloaded index is invalid: %d", ret);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
return EXIT_FAILURE;
}
ret = pkg_store_read_text(tmp, &text);
if (ret < 0)
{
pkg_store_remove_file(tmp);
pkg_error("unable to read fetched index: %d", ret);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
return EXIT_FAILURE;
}
ret = pkg_store_format_index_path(index_path, PATH_MAX);
if (ret < 0)
{
pkg_store_remove_file(tmp);
pkg_free(text);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
pkg_error("unable to resolve local index path: %d", ret);
return EXIT_FAILURE;
}
ret = pkg_store_write_text_atomic(index_path, text);
if (ret < 0)
{
pkg_store_remove_file(tmp);
pkg_free(text);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
pkg_error("unable to write local index: %d", ret);
return EXIT_FAILURE;
}
ret = pkg_store_format_repo_source_path(source_path, PATH_MAX);
if (ret < 0)
{
pkg_store_remove_file(tmp);
pkg_free(text);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
pkg_error("unable to resolve repository source path: %d", ret);
return EXIT_FAILURE;
}
ret = pkg_store_write_text_atomic(source_path, source);
if (ret < 0)
{
pkg_store_remove_file(tmp);
pkg_free(text);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
pkg_error("unable to save repository source: %d", ret);
return EXIT_FAILURE;
}
pkg_store_remove_file(tmp);
pkg_free(text);
pkg_free(index);
pkg_free(tmp);
pkg_free(index_path);
pkg_free(source_path);
pkg_info("synced package index from %s", source);
return EXIT_SUCCESS;
}
int pkg_available(FAR FILE *stream)
{
FAR struct pkg_index_s *index;
FAR const char *arch;
FAR const char *compat;
size_t i;
int ret;
if (stream == NULL)
{
return EXIT_FAILURE;
}
index = pkg_zalloc(sizeof(*index));
if (index == NULL)
{
pkg_error("unable to allocate index metadata buffer");
return EXIT_FAILURE;
}
ret = pkg_store_prepare_layout();
if (ret < 0)
{
pkg_free(index);
pkg_error("unable to prepare package layout: %d", ret);
return EXIT_FAILURE;
}
ret = pkg_metadata_load_index(index);
if (ret < 0)
{
pkg_free(index);
pkg_error("unable to load package index: %d", ret);
return EXIT_FAILURE;
}
arch = pkg_runtime_arch();
compat = pkg_runtime_compat();
for (i = 0; i < index->count; i++)
{
FAR const struct pkg_manifest_s *manifest = &index->manifests[i];
FAR const struct pkg_manifest_s *latest;
if (strcmp(manifest->arch, arch) != 0 ||
strcmp(manifest->compat, compat) != 0)
{
continue;
}
latest = pkg_metadata_find_latest(index, manifest->name);
if (latest != manifest)
{
continue;
}
fprintf(stream,
"%s version=%s type=%s arch=%s compat=%s artifact=%s\n",
manifest->name,
manifest->version,
pkg_manifest_type_str(manifest->type),
manifest->arch,
manifest->compat,
manifest->artifact);
}
pkg_free(index);
return EXIT_SUCCESS;
}

318
system/nxpkg/pkg_store.c
View file

@ -24,6 +24,7 @@
* Included Files
****************************************************************************/
#include <dirent.h>
#include <errno.h>
#include <fcntl.h>
#include <stdlib.h>
@ -228,6 +229,11 @@ int pkg_store_format_index_path(FAR char *buffer, size_t size)
return pkg_store_format(buffer, size, "%s", PKG_REPO_INDEX, "");
}
int pkg_store_format_repo_source_path(FAR char *buffer, size_t size)
{
return pkg_store_format(buffer, size, "%s", PKG_REPO_SOURCE, "");
}
int pkg_store_format_installed_path(FAR char *buffer, size_t size)
{
return pkg_store_format(buffer, size, "%s", PKG_REPO_INSTALLED, "");
@ -266,21 +272,52 @@ int pkg_store_format_previous_path(FAR char *buffer, size_t size,
int pkg_store_format_txn_path(FAR char *buffer, size_t size,
FAR const char *name)
{
return pkg_store_format(buffer, size, PKG_STORE_DIR "/%s/.txn", name, "");
return pkg_store_format(buffer, size, PKG_STORE_DIR "/%s/txn.tx", name,
"");
}
int pkg_store_format_lock_path(FAR char *buffer, size_t size,
FAR const char *name)
{
return pkg_store_format(buffer, size, PKG_STORE_DIR "/%s/.lock", name, "");
return pkg_store_format(buffer, size, PKG_STORE_DIR "/%s/lock.lk", name,
"");
}
int pkg_store_format_download_path(FAR char *buffer, size_t size,
FAR const char *name,
FAR const char *version)
{
return pkg_store_format(buffer, size, PKG_TMP_PKG_DIR "/%s-%s.npkg", name,
version);
int ret;
UNUSED(name);
UNUSED(version);
/* This used to be "PKG_TMP_PKG_DIR/name-version.pkg", which breaks on
* this SD card's short-name-only FAT mount as soon as name+version
* exceeds the 8.3 8-character base-name limit - e.g. "nxdoom-1" (8
* chars) fits and installs fine, but "nxdoom-10" or "nxdoom-9.1" (9+
* chars) fails the open(O_CREAT) in pkg_repo_fetch_url() with
* -EINVAL, surfacing as "acquire source failed: -22" for any
* multi-character version - independent of name/version length here,
* unlike pkg_store_make_tmp_path()'s already-FAT-safe scheme. The
* pid is small, bounded, and unique per concurrently running install
* (each `nxpkg install` is its own process with its own per-name
* lock), so it can't collide the way a single fixed name would if
* two different packages were being installed at once.
*/
ret = snprintf(buffer, size, PKG_TMP_PKG_DIR "/dl%d.pkg", (int)getpid());
if (ret < 0)
{
return ret;
}
if ((size_t)ret >= size)
{
return -ENAMETOOLONG;
}
return 0;
}
int pkg_store_format_payload_path(FAR char *buffer, size_t size,
@ -311,16 +348,18 @@ int pkg_store_format_manifest_path(FAR char *buffer, size_t size,
FAR const char *name,
FAR const char *version)
{
return pkg_store_format(buffer, size, PKG_STORE_DIR "/%s/%s/manifest.json",
return pkg_store_format(buffer, size, PKG_STORE_DIR "/%s/%s/manifest.jsn",
name, version);
}
int pkg_store_read_text(FAR const char *path, FAR char **buffer)
{
FAR FILE *stream;
FAR char *data;
long length;
struct stat st;
size_t length;
size_t nread;
size_t total;
int fd;
if (buffer == NULL)
{
@ -329,72 +368,169 @@ int pkg_store_read_text(FAR const char *path, FAR char **buffer)
*buffer = NULL;
stream = fopen(path, "rb");
if (stream == NULL)
fd = open(path, O_RDONLY);
if (fd < 0)
{
return errno == ENOENT ? -ENOENT : -errno;
}
if (fseek(stream, 0, SEEK_END) < 0)
if (fstat(fd, &st) < 0)
{
fclose(stream);
close(fd);
return -errno;
}
length = ftell(stream);
if (length < 0)
if (!S_ISREG(st.st_mode))
{
fclose(stream);
return -errno;
close(fd);
return -EINVAL;
}
if (fseek(stream, 0, SEEK_SET) < 0)
/* Reject anything unreasonably large before the size is trusted for an
* allocation: guards both against a malicious/oversized text file (this
* path is used for the network-fetched index.jsn) and against
* "length + 1" wrapping if st_size were ever attacker-influenced up to
* SIZE_MAX.
*/
if (st.st_size < 0 || st.st_size > (off_t)PKG_TEXT_MAX_SIZE)
{
fclose(stream);
return -errno;
close(fd);
return -EFBIG;
}
data = malloc((size_t)length + 1);
length = (size_t)st.st_size;
data = pkg_malloc((size_t)length + 1);
if (data == NULL)
{
fclose(stream);
close(fd);
return -ENOMEM;
}
nread = fread(data, 1, (size_t)length, stream);
if (nread != (size_t)length)
total = 0;
while (total < length)
{
int err = ferror(stream);
ssize_t ret;
fclose(stream);
free(data);
return err ? -EIO : -EINVAL;
ret = read(fd, data + total, length - total);
if (ret < 0)
{
if (errno == EINTR)
{
continue;
}
close(fd);
pkg_free(data);
return -errno;
}
if (ret == 0)
{
break;
}
total += (size_t)ret;
}
fclose(stream);
nread = total;
close(fd);
if (nread != length)
{
pkg_free(data);
return -EINVAL;
}
data[length] = '\0';
*buffer = data;
return 0;
}
int pkg_store_write_text_atomic(FAR const char *path, FAR const char *text)
#ifndef CONFIG_PSEUDOFS_FILE
/****************************************************************************
* Name: pkg_store_make_tmp_path
*
* Description:
* Derive a staging path for an atomic write/copy to "path", under a
* short-name-compatible extension instead of appending ".tmp" (which
* would produce a second '.' in the final path component and break on
* FAT filesystems without long file name support).
*
****************************************************************************/
static int pkg_store_make_tmp_path(FAR char *tmp, size_t size,
FAR const char *path)
{
char tmp[PATH_MAX];
int fd;
FAR char *dot;
FAR char *slash;
int ret;
ret = snprintf(tmp, sizeof(tmp), "%s.tmp", path);
ret = snprintf(tmp, size, "%s", path);
if (ret < 0)
{
return ret;
}
if ((size_t)ret >= sizeof(tmp))
if ((size_t)ret >= size)
{
return -ENAMETOOLONG;
}
slash = strrchr(tmp, '/');
dot = strrchr(slash != NULL ? slash : tmp, '.');
if (dot != NULL)
{
*dot = '\0';
}
if (strlcat(tmp, ".tm", size) >= size)
{
return -ENAMETOOLONG;
}
return 0;
}
#endif
int pkg_store_write_text_atomic(FAR const char *path, FAR const char *text)
{
#ifdef CONFIG_PSEUDOFS_FILE
int fd;
int ret;
fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd < 0)
{
return -errno;
}
ret = pkg_store_write_all(fd, text, strlen(text));
if (ret < 0)
{
close(fd);
unlink(path);
return ret;
}
if (close(fd) < 0)
{
unlink(path);
return -errno;
}
return 0;
#else
char tmp[PATH_MAX];
int fd;
int ret;
ret = pkg_store_make_tmp_path(tmp, sizeof(tmp), path);
if (ret < 0)
{
return ret;
}
fd = open(tmp, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd < 0)
{
@ -409,6 +545,22 @@ int pkg_store_write_text_atomic(FAR const char *path, FAR const char *text)
return ret;
}
/* Force the write through to disk before renaming. nxpkg writes
* several small, unrelated files back-to-back during install (per-
* package txn state, then the shared installed-packages database);
* without an explicit sync here, the FAT driver's single shared
* sector cache can still hold a not-yet-committed buffer for this
* file when the very next atomic write starts touching a different
* file, corrupting one or both.
*/
if (fsync(fd) < 0)
{
close(fd);
unlink(tmp);
return -errno;
}
if (close(fd) < 0)
{
unlink(tmp);
@ -422,6 +574,7 @@ int pkg_store_write_text_atomic(FAR const char *path, FAR const char *text)
}
return 0;
#endif
}
int pkg_store_copy_file(FAR const char *src, FAR const char *dest)
@ -430,6 +583,20 @@ int pkg_store_copy_file(FAR const char *src, FAR const char *dest)
int outfd;
int ret;
char buffer[512];
#ifndef CONFIG_PSEUDOFS_FILE
char tmp[PATH_MAX];
FAR const char *outpath;
ret = pkg_store_make_tmp_path(tmp, sizeof(tmp), dest);
if (ret < 0)
{
return ret;
}
outpath = tmp;
#else
FAR const char *outpath = dest;
#endif
infd = open(src, O_RDONLY);
if (infd < 0)
@ -437,7 +604,7 @@ int pkg_store_copy_file(FAR const char *src, FAR const char *dest)
return -errno;
}
outfd = open(dest, O_WRONLY | O_CREAT | O_TRUNC, 0644);
outfd = open(outpath, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (outfd < 0)
{
ret = -errno;
@ -475,18 +642,43 @@ int pkg_store_copy_file(FAR const char *src, FAR const char *dest)
close(infd);
#ifndef CONFIG_PSEUDOFS_FILE
/* Force the payload through to disk before renaming - this is the
* largest write in the whole install pipeline (WAD/game-ELF-sized
* payloads), so a hard power-loss here is the scenario the atomic
* temp+rename is specifically protecting against.
*/
if (fsync(outfd) < 0)
{
ret = -errno;
close(outfd);
unlink(outpath);
return ret;
}
#endif
if (close(outfd) < 0)
{
unlink(dest);
unlink(outpath);
return -errno;
}
#ifndef CONFIG_PSEUDOFS_FILE
if (rename(outpath, dest) < 0)
{
ret = -errno;
unlink(outpath);
return ret;
}
#endif
return 0;
errout:
close(infd);
close(outfd);
unlink(dest);
unlink(outpath);
return ret;
}
@ -499,3 +691,59 @@ int pkg_store_remove_file(FAR const char *path)
return 0;
}
int pkg_store_remove_version_dir(FAR const char *name,
FAR const char *version)
{
char path[PATH_MAX];
char entry_path[PATH_MAX];
FAR DIR *dir;
FAR struct dirent *ent;
int ret;
/* Generic directory-content removal (rather than unlinking the payload
* and manifest.jsn by their known names) so this same helper works both
* to reclaim a partially staged version directory after a failed
* install (pkg_install.c) and to prune/remove a fully-installed
* version, without needing to already know that version's artifact
* filename. Best-effort throughout: this runs from error/cleanup
* paths where a still-failing removal shouldn't itself abort the
* caller.
*/
ret = pkg_store_format_version_path(path, sizeof(path), name, version);
if (ret < 0)
{
return ret;
}
dir = opendir(path);
if (dir == NULL)
{
return errno == ENOENT ? 0 : -errno;
}
while ((ent = readdir(dir)) != NULL)
{
if (strcmp(ent->d_name, ".") == 0 || strcmp(ent->d_name, "..") == 0)
{
continue;
}
ret = snprintf(entry_path, sizeof(entry_path), "%s/%s", path,
ent->d_name);
if (ret > 0 && (size_t)ret < sizeof(entry_path))
{
unlink(entry_path);
}
}
closedir(dir);
if (rmdir(path) < 0)
{
return errno == ENOENT ? 0 : -errno;
}
return 0;
}