From bfb2728f8d6f7885dc75816c2a757e90c259f1ce Mon Sep 17 00:00:00 2001 From: forbes Date: Sat, 7 Feb 2026 09:34:41 -0600 Subject: [PATCH 1/2] docs: update OVERVIEW.md and CI_CD.md to match current repo state - Fix stale submodule URLs (gitea.kindred.internal -> git.kindred-systems.com) - Update silo submodule name and commit, add note about repo split - Document actual release mechanism (curl API, not release-action) - Mark macOS/Windows builds as disabled in platform matrix - Update ccache docs: date-based key rotation, rattler-build env forwarding - Document disk cleanup steps and runner cleanup daemon - Update appimagetool extraction note (FUSE unavailable in containers) --- docs/CI_CD.md | 75 +++++++++++++++++++++++++++++++++++------------- docs/OVERVIEW.md | 12 ++++---- 2 files changed, 62 insertions(+), 25 deletions(-) diff --git a/docs/CI_CD.md b/docs/CI_CD.md index ba4f89febf..5182f0b230 100644 --- a/docs/CI_CD.md +++ b/docs/CI_CD.md @@ -7,7 +7,7 @@ Kindred Create uses Gitea Actions for continuous integration and release builds. | Workflow | Trigger | Purpose | Artifacts | |----------|---------|---------|-----------| | `build.yml` | Push to `main`, pull requests | Build + test | Linux tarball | -| `release.yml` | Tags matching `v*` | Multi-platform release | AppImage, .deb, .dmg, .exe, .7z | +| `release.yml` | Tags matching `v*` or `latest` | Release build | AppImage, .deb | All builds run on public runners in dockerized mode. No host-mode or internal infrastructure is required. @@ -34,14 +34,16 @@ Runs on every push to `main` and on pull requests. Builds the project in an Ubun ### Caching -ccache is persisted between builds using `actions/cache`. Cache keys are scoped by branch and commit SHA, with fallback to the branch key then `main`. +ccache is persisted between builds using `actions/cache`. Cache keys use a date suffix so entries rotate daily (one save per day per branch). Saves are skipped when the exact key already exists, preventing duplicate entries that fill runner storage. ``` -Key: ccache-build-{branch}-{sha} +Key: ccache-build-{branch}-{YYYYMMDD} Fallback: ccache-build-{branch}- Fallback: ccache-build-main- ``` +Release builds use a separate key namespace (`ccache-release-linux-{YYYYMMDD}`) because they compile with different optimization flags (`-O3`). The rattler-build script (`build.sh`) explicitly sets `CCACHE_DIR` and `CCACHE_BASEDIR` since rattler-build does not forward environment variables from the parent process. + ccache configuration: 4 GB max, zlib compression level 6, sloppy mode for include timestamps and PCH. --- @@ -63,17 +65,19 @@ Tags containing `rc`, `beta`, or `alpha` are marked as pre-releases. ### Platform matrix -| Job | Runner | Container | Preset | Output | -|-----|--------|-----------|--------|--------| -| `build-linux` | `ubuntu-latest` | `ubuntu:24.04` | `conda-linux-release` | AppImage, .deb | -| `build-macos` (Intel) | `macos-13` | native | `conda-macos-release` | .dmg (x86_64) | -| `build-macos` (Apple Silicon) | `macos-14` | native | `conda-macos-release` | .dmg (arm64) | -| `build-windows` | `windows-latest` | native | `conda-windows-release` | .exe (NSIS), .7z | +| Job | Runner | Container | Preset | Output | Status | +|-----|--------|-----------|--------|--------|--------| +| `build-linux` | `ubuntu-latest` | `ubuntu:24.04` | `conda-linux-release` | AppImage, .deb | Active | +| `build-macos` (Intel) | `macos-13` | native | `conda-macos-release` | .dmg (x86_64) | Disabled | +| `build-macos` (Apple Silicon) | `macos-14` | native | `conda-macos-release` | .dmg (arm64) | Disabled | +| `build-windows` | `windows-latest` | native | `conda-windows-release` | .exe (NSIS), .7z | Disabled | -All four jobs run concurrently. After all succeed, `publish-release` collects artifacts and creates the Gitea release. +Only the Linux build is currently active. macOS and Windows jobs are defined but commented out pending runner availability or cross-compilation support. After `build-linux` succeeds, `publish-release` collects artifacts and creates the Gitea release. ### Linux build +Both workflows start with a disk cleanup step that removes pre-installed bloat (dotnet, Android SDK, etc.) to free space for the build. + Uses the rattler-build packaging pipeline: 1. `pixi install` in `package/rattler-build/` @@ -81,9 +85,10 @@ Uses the rattler-build packaging pipeline: 3. The bundle script: - Copies the pixi conda environment to an AppDir - Strips unnecessary files (includes, static libs, cmake files, `__pycache__`) - - Downloads `appimagetool` and creates a squashfs AppImage (zstd compressed) - - Generates SHA256 checksums -4. `package/debian/build-deb.sh` builds a .deb from the AppDir + - Downloads `appimagetool`, extracts it with `--appimage-extract` (FUSE unavailable in containers), and runs via `squashfs-root/AppRun` + - Creates a squashfs AppImage (zstd compressed) with SHA256 checksums +4. Intermediate build files are cleaned up to free space for the .deb step +5. `package/debian/build-deb.sh` builds a .deb from the AppDir - Installs to `/opt/kindred-create/` with wrapper scripts in `/usr/bin/` - Sets up LD_LIBRARY_PATH, QT_PLUGIN_PATH, PYTHONPATH in wrappers - Creates desktop entry, MIME types, AppStream metainfo @@ -123,10 +128,15 @@ Builds natively on Windows runner: `publish-release` runs after all platform builds succeed: -1. Downloads all artifacts from `build-linux`, `build-macos`, `build-windows` -2. Collects release files (AppImage, .deb, .dmg, .7z, .exe, checksums) -3. Creates a Gitea release via `gitea.com/actions/release-action` -4. Requires `RELEASE_TOKEN` secret with repository write permissions +1. Downloads all artifacts from completed build jobs +2. Collects release files (AppImage, .deb, checksums) into a `release/` directory +3. Deletes any existing Gitea release for the same tag (allows re-running) +4. Creates a new Gitea release via the REST API (`/api/v1/repos/{owner}/{repo}/releases`) +5. Uploads each artifact as a release attachment via the API + +The release payload (tag name, body, prerelease flag) is built entirely in Python to avoid shell/Python type mismatches. Tags containing `rc`, `beta`, or `alpha` are automatically marked as pre-releases. + +Requires `RELEASE_TOKEN` secret with repository write permissions. --- @@ -174,6 +184,27 @@ container: network: bridge ``` +### Runner cleanup daemon + +A cleanup script at `.gitea/runner/cleanup.sh` prevents disk exhaustion on self-hosted runners. It uses a tiered approach based on disk usage thresholds: + +| Threshold | Action | +|-----------|--------| +| 70% | Docker cleanup (stopped containers, dangling images, build cache) | +| 80% | Purge act_runner cache entries older than 7 days, clean inactive workspaces | +| 90% | System cleanup (apt cache, old logs, journal vacuum to 100 MB) | +| 95% | Emergency: remove all act_runner cache entries and Docker images | + +Install via the provided systemd units (`.gitea/runner/cleanup.service` and `.gitea/runner/cleanup.timer`) to run every 30 minutes: + +```bash +sudo cp .gitea/runner/cleanup.sh /usr/local/bin/runner-cleanup.sh +sudo cp .gitea/runner/cleanup.service /etc/systemd/system/ +sudo cp .gitea/runner/cleanup.timer /etc/systemd/system/ +sudo systemctl daemon-reload +sudo systemctl enable --now cleanup.timer +``` + --- ## Secrets @@ -214,11 +245,12 @@ Defined in `CMakePresets.json`. Release builds use: ### ccache -Compiler cache is used across all builds to speed up incremental compilation. Cache is persisted between CI runs via `actions/cache`. Configuration: +Compiler cache is used across all builds to speed up incremental compilation. Cache is persisted between CI runs via `actions/cache` with date-based key rotation. Configuration: - Max size: 4 GB - Compression: zlib level 6 - Sloppy mode: include timestamps, PCH defines, time macros +- `CCACHE_BASEDIR`: set to workspace root (build workflow) or `$SRC_DIR` (rattler-build) for path normalization across runs --- @@ -243,9 +275,12 @@ The Docker container installs only minimal dependencies. If a new dependency is ccache misses spike when: - The compiler version changes (pixi update) - CMake presets change configuration flags -- The cache key doesn't match (new branch, force-pushed SHA) +- First build of the day (date-based key rotates daily) +- New branch without a prior cache (falls back to `main` cache) -Check `pixi run ccache -s` output for hit/miss ratios. +For release builds, ensure `build.sh` is correctly setting `CCACHE_DIR=/tmp/ccache-kindred-create` -- rattler-build does not forward environment variables from the workflow, so ccache config must be set inside the script. + +Check `pixi run ccache -s` output (printed in the "Show ccache statistics" step) for hit/miss ratios. The "Prepare ccache" step also prints the full ccache configuration via `ccache -p`. ### Submodule checkout fails diff --git a/docs/OVERVIEW.md b/docs/OVERVIEW.md index 172cf91d4b..02203e036b 100644 --- a/docs/OVERVIEW.md +++ b/docs/OVERVIEW.md @@ -1,7 +1,7 @@ # Kindred Create -**Last updated:** 2026-02-06 -**Branch:** main @ `c858706d480` +**Last updated:** 2026-02-07 +**Branch:** main @ `7bec3d5c3b2` **Kindred Create:** v0.1.0 **FreeCAD base:** v1.0.0 @@ -19,11 +19,13 @@ | Submodule | Path | Source | Pinned commit | |-----------|------|--------|---------------| -| ztools | `mods/ztools` | `gitea.kindred.internal/kindred/ztools-0065` | `d2f94c3` | -| silo | `mods/silo` | `gitea.kindred.internal/kindred/silo-0062` | `27e112e` | -| OndselSolver | `src/3rdParty/OndselSolver` | `gitea.kindred.internal/kindred/ondsel` | `5d1988b` | +| ztools | `mods/ztools` | `git.kindred-systems.com/forbes/ztools` | `d2f94c3` | +| silo-mod | `mods/silo` | `git.kindred-systems.com/kindred/silo-mod` | `bf0b843` | +| OndselSolver | `src/3rdParty/OndselSolver` | `git.kindred-systems.com/kindred/solver` | `5d1988b` | | GSL | `src/3rdParty/GSL` | `github.com/microsoft/GSL` | `756c91a` | | AddonManager | `src/Mod/AddonManager` | `github.com/FreeCAD/AddonManager` | `01e242e` | | googletest | `tests/lib` | `github.com/google/googletest` | `56efe39` | +The silo submodule was split from a monorepo into three repos: `silo-client` (shared Python API client), `silo-mod` (FreeCAD workbench, used as Create's submodule), and `silo-calc` (LibreOffice Calc extension). The `silo-mod` repo includes `silo-client` as its own submodule. + OndselSolver is forked from `github.com/FreeCAD/OndselSolver` to carry a Newton-Raphson convergence fix (see [KNOWN_ISSUES.md](KNOWN_ISSUES.md#12)). From 0e5a259d14ee28786f4aa3dac9946f0a6a508bd1 Mon Sep 17 00:00:00 2001 From: forbes Date: Sat, 7 Feb 2026 10:42:48 -0600 Subject: [PATCH 2/2] fix(ci): fetch only latest tag and add patchelf build dep - Change both build.yml and release.yml to fetch only the latest v* tag via git ls-remote instead of fetching all tags - Add patchelf as a Linux build dependency in recipe.yaml to fix rattler-build packaging failure --- .gitea/workflows/build.yml | 8 ++++++-- .gitea/workflows/release.yml | 8 ++++++-- package/rattler-build/recipe.yaml | 4 ++++ 3 files changed, 16 insertions(+), 4 deletions(-) diff --git a/.gitea/workflows/build.yml b/.gitea/workflows/build.yml index 2a13fb4765..886ddee350 100644 --- a/.gitea/workflows/build.yml +++ b/.gitea/workflows/build.yml @@ -47,8 +47,12 @@ jobs: submodules: recursive fetch-depth: 1 - - name: Fetch tags (for git describe) - run: git fetch --no-recurse-submodules --force --depth=1 origin '+refs/tags/*:refs/tags/*' + - name: Fetch latest tag (for git describe) + run: | + latest_tag=$(git ls-remote --tags --sort=-v:refname origin 'refs/tags/v*' | head -n1 | awk '{print $2}') + if [ -n "$latest_tag" ]; then + git fetch --no-recurse-submodules --force --depth=1 origin "+${latest_tag}:${latest_tag}" + fi - name: Install pixi run: | diff --git a/.gitea/workflows/release.yml b/.gitea/workflows/release.yml index d6d7a19dd2..1904aff212 100644 --- a/.gitea/workflows/release.yml +++ b/.gitea/workflows/release.yml @@ -57,8 +57,12 @@ jobs: submodules: recursive fetch-depth: 1 - - name: Fetch tags - run: git fetch --tags --force --no-recurse-submodules origin + - name: Fetch latest tag (for git describe) + run: | + latest_tag=$(git ls-remote --tags --sort=-v:refname origin 'refs/tags/v*' | head -n1 | awk '{print $2}') + if [ -n "$latest_tag" ]; then + git fetch --no-recurse-submodules --force --depth=1 origin "+${latest_tag}:${latest_tag}" + fi - name: Install pixi run: | diff --git a/package/rattler-build/recipe.yaml b/package/rattler-build/recipe.yaml index d99238ca88..9b33accd80 100644 --- a/package/rattler-build/recipe.yaml +++ b/package/rattler-build/recipe.yaml @@ -25,6 +25,10 @@ requirements: - qt6-main>=6.8,<6.9 - swig >=4.0,<4.4 + - if: linux + then: + - patchelf + - if: linux and x86_64 then: - clang