From 0d88769189780bbc3c23f7adf13fced3c3e8f88d Mon Sep 17 00:00:00 2001 From: forbes-0023 Date: Thu, 5 Mar 2026 10:10:03 -0600 Subject: [PATCH] docs(dev): add testing guide covering Tier 1/2 and C++ test infrastructure New docs/src/development/testing.md covering: - Quick reference commands (pixi run test-kindred, pixi run test) - Tier 1 pure Python tests: mock strategy, test file table, addon loader test class breakdown, how to write a new test file - Tier 2 FreeCAD headless tests: runner, current status - C++ GoogleTest infrastructure: directory layout, test executables - Decision guide for which tier to use Add entry to docs/src/SUMMARY.md under Development section. --- docs/src/SUMMARY.md | 1 + docs/src/development/testing.md | 128 ++++++++++++++++++++++++++++++++ 2 files changed, 129 insertions(+) create mode 100644 docs/src/development/testing.md diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 6e7870a9be..22c0cab703 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -28,6 +28,7 @@ - [Gui Module Build](./development/gui-build-integration.md) - [Package.xml Schema Extensions](./development/package-xml-schema.md) - [Writing an Addon](./development/writing-an-addon.md) +- [Testing](./development/testing.md) # Silo Server diff --git a/docs/src/development/testing.md b/docs/src/development/testing.md new file mode 100644 index 0000000000..016e15c725 --- /dev/null +++ b/docs/src/development/testing.md @@ -0,0 +1,128 @@ +# Testing + +Kindred Create has a multi-tier testing system that separates fast pure-logic tests from tests requiring the FreeCAD runtime. + +## Quick reference + +```bash +pixi run test-kindred # Python Tier 1 tests (no FreeCAD binary) +pixi run test # C++ tests via ctest (requires build) +``` + +Or run directly without pixi: + +```bash +python3 tests/run_kindred_tests.py # Tier 1 only +python3 tests/run_kindred_tests.py --all # Tier 1 + Tier 2 (needs FreeCADCmd) +``` + +## Tier 1: Pure Python tests + +These tests exercise standalone Python functions **without** a FreeCAD binary. They run with the system Python interpreter and complete in under a second. + +**Framework:** `unittest` (standard library) + +**Test files:** + +| File | Module under test | Tests | +|------|-------------------|-------| +| `tests/test_kindred_pure.py` | `update_checker`, `silo_commands`, `silo_start`, `silo_origin` | ~39 | +| `tests/test_kindred_addon_loader.py` | `addon_loader` | ~71 | + +**Discovery:** The test runner uses `unittest.TestLoader().discover()` with the pattern `test_kindred_*.py`. New test files matching this pattern are automatically discovered. + +### How mocking works + +Modules under test import `FreeCAD` at the top level, which is unavailable outside the FreeCAD runtime. Each test file mocks the FreeCAD ecosystem **before** importing any target modules: + +```python +from unittest import mock + +_fc = mock.MagicMock() +_fc.Console = mock.MagicMock() + +# Insert mocks before any import touches these modules +for mod_name in ("FreeCAD", "FreeCADGui"): + sys.modules.setdefault(mod_name, mock.MagicMock()) +sys.modules["FreeCAD"] = _fc + +# Now safe to import +sys.path.insert(0, str(_REPO_ROOT / "src" / "Mod" / "Create")) +from addon_loader import parse_manifest, validate_manifest # etc. +``` + +For modules that need working FreeCAD subsystems (like the parameter store), functional stubs replace `MagicMock`. See `_MockParamGroup` in `test_kindred_pure.py` for an example. + +### Addon loader tests + +`test_kindred_addon_loader.py` covers the entire addon loading pipeline: + +| Test class | What it covers | +|------------|----------------| +| `TestAddonState` | Enum members and values | +| `TestValidVersion` | `_valid_version()` dotted-numeric regex | +| `TestParseVersionLoader` | `_parse_version()` string-to-tuple conversion | +| `TestAddonRegistry` | Registry CRUD, filtering by state, load order, context registration | +| `TestParseManifest` | XML parsing, field extraction, `` extensions, validation errors (bad priority, bad version format, invalid context IDs, malformed XML) | +| `TestValidateDependencies` | Cross-addon dependency name checking | +| `TestValidateManifest` | Version bounds, workbench path, Init.py presence, error accumulation | +| `TestScanAddons` | Directory discovery at depth 1 and 2 | +| `TestResolveLoadOrder` | Topological sort, priority ties, cycle detection, legacy fallback | + +Tests that need files on disk use `tempfile.TemporaryDirectory` with a `_write_package_xml()` helper that generates manifest XML programmatically. Tests that only need populated `AddonManifest` objects use a `_make_manifest()` shortcut. + +### Writing a new Tier 1 test file + +1. Create `tests/test_kindred_.py` +2. Mock FreeCAD modules before imports (copy the pattern from an existing file) +3. Add the source directory to `sys.path` +4. Import and test pure-logic functions +5. The runner discovers the file automatically via `test_kindred_*.py` + +Run your new file directly during development: + +```bash +python3 tests/test_kindred_.py -v +``` + +## Tier 2: FreeCAD headless tests + +These tests run inside `FreeCADCmd` (the headless FreeCAD binary) and can exercise code that depends on the full application context — document creation, GUI commands, origin resolution, etc. + +**Runner:** `tests/run_kindred_tests.py --all` + +**Status:** Infrastructure is in place but no Tier 2 test modules have been implemented yet. The runner searches for `FreeCADCmd` in `PATH` and the build directories (`build/debug/bin/`, `build/release/bin/`). If not found, Tier 2 is skipped without failure. + +## C++ tests + +C++ unit tests use [GoogleTest](https://github.com/google/googletest) (submodule at `tests/lib/`). + +**Runner:** `ctest` via pixi + +```bash +pixi run test # debug build +pixi run test-release # release build +``` + +Test source files live in `tests/src/` mirroring the main source layout: + +``` +tests/src/ +├── App/ # FreeCADApp tests +├── Base/ # Base library tests (Quantity, Matrix, Axis, etc.) +├── Gui/ # GUI tests (OriginManager, InputHint, etc.) +├── Mod/ # Module tests (Part, Sketcher, Assembly, etc.) +└── Misc/ # Miscellaneous tests +``` + +Each directory builds a separate test executable (e.g., `Base_tests_run`, `Gui_tests_run`) linked against GoogleTest and the relevant FreeCAD libraries. + +## What to test where + +| Scenario | Tier | Example | +|----------|------|---------| +| Pure function with no FreeCAD deps | Tier 1 | Version parsing, XML parsing, topological sort | +| Logic that only needs mock stubs | Tier 1 | Parameter reading, URL construction | +| Code that creates/modifies documents | Tier 2 | Origin ownership detection, document observer | +| Code that needs the GUI event loop | Tier 2 | Context resolution, toolbar visibility | +| C++ classes and algorithms | C++ | OriginManager, Base::Quantity, Sketcher solver |