docs(dev): add testing guide covering Tier 1/2 and C++ test infrastructure
All checks were successful
Build and Test / build (pull_request) Successful in 31m10s

New docs/src/development/testing.md covering:
- Quick reference commands (pixi run test-kindred, pixi run test)
- Tier 1 pure Python tests: mock strategy, test file table, addon
  loader test class breakdown, how to write a new test file
- Tier 2 FreeCAD headless tests: runner, current status
- C++ GoogleTest infrastructure: directory layout, test executables
- Decision guide for which tier to use

Add entry to docs/src/SUMMARY.md under Development section.
This commit is contained in:
2026-03-05 10:10:03 -06:00
parent 7381675a6e
commit 0d88769189
2 changed files with 129 additions and 0 deletions

View File

@@ -28,6 +28,7 @@
- [Gui Module Build](./development/gui-build-integration.md)
- [Package.xml Schema Extensions](./development/package-xml-schema.md)
- [Writing an Addon](./development/writing-an-addon.md)
- [Testing](./development/testing.md)
# Silo Server

View File

@@ -0,0 +1,128 @@
# Testing
Kindred Create has a multi-tier testing system that separates fast pure-logic tests from tests requiring the FreeCAD runtime.
## Quick reference
```bash
pixi run test-kindred # Python Tier 1 tests (no FreeCAD binary)
pixi run test # C++ tests via ctest (requires build)
```
Or run directly without pixi:
```bash
python3 tests/run_kindred_tests.py # Tier 1 only
python3 tests/run_kindred_tests.py --all # Tier 1 + Tier 2 (needs FreeCADCmd)
```
## Tier 1: Pure Python tests
These tests exercise standalone Python functions **without** a FreeCAD binary. They run with the system Python interpreter and complete in under a second.
**Framework:** `unittest` (standard library)
**Test files:**
| File | Module under test | Tests |
|------|-------------------|-------|
| `tests/test_kindred_pure.py` | `update_checker`, `silo_commands`, `silo_start`, `silo_origin` | ~39 |
| `tests/test_kindred_addon_loader.py` | `addon_loader` | ~71 |
**Discovery:** The test runner uses `unittest.TestLoader().discover()` with the pattern `test_kindred_*.py`. New test files matching this pattern are automatically discovered.
### How mocking works
Modules under test import `FreeCAD` at the top level, which is unavailable outside the FreeCAD runtime. Each test file mocks the FreeCAD ecosystem **before** importing any target modules:
```python
from unittest import mock
_fc = mock.MagicMock()
_fc.Console = mock.MagicMock()
# Insert mocks before any import touches these modules
for mod_name in ("FreeCAD", "FreeCADGui"):
sys.modules.setdefault(mod_name, mock.MagicMock())
sys.modules["FreeCAD"] = _fc
# Now safe to import
sys.path.insert(0, str(_REPO_ROOT / "src" / "Mod" / "Create"))
from addon_loader import parse_manifest, validate_manifest # etc.
```
For modules that need working FreeCAD subsystems (like the parameter store), functional stubs replace `MagicMock`. See `_MockParamGroup` in `test_kindred_pure.py` for an example.
### Addon loader tests
`test_kindred_addon_loader.py` covers the entire addon loading pipeline:
| Test class | What it covers |
|------------|----------------|
| `TestAddonState` | Enum members and values |
| `TestValidVersion` | `_valid_version()` dotted-numeric regex |
| `TestParseVersionLoader` | `_parse_version()` string-to-tuple conversion |
| `TestAddonRegistry` | Registry CRUD, filtering by state, load order, context registration |
| `TestParseManifest` | XML parsing, field extraction, `<kindred>` extensions, validation errors (bad priority, bad version format, invalid context IDs, malformed XML) |
| `TestValidateDependencies` | Cross-addon dependency name checking |
| `TestValidateManifest` | Version bounds, workbench path, Init.py presence, error accumulation |
| `TestScanAddons` | Directory discovery at depth 1 and 2 |
| `TestResolveLoadOrder` | Topological sort, priority ties, cycle detection, legacy fallback |
Tests that need files on disk use `tempfile.TemporaryDirectory` with a `_write_package_xml()` helper that generates manifest XML programmatically. Tests that only need populated `AddonManifest` objects use a `_make_manifest()` shortcut.
### Writing a new Tier 1 test file
1. Create `tests/test_kindred_<module>.py`
2. Mock FreeCAD modules before imports (copy the pattern from an existing file)
3. Add the source directory to `sys.path`
4. Import and test pure-logic functions
5. The runner discovers the file automatically via `test_kindred_*.py`
Run your new file directly during development:
```bash
python3 tests/test_kindred_<module>.py -v
```
## Tier 2: FreeCAD headless tests
These tests run inside `FreeCADCmd` (the headless FreeCAD binary) and can exercise code that depends on the full application context — document creation, GUI commands, origin resolution, etc.
**Runner:** `tests/run_kindred_tests.py --all`
**Status:** Infrastructure is in place but no Tier 2 test modules have been implemented yet. The runner searches for `FreeCADCmd` in `PATH` and the build directories (`build/debug/bin/`, `build/release/bin/`). If not found, Tier 2 is skipped without failure.
## C++ tests
C++ unit tests use [GoogleTest](https://github.com/google/googletest) (submodule at `tests/lib/`).
**Runner:** `ctest` via pixi
```bash
pixi run test # debug build
pixi run test-release # release build
```
Test source files live in `tests/src/` mirroring the main source layout:
```
tests/src/
├── App/ # FreeCADApp tests
├── Base/ # Base library tests (Quantity, Matrix, Axis, etc.)
├── Gui/ # GUI tests (OriginManager, InputHint, etc.)
├── Mod/ # Module tests (Part, Sketcher, Assembly, etc.)
└── Misc/ # Miscellaneous tests
```
Each directory builds a separate test executable (e.g., `Base_tests_run`, `Gui_tests_run`) linked against GoogleTest and the relevant FreeCAD libraries.
## What to test where
| Scenario | Tier | Example |
|----------|------|---------|
| Pure function with no FreeCAD deps | Tier 1 | Version parsing, XML parsing, topological sort |
| Logic that only needs mock stubs | Tier 1 | Parameter reading, URL construction |
| Code that creates/modifies documents | Tier 2 | Origin ownership detection, document observer |
| Code that needs the GUI event loop | Tier 2 | Context resolution, toolbar visibility |
| C++ classes and algorithms | C++ | OriginManager, Base::Quantity, Sketcher solver |