docs(dev): add testing guide covering Tier 1/2 and C++ test infrastructure
All checks were successful
Build and Test / build (pull_request) Successful in 31m10s
All checks were successful
Build and Test / build (pull_request) Successful in 31m10s
New docs/src/development/testing.md covering: - Quick reference commands (pixi run test-kindred, pixi run test) - Tier 1 pure Python tests: mock strategy, test file table, addon loader test class breakdown, how to write a new test file - Tier 2 FreeCAD headless tests: runner, current status - C++ GoogleTest infrastructure: directory layout, test executables - Decision guide for which tier to use Add entry to docs/src/SUMMARY.md under Development section.
This commit is contained in:
@@ -28,6 +28,7 @@
|
||||
- [Gui Module Build](./development/gui-build-integration.md)
|
||||
- [Package.xml Schema Extensions](./development/package-xml-schema.md)
|
||||
- [Writing an Addon](./development/writing-an-addon.md)
|
||||
- [Testing](./development/testing.md)
|
||||
|
||||
# Silo Server
|
||||
|
||||
|
||||
128
docs/src/development/testing.md
Normal file
128
docs/src/development/testing.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# Testing
|
||||
|
||||
Kindred Create has a multi-tier testing system that separates fast pure-logic tests from tests requiring the FreeCAD runtime.
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
pixi run test-kindred # Python Tier 1 tests (no FreeCAD binary)
|
||||
pixi run test # C++ tests via ctest (requires build)
|
||||
```
|
||||
|
||||
Or run directly without pixi:
|
||||
|
||||
```bash
|
||||
python3 tests/run_kindred_tests.py # Tier 1 only
|
||||
python3 tests/run_kindred_tests.py --all # Tier 1 + Tier 2 (needs FreeCADCmd)
|
||||
```
|
||||
|
||||
## Tier 1: Pure Python tests
|
||||
|
||||
These tests exercise standalone Python functions **without** a FreeCAD binary. They run with the system Python interpreter and complete in under a second.
|
||||
|
||||
**Framework:** `unittest` (standard library)
|
||||
|
||||
**Test files:**
|
||||
|
||||
| File | Module under test | Tests |
|
||||
|------|-------------------|-------|
|
||||
| `tests/test_kindred_pure.py` | `update_checker`, `silo_commands`, `silo_start`, `silo_origin` | ~39 |
|
||||
| `tests/test_kindred_addon_loader.py` | `addon_loader` | ~71 |
|
||||
|
||||
**Discovery:** The test runner uses `unittest.TestLoader().discover()` with the pattern `test_kindred_*.py`. New test files matching this pattern are automatically discovered.
|
||||
|
||||
### How mocking works
|
||||
|
||||
Modules under test import `FreeCAD` at the top level, which is unavailable outside the FreeCAD runtime. Each test file mocks the FreeCAD ecosystem **before** importing any target modules:
|
||||
|
||||
```python
|
||||
from unittest import mock
|
||||
|
||||
_fc = mock.MagicMock()
|
||||
_fc.Console = mock.MagicMock()
|
||||
|
||||
# Insert mocks before any import touches these modules
|
||||
for mod_name in ("FreeCAD", "FreeCADGui"):
|
||||
sys.modules.setdefault(mod_name, mock.MagicMock())
|
||||
sys.modules["FreeCAD"] = _fc
|
||||
|
||||
# Now safe to import
|
||||
sys.path.insert(0, str(_REPO_ROOT / "src" / "Mod" / "Create"))
|
||||
from addon_loader import parse_manifest, validate_manifest # etc.
|
||||
```
|
||||
|
||||
For modules that need working FreeCAD subsystems (like the parameter store), functional stubs replace `MagicMock`. See `_MockParamGroup` in `test_kindred_pure.py` for an example.
|
||||
|
||||
### Addon loader tests
|
||||
|
||||
`test_kindred_addon_loader.py` covers the entire addon loading pipeline:
|
||||
|
||||
| Test class | What it covers |
|
||||
|------------|----------------|
|
||||
| `TestAddonState` | Enum members and values |
|
||||
| `TestValidVersion` | `_valid_version()` dotted-numeric regex |
|
||||
| `TestParseVersionLoader` | `_parse_version()` string-to-tuple conversion |
|
||||
| `TestAddonRegistry` | Registry CRUD, filtering by state, load order, context registration |
|
||||
| `TestParseManifest` | XML parsing, field extraction, `<kindred>` extensions, validation errors (bad priority, bad version format, invalid context IDs, malformed XML) |
|
||||
| `TestValidateDependencies` | Cross-addon dependency name checking |
|
||||
| `TestValidateManifest` | Version bounds, workbench path, Init.py presence, error accumulation |
|
||||
| `TestScanAddons` | Directory discovery at depth 1 and 2 |
|
||||
| `TestResolveLoadOrder` | Topological sort, priority ties, cycle detection, legacy fallback |
|
||||
|
||||
Tests that need files on disk use `tempfile.TemporaryDirectory` with a `_write_package_xml()` helper that generates manifest XML programmatically. Tests that only need populated `AddonManifest` objects use a `_make_manifest()` shortcut.
|
||||
|
||||
### Writing a new Tier 1 test file
|
||||
|
||||
1. Create `tests/test_kindred_<module>.py`
|
||||
2. Mock FreeCAD modules before imports (copy the pattern from an existing file)
|
||||
3. Add the source directory to `sys.path`
|
||||
4. Import and test pure-logic functions
|
||||
5. The runner discovers the file automatically via `test_kindred_*.py`
|
||||
|
||||
Run your new file directly during development:
|
||||
|
||||
```bash
|
||||
python3 tests/test_kindred_<module>.py -v
|
||||
```
|
||||
|
||||
## Tier 2: FreeCAD headless tests
|
||||
|
||||
These tests run inside `FreeCADCmd` (the headless FreeCAD binary) and can exercise code that depends on the full application context — document creation, GUI commands, origin resolution, etc.
|
||||
|
||||
**Runner:** `tests/run_kindred_tests.py --all`
|
||||
|
||||
**Status:** Infrastructure is in place but no Tier 2 test modules have been implemented yet. The runner searches for `FreeCADCmd` in `PATH` and the build directories (`build/debug/bin/`, `build/release/bin/`). If not found, Tier 2 is skipped without failure.
|
||||
|
||||
## C++ tests
|
||||
|
||||
C++ unit tests use [GoogleTest](https://github.com/google/googletest) (submodule at `tests/lib/`).
|
||||
|
||||
**Runner:** `ctest` via pixi
|
||||
|
||||
```bash
|
||||
pixi run test # debug build
|
||||
pixi run test-release # release build
|
||||
```
|
||||
|
||||
Test source files live in `tests/src/` mirroring the main source layout:
|
||||
|
||||
```
|
||||
tests/src/
|
||||
├── App/ # FreeCADApp tests
|
||||
├── Base/ # Base library tests (Quantity, Matrix, Axis, etc.)
|
||||
├── Gui/ # GUI tests (OriginManager, InputHint, etc.)
|
||||
├── Mod/ # Module tests (Part, Sketcher, Assembly, etc.)
|
||||
└── Misc/ # Miscellaneous tests
|
||||
```
|
||||
|
||||
Each directory builds a separate test executable (e.g., `Base_tests_run`, `Gui_tests_run`) linked against GoogleTest and the relevant FreeCAD libraries.
|
||||
|
||||
## What to test where
|
||||
|
||||
| Scenario | Tier | Example |
|
||||
|----------|------|---------|
|
||||
| Pure function with no FreeCAD deps | Tier 1 | Version parsing, XML parsing, topological sort |
|
||||
| Logic that only needs mock stubs | Tier 1 | Parameter reading, URL construction |
|
||||
| Code that creates/modifies documents | Tier 2 | Origin ownership detection, document observer |
|
||||
| Code that needs the GUI event loop | Tier 2 | Context resolution, toolbar visibility |
|
||||
| C++ classes and algorithms | C++ | OriginManager, Base::Quantity, Sketcher solver |
|
||||
Reference in New Issue
Block a user