feat(solver): Phase 1e — regression tests and ASMT export compatibility #296

New Issue

forbes · 2026-02-19T21:29:04Z

forbes commented

2026-02-19 21:29:04 +00:00

Summary

Verify that the pluggable solver refactor produces identical results to the direct OndselSolver integration. Add unit tests for the new components.

Parent: #287 (Phase 1)
Depends on: #295 (1d — AssemblyObject refactor)

Scope

Test categories

1. SolverRegistry unit tests

scan() with empty directory — no error, empty registry
scan() with valid plugin — loads and registers
scan() with bad plugin (missing symbol, wrong version) — warns and skips
register_solver() — manual registration works
register_solver() with duplicate ID — warns and skips
get() for known/unknown IDs
set_default() / get_default()
joints_for() aggregation across multiple solvers

2. OndselAdapter unit tests

id(), name(), version() return expected values
supported_joints() returns all expected BaseJointKind mappings
is_deterministic() returns true
supports_bundle_fixed() returns false

3. Solve round-trip tests

Test assemblies that exercise each solver call pattern through the new interface:

One-shot solve:

Simple 2-part assembly with Fixed joint — parts stay in place
3-part chain with Revolute joints — convergence and correct DOF
Assembly with Distance constraint (Planar subtype) — distance maintained
Over-constrained assembly — SolveStatus::Redundant reported
Under-constrained assembly — correct DOF count

Drag protocol:

pre_drag() + drag_step() + post_drag() sequence
Dragged part moves, constrained parts follow
Multiple drag steps in sequence (stateful solver)

Simulation protocol:

run_kinematic() with rotational motion — frames generated
num_frames() returns expected count
update_for_frame() returns valid placements for each frame

4. Joint type decomposition tests

Verify buildSolveContext() correctly maps:

FreeCAD JointType	DistanceType	Expected BaseJointKind
Fixed	—	Fixed
Revolute	—	Revolute
Distance	PointPoint (d=0)	Coincident
Distance	PointPoint (d>0)	Coincident (with distance param)
Distance	PlanePlane	Planar
Distance	PointPlane	PointInPlane
Distance	PointLine	PointOnLine
Distance	LineLine	Concentric
Distance	CylinderCylinder	Concentric
Parallel	—	Parallel
Perpendicular	—	Perpendicular
Angle	—	Angle
RackPinion	—	Rack
Gears	—	Gear
Belt	—	Belt
Screw	—	Screw

5. Diagnostics tests

Redundant constraint detected via ConstraintDiagnostic (was: constraintSpec().starts_with("Redundant"))
DOF count matches previous updateSolveStatus() output
lastRedundantJoints, lastHasRedundancies flags set correctly

6. ASMT export compatibility

exportAsASMT() produces valid .asmt file
Output file structure matches pre-refactor output for same assembly
Round-trip: export → reimport validates

7. Orientation flip detection

validateNewPlacements() still catches >91-degree rotation flips
Grounded part movement detection still works
These are solver-agnostic and should work identically

Test file locations

src/Mod/Assembly/AssemblyTests/
├── TestCore.py                    # existing — extend with solver tests
├── TestSolverRegistry.py          # NEW
├── TestOndselAdapter.py           # NEW
├── TestSolveRoundTrip.py          # NEW
└── TestJointDecomposition.py      # NEW

Comparison strategy

For the critical "no behavioral change" guarantee:

Before refactor: Run existing test assemblies through the old direct-OndselSolver path, capture placements and DOF counts
After refactor: Run same assemblies through IKCSolver path, compare placements within tolerance (1e-10)
Constraint diagnostics: Compare redundancy detection output

Acceptance criteria

SolverRegistry unit tests pass (empty, load, duplicate, bad plugin)
OndselAdapter identity and capability tests pass
All 13 JointType + Distance subtypes correctly decompose to BaseJointKind
One-shot solve produces identical placements (within tolerance) to pre-refactor
Drag protocol round-trip works (pre/step/post sequence)
Simulation protocol round-trip works (run/frames/update sequence)
Redundancy detection produces identical results to pre-refactor
ASMT export still produces valid output
No existing test regressions
Orientation flip detection works through the new path

References

src/Mod/Assembly/AssemblyTests/TestCore.py — existing test framework
src/Mod/Assembly/AssemblyTests/ — test directory structure
docs/INTER_SOLVER.md §8 (Semi-deterministic behavior, tolerance-aware comparison)

## Summary Verify that the pluggable solver refactor produces identical results to the direct OndselSolver integration. Add unit tests for the new components. **Parent:** #287 (Phase 1) **Depends on:** #295 (1d — AssemblyObject refactor) ## Scope ### Test categories #### 1. SolverRegistry unit tests - `scan()` with empty directory — no error, empty registry - `scan()` with valid plugin — loads and registers - `scan()` with bad plugin (missing symbol, wrong version) — warns and skips - `register_solver()` — manual registration works - `register_solver()` with duplicate ID — warns and skips - `get()` for known/unknown IDs - `set_default()` / `get_default()` - `joints_for()` aggregation across multiple solvers #### 2. OndselAdapter unit tests - `id()`, `name()`, `version()` return expected values - `supported_joints()` returns all expected BaseJointKind mappings - `is_deterministic()` returns true - `supports_bundle_fixed()` returns false #### 3. Solve round-trip tests Test assemblies that exercise each solver call pattern through the new interface: **One-shot solve:** - Simple 2-part assembly with Fixed joint — parts stay in place - 3-part chain with Revolute joints — convergence and correct DOF - Assembly with Distance constraint (Planar subtype) — distance maintained - Over-constrained assembly — SolveStatus::Redundant reported - Under-constrained assembly — correct DOF count **Drag protocol:** - `pre_drag()` + `drag_step()` + `post_drag()` sequence - Dragged part moves, constrained parts follow - Multiple drag steps in sequence (stateful solver) **Simulation protocol:** - `run_kinematic()` with rotational motion — frames generated - `num_frames()` returns expected count - `update_for_frame()` returns valid placements for each frame #### 4. Joint type decomposition tests Verify `buildSolveContext()` correctly maps: | FreeCAD JointType | DistanceType | Expected BaseJointKind | |---|---|---| | Fixed | — | Fixed | | Revolute | — | Revolute | | Distance | PointPoint (d=0) | Coincident | | Distance | PointPoint (d>0) | Coincident (with distance param) | | Distance | PlanePlane | Planar | | Distance | PointPlane | PointInPlane | | Distance | PointLine | PointOnLine | | Distance | LineLine | Concentric | | Distance | CylinderCylinder | Concentric | | Parallel | — | Parallel | | Perpendicular | — | Perpendicular | | Angle | — | Angle | | RackPinion | — | Rack | | Gears | — | Gear | | Belt | — | Belt | | Screw | — | Screw | #### 5. Diagnostics tests - Redundant constraint detected via `ConstraintDiagnostic` (was: `constraintSpec().starts_with("Redundant")`) - DOF count matches previous `updateSolveStatus()` output - `lastRedundantJoints`, `lastHasRedundancies` flags set correctly #### 6. ASMT export compatibility - `exportAsASMT()` produces valid .asmt file - Output file structure matches pre-refactor output for same assembly - Round-trip: export → reimport validates #### 7. Orientation flip detection - `validateNewPlacements()` still catches >91-degree rotation flips - Grounded part movement detection still works - These are solver-agnostic and should work identically ### Test file locations ``` src/Mod/Assembly/AssemblyTests/ ├── TestCore.py # existing — extend with solver tests ├── TestSolverRegistry.py # NEW ├── TestOndselAdapter.py # NEW ├── TestSolveRoundTrip.py # NEW └── TestJointDecomposition.py # NEW ``` ### Comparison strategy For the critical "no behavioral change" guarantee: 1. **Before refactor**: Run existing test assemblies through the old direct-OndselSolver path, capture placements and DOF counts 2. **After refactor**: Run same assemblies through IKCSolver path, compare placements within tolerance (1e-10) 3. **Constraint diagnostics**: Compare redundancy detection output ## Acceptance criteria - [ ] SolverRegistry unit tests pass (empty, load, duplicate, bad plugin) - [ ] OndselAdapter identity and capability tests pass - [ ] All 13 JointType + Distance subtypes correctly decompose to BaseJointKind - [ ] One-shot solve produces identical placements (within tolerance) to pre-refactor - [ ] Drag protocol round-trip works (pre/step/post sequence) - [ ] Simulation protocol round-trip works (run/frames/update sequence) - [ ] Redundancy detection produces identical results to pre-refactor - [ ] ASMT export still produces valid output - [ ] No existing test regressions - [ ] Orientation flip detection works through the new path ## References - `src/Mod/Assembly/AssemblyTests/TestCore.py` — existing test framework - `src/Mod/Assembly/AssemblyTests/` — test directory structure - `docs/INTER_SOLVER.md` §8 (Semi-deterministic behavior, tolerance-aware comparison)

forbes added the enhancement label 2026-02-19 21:29:04 +00:00

forbes referenced this issue from a commit

2026-02-19 22:57:28 +00:00

test(assembly): regression tests for KCSolve solver refactor (#296)

forbes referenced a pull request that will close this issue

2026-02-19 22:58:02 +00:00

feat(assembly): pluggable solver system — Phase 1 (#287) #297

forbes closed this issue

2026-02-19 22:58:36 +00:00

Sign in to join this conversation.