test(assembly): expand drag solve test coverage — multi-step, combined joints, orientation tracking #339

New Issue

forbes · 2026-02-27T15:03:08Z

forbes commented

2026-02-27 15:03:08 +00:00

Summary

The drag solve test suite has insufficient coverage, which allowed the planar half-space drag flip bug (#338) and its five preceding fix attempts to pass all tests while the bug persisted in interactive use. Every iteration of the fix passed the existing tests — meaning the tests don't exercise the code paths that actually break.

This issue tracks the gaps and proposes specific test scenarios to close them.

Current test landscape

Solver tests (`mods/solver/tests/`) — 17 files

File	What it tests	Drag coverage
`test_preference.py`	Half-space tracking, sign preservation, weighted solve	None — single-shot `newton_solve()` only
`test_joints.py`	All joint types: DOF counting + convergence from displaced ICs	None — single-shot solve, no drag API
`test_solver.py`	End-to-end solver pipeline (Coincident, Distance, Fixed)	None
`test_newton.py`	Newton-Raphson convergence, callbacks	None
`test_bfgs.py`	BFGS fallback solver	None
`test_constraints.py`	Basic constraint residual generation	None
`test_constraints_phase2.py`	All constraint types' residual generation	None
`test_decompose.py`	Graph decomposition, cluster solving	None
`test_diagnostics.py`	DOF diagnostics, overconstrained detection	None
`test_dof.py`	DOF counting	None
`test_entities.py`	RigidBody transformations	None
`test_geometry.py`	Geometry helpers (dot, cross, point-plane)	None
`test_expr.py`	Expression DAG	None
`test_params.py`	Parameter table operations	None
`test_prepass.py`	Pre-solve substitution passes	None
`test_codegen.py`	Code generation, CSE, compilation	None
`console_test_phase5.py`	In-client integration (not automated)	None

Assembly C++ tests (`src/Mod/Assembly/AssemblyTests/`) — 7 files

File	What it tests	Drag coverage
`TestCore.py`	Assembly/joint object creation, basic solve	None
`TestSolverIntegration.py`	Full-stack solver (default backend)	None
`TestKindredSolverIntegration.py`	Full-stack solver (kindred backend)	None
`TestKCSolvePy.py`	pybind11 kcsolve module bindings	None
`TestDatumClassification.py`	Distance joint datum plane classification	None
`TestAssemblyOriginPlanes.py`	Origin plane setup, grounding, joint solving	None
`TestCommandInsertLink.py`	GUI insert-link command	None

Total drag API test coverage: zero. No test anywhere calls pre_drag(), drag_step(), or post_drag().

Coverage gaps

Gap 1: No drag protocol tests

The three-phase drag API (pre_drag / drag_step / post_drag) is completely untested. This is the primary interaction model for users manipulating assemblies.

Missing scenarios:

Basic drag flow: pre_drag() -> N x drag_step() -> post_drag()
Drag step acceptance vs rejection
Drag step counter accuracy
Solver state continuity across drag steps (cached Jacobian, compiled evaluator)

Gap 2: No multi-step incremental drag sequences

All existing solve tests use a single-shot solve from displaced initial conditions. None test the incremental pattern where each step's output becomes the next step's initial guess. The planar drift bug only manifests across multiple incremental steps as numerical error accumulates.

Missing scenarios:

10-step drag sequence with small increments
50-step drag sequence simulating a smooth mouse arc
Verification that constraint residuals stay below tolerance at every step (not just the final one)

Gap 3: No large-rotation drag tests

The largest rotation tested in the suite is 90 degrees (test_entities.py, test_joints.py), and only as a static initial condition — never as a drag target.

Missing scenarios:

Drag through 90 degrees (the critical half-space crossing point)
Drag through 180 degrees (full normal flip)
Drag through 360 degrees (full revolution, return to start)
Drag through 45-degree increments verifying constraint satisfaction at each step

Gap 4: No combined-joint drag tests

test_joints.py tests each joint type in isolation. The planar drift bug specifically occurs with Cylindrical + Planar combined. No test exercises joint combinations under drag.

Missing scenarios:

Cylindrical + Planar (the #338 bug scenario): drag-rotate around cylinder axis while constrained to XY plane
Revolute + Planar: same concept, single rotational DOF
Revolute + Distance (plane-plane): rotation with offset plane constraint
Ball + Planar: spherical rotation constrained to a plane

Gap 5: No orientation preservation tests across drag steps

test_preference.py tests half-space sign preservation for a single solve, but never verifies that orientation is preserved across a drag sequence. The 91-degree validator in C++ catches catastrophic flips, but there's no test that the solver doesn't gradually drift orientation across steps.

Missing scenarios:

Verify dot(z_body, z_fixed) > 0 holds at every drag step for a Planar constraint
Verify quaternion continuity (no hemisphere jumps) across drag steps
Verify cumulative rotation from drag start stays within expected bounds

Gap 6: No half-space tracking under drag

test_preference.py tests compute_half_spaces() and apply_half_space_correction() in isolation. It never tests half-space tracking through the drag pipeline where the correction interacts with cached state, baseline updates, and the C++ validator.

Missing scenarios:

Half-space indicator value tracked across 10+ drag steps
Verify correction_fn is NOT called for on-plane (distance=0) planar constraints during drag
Verify correction_fn IS called for off-plane planar constraints when the solver crosses branches
Verify half-space reference_sign updates correctly after each accepted step

Gap 7: No `validateNewPlacements()` unit tests

The C++ 91-degree orientation validator is untested. It's the final safety net and the symptom that surfaces when the solver produces bad results.

Missing scenarios:

Small rotation (<91 deg) passes validation
Large rotation (>91 deg) fails validation
Grounded part movement fails validation
Baseline update (savePlacementsForUndo()) resets the comparison point
Cumulative small rotations don't false-positive when baseline is updated per step

Proposed test additions

Priority 1 — Regression test for #338

Add test_planar_cylindrical_drag.py (or extend test_joints.py) with:

def test_cylindrical_plus_planar_drag_360():
    """Cylindrical + Planar: full rotation drag must keep body on plane."""
    # Setup: ground + free body, Cylindrical on Z + Planar on XY (distance=0)
    # Drag: 36 steps of 10 degrees each around Z axis
    # Assert: at every step, body Z position == 0 (within tolerance)
    # Assert: at every step, residuals < 1e-8
    # Assert: solver converges at every step

Priority 2 — Drag protocol integration tests

Add test_drag_protocol.py exercising the Python solver's drag API:

test_drag_basic_flow — pre_drag/drag_step/post_drag lifecycle
test_drag_step_convergence — every step converges
test_drag_residuals_stay_satisfied — residuals < tol at each step
test_drag_50_steps_smooth_arc — long sequence stability

Priority 3 — Combined joint drag tests

Extend joint tests with drag sequences for:

Revolute + Planar (1 DOF rotation on plane)
Cylindrical + Planar (2 DOF: rotation + translation along axis, constrained to plane)
Ball + Planar (3 DOF rotation, constrained to plane)

Priority 4 — Half-space tracking under drag

Extend test_preference.py with drag-sequence half-space tests:

test_planar_halfspace_no_correction_on_plane — verify correction_fn=None for distance=0
test_halfspace_indicator_stable_across_drag — sign doesn't flip during smooth drag
test_halfspace_correction_off_plane — correction fires correctly for offset planar constraints

Priority 5 — C++ validateNewPlacements unit tests

If feasible via pybind11/FreeCADCmd:

test_validate_small_rotation_passes
test_validate_large_rotation_rejected
test_validate_grounded_movement_rejected
test_validate_baseline_update_resets_comparison

#338 — The planar constraint drift bug this would have caught
#254 — Original Planar joint implementation
#298 — JCS visual feedback and constraint drag robustness

## Summary The drag solve test suite has insufficient coverage, which allowed the planar half-space drag flip bug (#338) and its five preceding fix attempts to pass all tests while the bug persisted in interactive use. Every iteration of the fix passed the existing tests — meaning the tests don't exercise the code paths that actually break. This issue tracks the gaps and proposes specific test scenarios to close them. ## Current test landscape ### Solver tests (`mods/solver/tests/`) — 17 files | File | What it tests | Drag coverage | |------|---------------|---------------| | `test_preference.py` | Half-space tracking, sign preservation, weighted solve | None — single-shot `newton_solve()` only | | `test_joints.py` | All joint types: DOF counting + convergence from displaced ICs | None — single-shot solve, no drag API | | `test_solver.py` | End-to-end solver pipeline (Coincident, Distance, Fixed) | None | | `test_newton.py` | Newton-Raphson convergence, callbacks | None | | `test_bfgs.py` | BFGS fallback solver | None | | `test_constraints.py` | Basic constraint residual generation | None | | `test_constraints_phase2.py` | All constraint types' residual generation | None | | `test_decompose.py` | Graph decomposition, cluster solving | None | | `test_diagnostics.py` | DOF diagnostics, overconstrained detection | None | | `test_dof.py` | DOF counting | None | | `test_entities.py` | RigidBody transformations | None | | `test_geometry.py` | Geometry helpers (dot, cross, point-plane) | None | | `test_expr.py` | Expression DAG | None | | `test_params.py` | Parameter table operations | None | | `test_prepass.py` | Pre-solve substitution passes | None | | `test_codegen.py` | Code generation, CSE, compilation | None | | `console_test_phase5.py` | In-client integration (not automated) | None | ### Assembly C++ tests (`src/Mod/Assembly/AssemblyTests/`) — 7 files | File | What it tests | Drag coverage | |------|---------------|---------------| | `TestCore.py` | Assembly/joint object creation, basic solve | None | | `TestSolverIntegration.py` | Full-stack solver (default backend) | None | | `TestKindredSolverIntegration.py` | Full-stack solver (kindred backend) | None | | `TestKCSolvePy.py` | pybind11 kcsolve module bindings | None | | `TestDatumClassification.py` | Distance joint datum plane classification | None | | `TestAssemblyOriginPlanes.py` | Origin plane setup, grounding, joint solving | None | | `TestCommandInsertLink.py` | GUI insert-link command | None | **Total drag API test coverage: zero.** No test anywhere calls `pre_drag()`, `drag_step()`, or `post_drag()`. ## Coverage gaps ### Gap 1: No drag protocol tests The three-phase drag API (`pre_drag` / `drag_step` / `post_drag`) is completely untested. This is the primary interaction model for users manipulating assemblies. Missing scenarios: - Basic drag flow: `pre_drag()` -> N x `drag_step()` -> `post_drag()` - Drag step acceptance vs rejection - Drag step counter accuracy - Solver state continuity across drag steps (cached Jacobian, compiled evaluator) ### Gap 2: No multi-step incremental drag sequences All existing solve tests use a **single-shot solve from displaced initial conditions**. None test the incremental pattern where each step's output becomes the next step's initial guess. The planar drift bug only manifests across multiple incremental steps as numerical error accumulates. Missing scenarios: - 10-step drag sequence with small increments - 50-step drag sequence simulating a smooth mouse arc - Verification that constraint residuals stay below tolerance at every step (not just the final one) ### Gap 3: No large-rotation drag tests The largest rotation tested in the suite is 90 degrees (`test_entities.py`, `test_joints.py`), and only as a static initial condition — never as a drag target. Missing scenarios: - Drag through 90 degrees (the critical half-space crossing point) - Drag through 180 degrees (full normal flip) - Drag through 360 degrees (full revolution, return to start) - Drag through 45-degree increments verifying constraint satisfaction at each step ### Gap 4: No combined-joint drag tests `test_joints.py` tests each joint type in isolation. The planar drift bug specifically occurs with **Cylindrical + Planar** combined. No test exercises joint combinations under drag. Missing scenarios: - Cylindrical + Planar (the #338 bug scenario): drag-rotate around cylinder axis while constrained to XY plane - Revolute + Planar: same concept, single rotational DOF - Revolute + Distance (plane-plane): rotation with offset plane constraint - Ball + Planar: spherical rotation constrained to a plane ### Gap 5: No orientation preservation tests across drag steps `test_preference.py` tests half-space sign preservation for a single solve, but never verifies that orientation is preserved across a drag sequence. The 91-degree validator in C++ catches catastrophic flips, but there's no test that the solver doesn't gradually drift orientation across steps. Missing scenarios: - Verify `dot(z_body, z_fixed) > 0` holds at every drag step for a Planar constraint - Verify quaternion continuity (no hemisphere jumps) across drag steps - Verify cumulative rotation from drag start stays within expected bounds ### Gap 6: No half-space tracking under drag `test_preference.py` tests `compute_half_spaces()` and `apply_half_space_correction()` in isolation. It never tests half-space tracking through the drag pipeline where the correction interacts with cached state, baseline updates, and the C++ validator. Missing scenarios: - Half-space indicator value tracked across 10+ drag steps - Verify correction_fn is NOT called for on-plane (distance=0) planar constraints during drag - Verify correction_fn IS called for off-plane planar constraints when the solver crosses branches - Verify half-space reference_sign updates correctly after each accepted step ### Gap 7: No `validateNewPlacements()` unit tests The C++ 91-degree orientation validator is untested. It's the final safety net and the symptom that surfaces when the solver produces bad results. Missing scenarios: - Small rotation (<91 deg) passes validation - Large rotation (>91 deg) fails validation - Grounded part movement fails validation - Baseline update (`savePlacementsForUndo()`) resets the comparison point - Cumulative small rotations don't false-positive when baseline is updated per step ## Proposed test additions ### Priority 1 — Regression test for #338 Add `test_planar_cylindrical_drag.py` (or extend `test_joints.py`) with: ```python def test_cylindrical_plus_planar_drag_360(): """Cylindrical + Planar: full rotation drag must keep body on plane.""" # Setup: ground + free body, Cylindrical on Z + Planar on XY (distance=0) # Drag: 36 steps of 10 degrees each around Z axis # Assert: at every step, body Z position == 0 (within tolerance) # Assert: at every step, residuals < 1e-8 # Assert: solver converges at every step ``` ### Priority 2 — Drag protocol integration tests Add `test_drag_protocol.py` exercising the Python solver's drag API: - `test_drag_basic_flow` — pre_drag/drag_step/post_drag lifecycle - `test_drag_step_convergence` — every step converges - `test_drag_residuals_stay_satisfied` — residuals < tol at each step - `test_drag_50_steps_smooth_arc` — long sequence stability ### Priority 3 — Combined joint drag tests Extend joint tests with drag sequences for: - Revolute + Planar (1 DOF rotation on plane) - Cylindrical + Planar (2 DOF: rotation + translation along axis, constrained to plane) - Ball + Planar (3 DOF rotation, constrained to plane) ### Priority 4 — Half-space tracking under drag Extend `test_preference.py` with drag-sequence half-space tests: - `test_planar_halfspace_no_correction_on_plane` — verify correction_fn=None for distance=0 - `test_halfspace_indicator_stable_across_drag` — sign doesn't flip during smooth drag - `test_halfspace_correction_off_plane` — correction fires correctly for offset planar constraints ### Priority 5 — C++ validateNewPlacements unit tests If feasible via pybind11/FreeCADCmd: - `test_validate_small_rotation_passes` - `test_validate_large_rotation_rejected` - `test_validate_grounded_movement_rejected` - `test_validate_baseline_update_resets_comparison` ## Related - #338 — The planar constraint drift bug this would have caught - #254 — Original Planar joint implementation - #298 — JCS visual feedback and constraint drag robustness

forbes added the enhancement label 2026-02-27 15:03:08 +00:00

forbes referenced this issue from kindred/solver

2026-02-27 15:30:49 +00:00

test: add console test reproducing planar drag quaternion flip (#338) #39

forbes referenced this issue

2026-02-27 15:31:52 +00:00

chore: update solver submodule — add planar drag console test (#338) #340

forbes commented

2026-02-27 18:58:07 +00:00

Completed successfully.

forbes closed this issue

2026-02-27 18:58:07 +00:00