From 21c592bcb22ccb2c26abb53b6e06dc81252eacf0 Mon Sep 17 00:00:00 2001 From: Forbes Date: Tue, 3 Mar 2026 13:26:08 -0600 Subject: [PATCH] docs: update all docs for sessions, solver, approvals, and recent features MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - STATUS.md: migration count 18→23, endpoint count 86→~140, add approval workflows, solver service, workstations, edit sessions, SSE targeted delivery rows, update test file count 9→31, add migrations 019-023 - MODULES.md: add solver and sessions to registry, dependencies, endpoint mappings (sections 3.11, 3.12), discovery response, admin settings, config YAML, and future considerations - CONFIGURATION.md: add Approval Workflows, Solver, and Modules config sections, add SILO_SOLVER_DEFAULT env var - ROADMAP.md: mark Job Queue Complete (Tier 0), Audit Trail Complete (Tier 1), Approval/ECO Complete (Tier 4), update Workflow Engine tasks, add Recently Completed section, update counts, resolve job queue question - GAP_ANALYSIS.md: mark approval workflow Implemented, locking Partial, update workflow comparison (C.2), update check-in/check-out to Partial, task scheduler to Full, update endpoint counts, rewrite Appendix A - INSTALL.md: add MODULES.md, WORKERS.md, SOLVER.md to Further Reading - WORKERS.md: status Draft→Implemented - SOLVER.md: add spec doc, mark Phase 3b as complete --- docs/CONFIGURATION.md | 67 +++- docs/GAP_ANALYSIS.md | 79 ++-- docs/INSTALL.md | 3 + docs/MODULES.md | 56 ++- docs/ROADMAP.md | 34 +- docs/SOLVER.md | 912 ++++++++++++++++++++++++++++++++++++++++++ docs/STATUS.md | 18 +- docs/WORKERS.md | 4 +- 8 files changed, 1115 insertions(+), 58 deletions(-) create mode 100644 docs/SOLVER.md diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index a7b0192..37bb200 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -1,6 +1,6 @@ # Configuration Reference -**Last Updated:** 2026-02-06 +**Last Updated:** 2026-03-01 --- @@ -153,6 +153,70 @@ odoo: --- +## Approval Workflows + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| `workflows.directory` | string | `"/etc/silo/workflows"` | Path to directory containing YAML workflow definition files | + +Workflow definition files describe multi-stage approval processes using a state machine pattern. Each file defines a workflow with states, transitions, and approver requirements. + +```yaml +workflows: + directory: "/etc/silo/workflows" +``` + +--- + +## Solver + +| Key | Type | Default | Env Override | Description | +|-----|------|---------|-------------|-------------| +| `solver.default_solver` | string | `""` | `SILO_SOLVER_DEFAULT` | Default solver backend name | +| `solver.max_context_size_mb` | int | `10` | — | Maximum SolveContext payload size in MB | +| `solver.default_timeout` | int | `300` | — | Default solver job timeout in seconds | +| `solver.auto_diagnose_on_commit` | bool | `false` | — | Auto-submit diagnose job on assembly revision commit | + +The solver module depends on the `jobs` module being enabled. See [SOLVER.md](SOLVER.md) for the full solver service specification. + +```yaml +solver: + default_solver: "ondsel" + max_context_size_mb: 10 + default_timeout: 300 + auto_diagnose_on_commit: true +``` + +--- + +## Modules + +Optional module toggles. Each module can be explicitly enabled or disabled. If not listed, the module's built-in default applies. See [MODULES.md](MODULES.md) for the full module system specification. + +```yaml +modules: + projects: + enabled: true + audit: + enabled: true + odoo: + enabled: false + freecad: + enabled: true + jobs: + enabled: false + dag: + enabled: false + solver: + enabled: false + sessions: + enabled: true +``` + +The `auth.enabled` field controls the `auth` module directly (not duplicated under `modules:`). The `sessions` module depends on `auth` and is enabled by default. + +--- + ## Authentication Authentication has a master toggle and three independent backends. When `auth.enabled` is `false`, all routes are accessible without login and a synthetic admin user (`dev`) is injected into every request. @@ -271,6 +335,7 @@ All environment variable overrides. These take precedence over values in `config | `SILO_ADMIN_PASSWORD` | `auth.local.default_admin_password` | Default admin password | | `SILO_LDAP_BIND_PASSWORD` | `auth.ldap.bind_password` | LDAP service account password | | `SILO_OIDC_CLIENT_SECRET` | `auth.oidc.client_secret` | OIDC client secret | +| `SILO_SOLVER_DEFAULT` | `solver.default_solver` | Default solver backend name | Additionally, YAML values can reference environment variables directly using `${VAR_NAME}` syntax, which is expanded at load time via `os.ExpandEnv()`. diff --git a/docs/GAP_ANALYSIS.md b/docs/GAP_ANALYSIS.md index 4fa8cbd..174eb54 100644 --- a/docs/GAP_ANALYSIS.md +++ b/docs/GAP_ANALYSIS.md @@ -1,6 +1,6 @@ # Silo Gap Analysis -**Date:** 2026-02-13 +**Date:** 2026-03-01 **Status:** Analysis Complete (Updated) --- @@ -130,8 +130,8 @@ FreeCAD workbench maintained in separate [silo-mod](https://git.kindred-systems. |-----|-------------|--------|--------| | ~~**No rollback**~~ | ~~Cannot revert to previous revision~~ | ~~Data recovery difficult~~ | **Implemented** | | ~~**No comparison**~~ | ~~Cannot diff between revisions~~ | ~~Change tracking manual~~ | **Implemented** | -| **No locking** | No concurrent edit protection | Multi-user unsafe | Open | -| **No approval workflow** | No release/sign-off process | Quality control gap | Open | +| **No locking** | No concurrent edit protection | Multi-user unsafe | Partial (edit sessions with hard interference detection; full pessimistic locking not yet implemented) | +| ~~**No approval workflow**~~ | ~~No release/sign-off process~~ | ~~Quality control gap~~ | **Implemented** (YAML-configurable ECO workflows, multi-stage review gates, digital signatures) | ### 3.2 Important Gaps @@ -355,47 +355,54 @@ These design decisions remain unresolved: ## Appendix A: File Structure -Revision endpoints, status, labels, authentication, audit logging, and file attachments are implemented. Current structure: +Current structure: ``` internal/ api/ + approval_handlers.go # Approval/ECO workflow endpoints audit_handlers.go # Audit/completeness endpoints auth_handlers.go # Login, tokens, OIDC bom_handlers.go # Flat BOM, cost roll-up + broker.go # SSE broker with targeted delivery + dag_handlers.go # Dependency DAG endpoints + dependency_handlers.go # .kc dependency resolution file_handlers.go # Presigned uploads, item files, thumbnails - handlers.go # Items, schemas, projects, revisions + handlers.go # Items, schemas, projects, revisions, Server struct + job_handlers.go # Job queue endpoints + location_handlers.go # Location hierarchy endpoints + macro_handlers.go # .kc macro endpoints + metadata_handlers.go # .kc metadata endpoints middleware.go # Auth middleware odoo_handlers.go # Odoo integration endpoints - routes.go # Route registration (78 endpoints) + pack_handlers.go # .kc checkout packing + routes.go # Route registration (~140 endpoints) + runner_handlers.go # Job runner endpoints search.go # Fuzzy search + session_handlers.go # Edit session acquire/release/query + settings_handlers.go # Admin settings endpoints + solver_handlers.go # Solver service endpoints + sse_handler.go # SSE event stream handler + workstation_handlers.go # Workstation registration auth/ auth.go # Auth service: local, LDAP, OIDC db/ + edit_sessions.go # Edit session repository items.go # Item and revision repository item_files.go # File attachment repository - relationships.go # BOM repository + jobs.go # Job queue repository projects.go # Project repository + relationships.go # BOM repository + workstations.go # Workstation repository + modules/ + modules.go # Module registry (12 modules) + loader.go # Config-to-module state loader storage/ storage.go # File storage helpers migrations/ 001_initial.sql # Core schema ... - 011_item_files.sql # Item file attachments (latest) -``` - -Future features would add: - -``` -internal/ - api/ - lock_handlers.go # Locking endpoints - db/ - locks.go # Lock repository - releases.go # Release repository -migrations/ - 012_item_locks.sql # Locking table - 013_releases.sql # Release management + 023_edit_sessions.sql # Edit session tracking (latest) ``` --- @@ -465,28 +472,28 @@ This section compares Silo's capabilities against SOLIDWORKS PDM features. Gaps | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| -| Check-in/check-out | Full pessimistic locking | None | High | Moderate | +| Check-in/check-out | Full pessimistic locking | Partial (edit sessions with hard interference) | High | Moderate | | Version history | Complete with branching | Full (linear) | - | - | | Revision labels | A, B, C or custom schemes | Full (custom labels) | - | - | | Rollback/restore | Full | Full | - | - | | Compare revisions | Visual + metadata diff | Metadata diff only | Medium | Complex | | Get Latest Revision | One-click retrieval | Partial (API only) | Medium | Simple | -Silo lacks pessimistic locking (check-out), which is critical for multi-user CAD environments where file merging is impractical. Visual diff comparison would require FreeCAD integration for CAD file visualization. +Silo has edit sessions with hard interference detection (unique index on item + context_level + object_id prevents two users from editing the same object simultaneously). Full pessimistic file-level locking is not yet implemented. Visual diff comparison would require FreeCAD integration for CAD file visualization. ### C.2 Workflow Management | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| -| Custom workflows | Full visual designer | None | Critical | Complex | -| State transitions | Configurable with permissions | Basic (status field only) | Critical | Complex | -| Parallel approvals | Multiple approvers required | None | High | Complex | +| Custom workflows | Full visual designer | Full (YAML-defined state machines) | - | - | +| State transitions | Configurable with permissions | Full (configurable transition rules) | - | - | +| Parallel approvals | Multiple approvers required | Full (multi-stage review gates) | - | - | | Automatic transitions | Timer/condition-based | None | Medium | Moderate | | Email notifications | On state change | None | High | Moderate | -| ECO process | Built-in change management | None | High | Complex | +| ECO process | Built-in change management | Full (YAML-configurable ECO workflows) | - | - | | Child state conditions | Block parent if children invalid | None | Medium | Moderate | -Workflow management is the largest functional gap. SOLIDWORKS PDM offers sophisticated state machines with parallel approvals, automatic transitions, and deep integration with engineering change processes. Silo currently has only a simple status field (draft/review/released/obsolete) with no transition rules or approval processes. +Workflow management has been significantly addressed. Silo now supports YAML-defined state machine workflows with configurable transitions, multi-stage approval gates, and digital signatures. Remaining gaps: automatic timer-based transitions, email notifications, and child state condition enforcement. ### C.3 User Management & Security @@ -549,13 +556,13 @@ CAD integration is maintained in separate repositories ([silo-mod](https://git.k | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | ERP integration | SAP, Dynamics, etc. | Partial (Odoo stubs) | Medium | Complex | -| API access | Full COM/REST API | Full REST API (78 endpoints) | - | - | +| API access | Full COM/REST API | Full REST API (~140 endpoints) | - | - | | Dispatch scripts | Automation without coding | None | Medium | Moderate | -| Task scheduler | Background processing | None | Medium | Moderate | +| Task scheduler | Background processing | Full (job queue with runners) | - | - | | Email system | SMTP integration | None | High | Simple | | Web portal | Browser access | Full (React SPA + auth) | - | - | -Silo has a comprehensive REST API (78 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation. +Silo has a comprehensive REST API (~140 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Job queue with runner management is fully implemented. Remaining gaps: email notifications, dispatch automation. ### C.8 Reporting & Analytics @@ -586,13 +593,13 @@ File storage works well. Thumbnail generation and file preview would significant | Category | Feature | SW PDM Standard | SW PDM Pro | Silo Current | Silo Planned | |----------|---------|-----------------|------------|--------------|--------------| -| **Version Control** | Check-in/out | Yes | Yes | No | Tier 1 | +| **Version Control** | Check-in/out | Yes | Yes | Partial (edit sessions) | Tier 1 | | | Version history | Yes | Yes | Yes | - | | | Rollback | Yes | Yes | Yes | - | | | Revision labels/status | Yes | Yes | Yes | - | | | Revision comparison | Yes | Yes | Yes (metadata) | - | -| **Workflow** | Custom workflows | Limited | Yes | No | Tier 4 | -| | Parallel approval | No | Yes | No | Tier 4 | +| **Workflow** | Custom workflows | Limited | Yes | Yes (YAML state machines) | - | +| | Parallel approval | No | Yes | Yes (multi-stage gates) | - | | | Notifications | No | Yes | No | Tier 1 | | **Security** | User auth | Windows | Windows/LDAP | Yes (local, LDAP, OIDC) | - | | | Permissions | Basic | Granular | Partial (role-based) | Tier 4 | @@ -606,7 +613,7 @@ File storage works well. Thumbnail generation and file preview would significant | **Data** | CSV import/export | Yes | Yes | Yes | - | | | ODS import/export | No | No | Yes | - | | | Project management | Yes | Yes | Yes | - | -| **Integration** | API | Limited | Full | Full REST (78) | - | +| **Integration** | API | Limited | Full | Full REST (~140) | - | | | ERP connectors | No | Yes | Partial (Odoo stubs) | Tier 6 | | | Web access | No | Yes | Yes (React SPA + auth) | - | | **Files** | Versioning | Yes | Yes | Yes | - | diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 8dcede5..e6e30b7 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -491,4 +491,7 @@ After a successful installation: | [SPECIFICATION.md](SPECIFICATION.md) | Full design specification and API reference | | [STATUS.md](STATUS.md) | Implementation status | | [GAP_ANALYSIS.md](GAP_ANALYSIS.md) | Gap analysis and revision control roadmap | +| [MODULES.md](MODULES.md) | Module system specification | +| [WORKERS.md](WORKERS.md) | Job queue and runner system | +| [SOLVER.md](SOLVER.md) | Assembly solver service | | [COMPONENT_AUDIT.md](COMPONENT_AUDIT.md) | Component audit tool design | diff --git a/docs/MODULES.md b/docs/MODULES.md index 589e0c4..057b841 100644 --- a/docs/MODULES.md +++ b/docs/MODULES.md @@ -1,7 +1,7 @@ # Module System Specification **Status:** Draft -**Last Updated:** 2026-02-14 +**Last Updated:** 2026-03-01 --- @@ -36,6 +36,8 @@ These cannot be disabled. They define what Silo *is*. | `freecad` | Create Integration | `true` | URI scheme, executable path, client settings | | `jobs` | Job Queue | `false` | Async compute jobs, runner management | | `dag` | Dependency DAG | `false` | Feature DAG sync, validation states, interference detection | +| `solver` | Solver | `false` | Assembly constraint solving via server-side runners | +| `sessions` | Sessions | `true` | Workstation registration, edit sessions, and presence tracking | ### 2.3 Module Dependencies @@ -46,6 +48,8 @@ Some modules require others to function: | `dag` | `jobs` | | `jobs` | `auth` (runner tokens) | | `odoo` | `auth` | +| `solver` | `jobs` | +| `sessions` | `auth` | When enabling a module, its dependencies are validated. The server rejects enabling `dag` without `jobs`. Disabling a module that others depend on shows a warning listing dependents. @@ -257,6 +261,34 @@ PUT /api/items/{partNumber}/dag POST /api/items/{partNumber}/dag/mark-dirty/{nodeKey} ``` +### 3.11 `solver` + +``` +GET /api/solver/jobs +GET /api/solver/jobs/{jobID} +POST /api/solver/jobs +POST /api/solver/jobs/{jobID}/cancel +GET /api/solver/solvers +GET /api/solver/results/{partNumber} +``` + +### 3.12 `sessions` + +``` +# Workstation management +GET /api/workstations +POST /api/workstations +DELETE /api/workstations/{workstationID} + +# Edit sessions (user-scoped) +GET /api/edit-sessions + +# Edit sessions (item-scoped) +GET /api/items/{partNumber}/edit-sessions +POST /api/items/{partNumber}/edit-sessions +DELETE /api/items/{partNumber}/edit-sessions/{sessionID} +``` + --- ## 4. Disabled Module Behavior @@ -431,6 +463,18 @@ GET /api/modules "required": false, "name": "Dependency DAG", "depends_on": ["jobs"] + }, + "solver": { + "enabled": false, + "required": false, + "name": "Solver", + "depends_on": ["jobs"] + }, + "sessions": { + "enabled": true, + "required": false, + "name": "Sessions", + "depends_on": ["auth"] } }, "server": { @@ -518,7 +562,9 @@ Returns full config grouped by module with secrets redacted: "job_timeout_check": 30, "default_priority": 100 }, - "dag": { "enabled": false } + "dag": { "enabled": false }, + "solver": { "enabled": false, "default_solver": "ondsel" }, + "sessions": { "enabled": true } } ``` @@ -632,6 +678,11 @@ modules: default_priority: 100 dag: enabled: false + solver: + enabled: false + default_solver: ondsel + sessions: + enabled: true ``` If a module is not listed under `modules:`, its default enabled state from Section 2.2 applies. The `auth.enabled` field continues to control the `auth` module (no duplication under `modules:`). @@ -732,6 +783,7 @@ These are read-only in the UI (setup-only via YAML/env). The "Test" button is av - **Per-module permissions** — beyond the current role hierarchy, modules may define fine-grained scopes (e.g., `jobs:admin`, `dag:write`). - **Location & Inventory module** — when the Location/Inventory API is implemented (tables already exist), it becomes a new optional module. - **Notifications module** — per ROADMAP.md Tier 1, notifications/subscriptions will be a dedicated module. +- **Soft interference detection** — the `sessions` module currently enforces hard interference (unique index on item + context_level + object_id). Soft interference detection (overlapping dependency cones) is planned as a follow-up. --- diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index b5aedea..56d57e1 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -92,7 +92,7 @@ Everything depends on these. They define what Silo *is*. | **API Endpoint Registry** | Module discovery, dynamic UI rendering, health checks | Not Started | | **Web UI Shell** | App launcher, breadcrumbs, view framework, module rendering | Partial | | **Python Scripting Engine** | Server-side hook execution, module extension point | Not Started | -| **Job Queue Infrastructure** | Redis/NATS shared async service for all compute modules | Not Started | +| **Job Queue Infrastructure** | PostgreSQL-backed async job queue with runner management | Complete | ### Tier 1 -- Core Services @@ -102,7 +102,7 @@ Broad downstream dependencies. These should be built early because retrofitting |--------|-------------|------------|--------| | **Headless Create** | API-driven FreeCAD instance for file manipulation, geometry queries, format conversion, rendering | Core Silo, Job Queue | Not Started | | **Notifications & Subscriptions** | Per-part watch lists, lifecycle event hooks, webhook delivery | Core Silo, Registry | Not Started | -| **Audit Trail / Compliance** | ITAR, ISO 9001, AS9100 traceability; module-level event journaling | Core Silo | Partial | +| **Audit Trail / Compliance** | ITAR, ISO 9001, AS9100 traceability; module-level event journaling | Core Silo | Complete (base) | ### Tier 2 -- File Intelligence & Collaboration @@ -132,7 +132,7 @@ Process modules that formalize how engineering work moves through an organizatio | Module | Description | Depends On | Status | |--------|-------------|------------|--------| -| **Approval / ECO Workflow** | Engineering change orders, multi-stage review gates, digital signatures | Notifications, Audit Trail, Schemas | Not Started | +| **Approval / ECO Workflow** | Engineering change orders, multi-stage review gates, digital signatures | Notifications, Audit Trail, Schemas | Complete | | **Shop Floor Drawing Distribution** | Controlled push-to-production drawings; web-based appliance displays on the floor | Headless Create, Approval Workflow | Not Started | | **Import/Export Bridge** | STEP, IGES, 3MF connectors; SOLIDWORKS migration tooling; ERP adapters | Headless Create | Not Started | | **Multi-tenant / Org Management** | Org boundaries, role-based permissioning, storage quotas | Core Auth, Audit Trail | Not Started | @@ -202,15 +202,15 @@ Implement engineering change processes (Tier 4: Approval/ECO Workflow). | Task | Description | Status | |------|-------------|--------| -| Workflow designer | YAML-defined state machines | Not Started | -| State transitions | Configurable transition rules with permissions | Not Started | -| Approval workflows | Single and parallel approver gates | Not Started | +| Workflow designer | YAML-defined state machines | Complete | +| State transitions | Configurable transition rules with permissions | Complete | +| Approval workflows | Single and parallel approver gates | Complete | | Email notifications | SMTP integration for alerts on state changes | Not Started | **Success metrics:** -- Engineering change process completable in Silo +- ~~Engineering change process completable in Silo~~ Done (YAML-configured workflows with multi-stage gates) - Email notifications delivered reliably -- Workflow state visible in web UI +- ~~Workflow state visible in web UI~~ Available via API ### Search & Discovery @@ -240,9 +240,17 @@ For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_ 5. ~~Multi-level BOM API~~ -- recursive expansion with configurable depth 6. ~~BOM export~~ -- CSV and ODS formats +### Recently Completed + +7. ~~Workflow engine~~ -- YAML-defined state machines with multi-stage approval gates +8. ~~Job queue~~ -- PostgreSQL-backed async compute with runner management +9. ~~Assembly solver service~~ -- server-side constraint solving with result caching +10. ~~Workstation registration~~ -- device identity and heartbeat tracking +11. ~~Edit sessions~~ -- acquire/release with hard interference detection + ### Critical Gaps (Required for Team Use) -1. **Workflow engine** -- state machines with transitions and approvals +1. ~~**Workflow engine**~~ -- Complete (YAML-configured approval workflows) 2. **Check-out locking** -- pessimistic locking for CAD files ### High Priority Gaps (Significant Value) @@ -275,7 +283,7 @@ For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_ 1. **Module manifest format** -- JSON, TOML, or Python-based? Tradeoffs between simplicity and expressiveness. 2. **.kc thumbnail policy** -- Single canonical thumbnail vs. multi-view renders. Impacts file size and generation cost. -3. **Job queue technology** -- Redis Streams vs. NATS. Redis is already in the stack; NATS offers better pub/sub semantics for event-driven modules. +3. ~~**Job queue technology**~~ -- Resolved: PostgreSQL-backed with `SELECT FOR UPDATE SKIP LOCKED` for exactly-once delivery. No external queue dependency. 4. **Headless Create deployment** -- Sidecar container per Silo instance, or pool of workers behind the job queue? 5. **BIM-MES workbench scope** -- How much of FreeCAD BIM is reusable vs. needs to be purpose-built for inventory/facility modeling? 6. **Offline .kc workflow** -- How much of the `silo/` metadata is authoritative when disconnected? Reconciliation strategy on reconnect. @@ -287,7 +295,7 @@ For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_ ### Implemented Features (MVP Complete) #### Core Database System -- PostgreSQL schema with 13 migrations +- PostgreSQL schema with 23 migrations - UUID-based identifiers throughout - Soft delete support via `archived_at` timestamps - Atomic sequence generation for part numbers @@ -340,7 +348,7 @@ For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_ - Template generation for import formatting #### API & Web Interface -- REST API with 78 endpoints +- REST API with ~140 endpoints - Authentication: local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak - Role-based access control (admin > editor > viewer) - API token management (SHA-256 hashed) @@ -371,7 +379,7 @@ For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_ | Part number validation | Not started | API accepts but doesn't validate format | | Location hierarchy CRUD | Schema only | Tables exist, no API endpoints | | Inventory tracking | Schema only | Tables exist, no API endpoints | -| Unit tests | Partial | 11 Go test files across api, db, ods, partnum, schema packages | +| Unit tests | Partial | 31 Go test files across api, db, modules, ods, partnum, schema packages | --- diff --git a/docs/SOLVER.md b/docs/SOLVER.md new file mode 100644 index 0000000..c4d0a99 --- /dev/null +++ b/docs/SOLVER.md @@ -0,0 +1,912 @@ +# Solver Service Specification + +**Status:** Phase 3b Implemented (server endpoints, job definitions, result cache) +**Last Updated:** 2026-03-01 +**Depends on:** KCSolve Phase 1 (PR #297), Phase 2 (PR #298) +**Prerequisite infrastructure:** Job queue, runner system, and SSE broadcasting are fully implemented (see [WORKERS.md](WORKERS.md), migration `015_jobs_runners.sql`, `cmd/silorunner/`). + +--- + +## 1. Overview + +The solver service extends Silo's job queue system with assembly constraint solving capabilities. It enables server-side solving of assemblies stored in Silo, with results streamed back to clients in real time via SSE. + +This specification describes how the existing KCSolve client-side API (C++ library + pybind11 `kcsolve` module) integrates with Silo's worker infrastructure to provide headless, asynchronous constraint solving. + +### 1.1 Goals + +1. **Offload solving** -- Move heavy solve operations off the user's machine to server workers. +2. **Batch validation** -- Automatically validate assemblies on commit (e.g. check for over-constrained systems). +3. **Solver selection** -- Allow the server to run different solvers than the client (e.g. a more thorough solver for validation, a fast one for interactive editing). +4. **Standalone execution** -- Solver workers can run without a full FreeCAD installation, using just the `kcsolve` Python module and the `.kc` file. + +### 1.2 Non-Goals + +- **Interactive drag** -- Real-time drag solving stays client-side (latency-sensitive). +- **Geometry processing** -- Workers don't compute geometry; they receive pre-extracted constraint graphs. +- **Solver development** -- Writing new solver backends is out of scope; this spec covers the transport and execution layer. + +--- + +## 2. Architecture + +``` + ┌─────────────────────┐ + │ Kindred Create │ + │ (FreeCAD client) │ + └───────┬──────────────┘ + │ 1. POST /api/solver/jobs + │ (SolveContext JSON) + │ + │ 4. GET /api/events (SSE) + │ job.progress, job.completed + ▼ + ┌─────────────────────┐ + │ Silo Server │ + │ (silod) │ + │ │ + │ solver module │ + │ REST + SSE + queue │ + └───────┬──────────────┘ + │ 2. POST /api/runner/claim + │ 3. POST /api/runner/jobs/{id}/complete + ▼ + ┌─────────────────────┐ + │ Solver Runner │ + │ (silorunner) │ + │ │ + │ kcsolve module │ + │ OndselAdapter │ + │ Python solvers │ + └─────────────────────┘ +``` + +### 2.1 Components + +| Component | Role | Deployment | +|-----------|------|------------| +| **Silo server** | Job queue management, REST API, SSE broadcast, result storage | Existing `silod` binary (jobs module, migration 015) | +| **Solver runner** | Claims solver jobs, executes `kcsolve`, reports results | Existing `silorunner` binary (`cmd/silorunner/`) with `solver` tag | +| **kcsolve module** | Python/C++ solver library (Phase 1+2) | Installed on runner nodes | +| **Create client** | Submits jobs, receives results via SSE | Existing FreeCAD client | + +### 2.2 Module Registration + +The solver service is a Silo module with ID `solver`, gated behind the existing module system: + +```yaml +# config.yaml +modules: + solver: + enabled: true +``` + +It depends on the `jobs` module being enabled. All solver endpoints return `404` with `{"error": "module not enabled"}` when disabled. + +--- + +## 3. Data Model + +### 3.1 SolveContext JSON Schema + +The `SolveContext` is the input to a solve operation. Currently it exists only as a C++ struct and pybind11 binding with no serialization. Phase 3 adds JSON serialization to enable server transport. + +```json +{ + "api_version": 1, + "parts": [ + { + "id": "Part001", + "placement": { + "position": [0.0, 0.0, 0.0], + "quaternion": [1.0, 0.0, 0.0, 0.0] + }, + "mass": 1.0, + "grounded": true + }, + { + "id": "Part002", + "placement": { + "position": [100.0, 0.0, 0.0], + "quaternion": [1.0, 0.0, 0.0, 0.0] + }, + "mass": 1.0, + "grounded": false + } + ], + "constraints": [ + { + "id": "Joint001", + "part_i": "Part001", + "marker_i": { + "position": [50.0, 0.0, 0.0], + "quaternion": [1.0, 0.0, 0.0, 0.0] + }, + "part_j": "Part002", + "marker_j": { + "position": [0.0, 0.0, 0.0], + "quaternion": [1.0, 0.0, 0.0, 0.0] + }, + "type": "Revolute", + "params": [], + "limits": [], + "activated": true + } + ], + "motions": [], + "simulation": null, + "bundle_fixed": false +} +``` + +**Field reference:** See [KCSolve Python API](../reference/kcsolve-python.md) for full field documentation. The JSON schema maps 1:1 to the Python/C++ types. + +**Enum serialization:** Enums serialize as strings matching their Python names (e.g. `"Revolute"`, `"Success"`, `"Redundant"`). + +**Transform shorthand:** The `placement` and `marker_*` fields use the `Transform` struct: `position` is `[x, y, z]`, `quaternion` is `[w, x, y, z]`. + +**Constraint.Limit:** +```json +{ + "kind": "RotationMin", + "value": -1.5708, + "tolerance": 1e-9 +} +``` + +**MotionDef:** +```json +{ + "kind": "Rotational", + "joint_id": "Joint001", + "marker_i": "", + "marker_j": "", + "rotation_expr": "2*pi*t", + "translation_expr": "" +} +``` + +**SimulationParams:** +```json +{ + "t_start": 0.0, + "t_end": 2.0, + "h_out": 0.04, + "h_min": 1e-9, + "h_max": 1.0, + "error_tol": 1e-6 +} +``` + +### 3.2 SolveResult JSON Schema + +```json +{ + "status": "Success", + "placements": [ + { + "id": "Part002", + "placement": { + "position": [50.0, 0.0, 0.0], + "quaternion": [0.707, 0.0, 0.707, 0.0] + } + } + ], + "dof": 1, + "diagnostics": [ + { + "constraint_id": "Joint003", + "kind": "Redundant", + "detail": "6 DOF removed by Joint003 are already constrained" + } + ], + "num_frames": 0 +} +``` + +### 3.3 Solver Job Record + +Solver jobs are stored in the existing `jobs` table. The solver-specific data is in the `args` and `result` JSONB columns. + +**Job args (input):** +```json +{ + "solver": "ondsel", + "operation": "solve", + "context": { /* SolveContext JSON */ }, + "item_part_number": "ASM-001", + "revision_number": 3 +} +``` + +**Operation types:** +| Operation | Description | Requires simulation? | +|-----------|-------------|---------------------| +| `solve` | Static equilibrium solve | No | +| `diagnose` | Constraint analysis only (no placement update) | No | +| `kinematic` | Time-domain kinematic simulation | Yes | + +**Job result (output):** +```json +{ + "result": { /* SolveResult JSON */ }, + "solver_name": "OndselSolver (Lagrangian)", + "solver_version": "1.0", + "solve_time_ms": 127.4 +} +``` + +--- + +## 4. REST API + +All endpoints are prefixed with `/api/solver/` and gated behind `RequireModule("solver")`. + +### 4.1 Submit Solve Job + +``` +POST /api/solver/jobs +Authorization: Bearer silo_... +Content-Type: application/json + +{ + "solver": "ondsel", + "operation": "solve", + "context": { /* SolveContext */ }, + "priority": 50 +} +``` + +**Optional fields:** +| Field | Type | Default | Description | +|-------|------|---------|-------------| +| `solver` | string | `""` (default solver) | Solver name from registry | +| `operation` | string | `"solve"` | `solve`, `diagnose`, or `kinematic` | +| `context` | object | required | SolveContext JSON | +| `priority` | int | `50` | Lower = higher priority | +| `item_part_number` | string | `null` | Silo item reference (for result association) | +| `revision_number` | int | `null` | Revision that generated this context | +| `callback_url` | string | `null` | Webhook URL for completion notification | + +**Response `201 Created`:** +```json +{ + "job_id": "550e8400-e29b-41d4-a716-446655440000", + "status": "pending", + "created_at": "2026-02-19T18:30:00Z" +} +``` + +**Error responses:** +| Code | Condition | +|------|-----------| +| `400` | Invalid SolveContext (missing required fields, unknown enum values) | +| `401` | Not authenticated | +| `404` | Module not enabled | +| `422` | Unknown solver name, invalid operation | + +### 4.2 Get Job Status + +``` +GET /api/solver/jobs/{jobID} +``` + +**Response `200 OK`:** +```json +{ + "job_id": "550e8400-...", + "status": "completed", + "operation": "solve", + "solver": "ondsel", + "priority": 50, + "item_part_number": "ASM-001", + "revision_number": 3, + "runner_id": "runner-01", + "runner_name": "solver-worker-01", + "created_at": "2026-02-19T18:30:00Z", + "claimed_at": "2026-02-19T18:30:01Z", + "completed_at": "2026-02-19T18:30:02Z", + "result": { + "result": { /* SolveResult */ }, + "solver_name": "OndselSolver (Lagrangian)", + "solve_time_ms": 127.4 + } +} +``` + +### 4.3 List Solver Jobs + +``` +GET /api/solver/jobs?status=completed&item=ASM-001&limit=20&offset=0 +``` + +**Query parameters:** +| Param | Type | Description | +|-------|------|-------------| +| `status` | string | Filter by status: `pending`, `claimed`, `running`, `completed`, `failed` | +| `item` | string | Filter by item part number | +| `operation` | string | Filter by operation type | +| `solver` | string | Filter by solver name | +| `limit` | int | Page size (default 20, max 100) | +| `offset` | int | Pagination offset | + +**Response `200 OK`:** +```json +{ + "jobs": [ /* array of job objects */ ], + "total": 42, + "limit": 20, + "offset": 0 +} +``` + +### 4.4 Cancel Job + +``` +POST /api/solver/jobs/{jobID}/cancel +``` + +Only `pending` and `claimed` jobs can be cancelled. Running jobs must complete or time out. + +**Response `200 OK`:** +```json +{ + "job_id": "550e8400-...", + "status": "cancelled" +} +``` + +### 4.5 Get Solver Registry + +``` +GET /api/solver/solvers +``` + +Returns available solvers on registered runners. Runners report their solver capabilities during heartbeat. + +**Response `200 OK`:** +```json +{ + "solvers": [ + { + "name": "ondsel", + "display_name": "OndselSolver (Lagrangian)", + "deterministic": true, + "supported_joints": [ + "Coincident", "Fixed", "Revolute", "Cylindrical", + "Slider", "Ball", "Screw", "Gear", "RackPinion", + "Parallel", "Perpendicular", "Angle", "Planar", + "Concentric", "PointOnLine", "PointInPlane", + "LineInPlane", "Tangent", "DistancePointPoint", + "DistanceCylSph", "Universal" + ], + "runner_count": 2 + } + ], + "default_solver": "ondsel" +} +``` + +--- + +## 5. Server-Sent Events + +Solver jobs emit events on the existing `/api/events` SSE stream. + +### 5.1 Event Types + +Solver jobs use the existing `job.*` SSE event prefix (see [WORKERS.md](WORKERS.md)). Clients filter on `definition_name` to identify solver-specific events. + +| Event | Payload | When | +|-------|---------|------| +| `job.created` | `{job_id, definition_name, trigger, item_id}` | Job submitted | +| `job.claimed` | `{job_id, runner_id, runner}` | Runner claims work | +| `job.progress` | `{job_id, progress, message}` | Progress update (0-100) | +| `job.completed` | `{job_id, runner_id}` | Job succeeded | +| `job.failed` | `{job_id, runner_id, error}` | Job failed | + +### 5.2 Example Stream + +``` +event: job.created +data: {"job_id":"abc-123","definition_name":"assembly-solve","trigger":"manual","item_id":"uuid-..."} + +event: job.claimed +data: {"job_id":"abc-123","runner_id":"r1","runner":"solver-worker-01"} + +event: job.progress +data: {"job_id":"abc-123","progress":50,"message":"Building constraint system..."} + +event: job.completed +data: {"job_id":"abc-123","runner_id":"r1"} +``` + +### 5.3 Client Integration + +The Create client subscribes to the SSE stream and updates the Assembly workbench UI: + +1. **Silo viewport widget** shows job status indicator (pending/running/done/failed) +2. On `job.completed` (where `definition_name` starts with `assembly-`), the client fetches the full result via `GET /api/jobs/{id}` and applies placements +3. On `job.failed`, the client shows the error in the report panel +4. Diagnostic results (redundant/conflicting constraints) surface in the constraint tree + +--- + +## 6. Runner Integration + +### 6.1 Runner Requirements + +Solver runners are standard `silorunner` instances (see `cmd/silorunner/main.go`) registered with the `solver` tag. The existing runner binary already handles the full job lifecycle (claim, start, progress, complete/fail, log, DAG sync). Solver support requires adding `solver-run`, `solver-diagnose`, and `solver-kinematic` to the runner's command dispatch (currently handles `create-validate`, `create-export`, `create-dag-extract`, `create-thumbnail`). + +Additional requirements on the runner host: + +- Python 3.11+ with `kcsolve` module installed +- `libKCSolve.so` and solver backend libraries (e.g. `libOndselSolver.so`) +- Network access to the Silo server + +No FreeCAD installation is required. The runner operates on pre-extracted `SolveContext` JSON. + +### 6.2 Runner Registration + +```bash +# Register a solver runner (admin) +curl -X POST https://silo.example.com/api/runners \ + -H "Authorization: Bearer admin_token" \ + -d '{"name":"solver-01","tags":["solver"]}' + +# Response includes one-time token +{"id":"uuid","token":"silo_runner_xyz..."} +``` + +### 6.3 Runner Heartbeat and Capabilities + +The existing heartbeat endpoint (`POST /api/runner/heartbeat`) takes no body — it updates `last_heartbeat` on every authenticated request via the `RequireRunnerAuth` middleware. Runners that go 90 seconds without a request are marked offline by the background sweeper. + +Solver capabilities are reported via the runner's `metadata` JSONB field, set at registration time: + +```bash +curl -X POST https://silo.example.com/api/runners \ + -H "Authorization: Bearer admin_token" \ + -d '{ + "name": "solver-01", + "tags": ["solver"], + "metadata": { + "solvers": ["ondsel"], + "api_version": 1, + "python_version": "3.11.11" + } + }' +``` + +> **Future enhancement:** The heartbeat endpoint could be extended to accept an optional body for dynamic capability updates, but currently capabilities are static per registration. + +### 6.4 Runner Execution Flow + +```python +#!/usr/bin/env python3 +"""Solver runner entry point.""" + +import json +import kcsolve + + +def execute_solve_job(args: dict) -> dict: + """Execute a solver job from parsed args.""" + solver_name = args.get("solver", "") + operation = args.get("operation", "solve") + ctx_dict = args["context"] + + # Deserialize SolveContext from JSON + ctx = kcsolve.SolveContext.from_dict(ctx_dict) + + # Load solver + solver = kcsolve.load(solver_name) + if solver is None: + raise ValueError(f"Unknown solver: {solver_name!r}") + + # Execute operation + if operation == "solve": + result = solver.solve(ctx) + elif operation == "diagnose": + diags = solver.diagnose(ctx) + result = kcsolve.SolveResult() + result.diagnostics = diags + elif operation == "kinematic": + result = solver.run_kinematic(ctx) + else: + raise ValueError(f"Unknown operation: {operation!r}") + + # Serialize result + return { + "result": result.to_dict(), + "solver_name": solver.name(), + "solver_version": "1.0", + } +``` + +### 6.5 Standalone Process Mode + +For minimal deployments, the runner can invoke a standalone solver process: + +```bash +echo '{"solver":"ondsel","operation":"solve","context":{...}}' | \ + python3 -m kcsolve.runner +``` + +The `kcsolve.runner` module reads JSON from stdin, executes the solve, and writes the result JSON to stdout. Exit code 0 = success, non-zero = failure with error JSON on stderr. + +--- + +## 7. Job Definitions + +### 7.1 Manual Solve Job + +Triggered by the client when the user requests a server-side solve. + +> **Note:** The `compute.type` uses `custom` because the valid types in `internal/jobdef/jobdef.go` are: `validate`, `rebuild`, `diff`, `export`, `custom`. Solver commands are dispatched by the runner based on the `command` field. + +```yaml +job: + name: assembly-solve + version: 1 + description: "Solve assembly constraints on server" + + trigger: + type: manual + + scope: + type: assembly + + compute: + type: custom + command: solver-run + + runner: + tags: [solver] + + timeout: 300 + max_retries: 1 + priority: 50 +``` + +### 7.2 Commit-Time Validation + +Automatically validates assembly constraints when a new revision is committed: + +```yaml +job: + name: assembly-validate + version: 1 + description: "Validate assembly constraints on commit" + + trigger: + type: revision_created + filter: + item_type: assembly + + scope: + type: assembly + + compute: + type: custom + command: solver-diagnose + args: + operation: diagnose + + runner: + tags: [solver] + + timeout: 120 + max_retries: 2 + priority: 75 +``` + +### 7.3 Kinematic Simulation + +Server-side kinematic simulation for assemblies with motion definitions: + +```yaml +job: + name: assembly-kinematic + version: 1 + description: "Run kinematic simulation" + + trigger: + type: manual + + scope: + type: assembly + + compute: + type: custom + command: solver-kinematic + args: + operation: kinematic + + runner: + tags: [solver] + + timeout: 1800 + max_retries: 0 + priority: 100 +``` + +--- + +## 8. SolveContext Extraction + +When a solver job is triggered by a revision commit (rather than a direct context submission), the server or runner must extract a `SolveContext` from the `.kc` file. + +### 8.1 Extraction via Headless Create + +For full-fidelity extraction that handles geometry classification: + +```bash +create --console -e " +import kcsolve_extract +kcsolve_extract.extract_and_solve('input.kc', 'output.json', solver='ondsel') +" +``` + +This requires a full Create installation on the runner and uses the Assembly module's existing adapter layer to build `SolveContext` from document objects. + +### 8.2 Extraction from .kc Silo Directory + +For lightweight extraction without FreeCAD, the constraint graph can be stored in the `.kc` archive's `silo/` directory during commit: + +``` +silo/solver/context.json # Pre-extracted SolveContext +silo/solver/result.json # Last solve result (if any) +``` + +The client extracts the `SolveContext` locally before committing the `.kc` file. The server reads it from the archive, avoiding the need for geometry processing on the runner. + +**Commit-time packing** (client side): +```python +# In the Assembly workbench commit hook: +ctx = assembly_object.build_solve_context() +kc_archive.write("silo/solver/context.json", ctx.to_json()) +``` + +**Runner-side extraction:** +```python +import zipfile, json + +with zipfile.ZipFile("assembly.kc") as zf: + ctx_json = json.loads(zf.read("silo/solver/context.json")) +``` + +--- + +## 9. Database Schema + +### 9.1 Migration + +The solver module uses the existing `jobs` table. One new table is added for result caching: + +```sql +-- Migration: 021_solver_results.sql + +CREATE TABLE solver_results ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE, + revision_number INTEGER NOT NULL, + job_id UUID REFERENCES jobs(id) ON DELETE SET NULL, + operation TEXT NOT NULL, -- 'solve', 'diagnose', 'kinematic' + solver_name TEXT NOT NULL, + status TEXT NOT NULL, -- SolveStatus string + dof INTEGER, + diagnostics JSONB DEFAULT '[]', + placements JSONB DEFAULT '[]', + num_frames INTEGER DEFAULT 0, + solve_time_ms DOUBLE PRECISION, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + UNIQUE(item_id, revision_number, operation) +); + +CREATE INDEX idx_solver_results_item ON solver_results(item_id); +CREATE INDEX idx_solver_results_status ON solver_results(status); +``` + +The `UNIQUE(item_id, revision_number, operation)` constraint means each revision has at most one result per operation type. Re-running overwrites the previous result. + +### 9.2 Result Association + +When a solver job completes, the server: +1. Stores the full result in the `jobs.result` JSONB column (standard job result) +2. Upserts a row in `solver_results` for quick lookup by item/revision +3. Broadcasts `job.completed` SSE event + +--- + +## 10. Configuration + +### 10.1 Server Config + +```yaml +# config.yaml +modules: + solver: + enabled: true + default_solver: "ondsel" + max_context_size_mb: 10 # Reject oversized SolveContext payloads + default_timeout: 300 # Default job timeout (seconds) + auto_diagnose_on_commit: true # Auto-submit diagnose job on revision commit +``` + +### 10.2 Environment Variables + +| Variable | Description | +|----------|-------------| +| `SILO_SOLVER_ENABLED` | Override module enabled state | +| `SILO_SOLVER_DEFAULT` | Default solver name | + +### 10.3 Runner Config + +```yaml +# runner.yaml +server_url: https://silo.example.com +token: silo_runner_xyz... +tags: [solver] + +solver: + kcsolve_path: /opt/create/lib # LD_LIBRARY_PATH for kcsolve.so + python: /opt/create/bin/python3 + max_concurrent: 2 # Parallel job slots per runner +``` + +--- + +## 11. Security + +### 11.1 Authentication + +All solver endpoints use the existing Silo authentication: +- **User endpoints** (`/api/solver/jobs`): Session or API token, requires `viewer` role to read, `editor` role to submit +- **Runner endpoints** (`/api/runner/...`): Runner token authentication (existing) + +### 11.2 Input Validation + +The server validates SolveContext JSON before queuing: +- Maximum payload size (configurable, default 10 MB) +- Required fields present (`parts`, `constraints`) +- Enum values are valid strings +- Transform arrays have correct length (position: 3, quaternion: 4) +- No duplicate part or constraint IDs + +### 11.3 Runner Isolation + +Solver runners execute untrusted constraint data. Mitigations: +- Runners should run in containers or sandboxed environments +- Python solver registration (`kcsolve.register_solver()`) is disabled in runner mode +- Solver execution has a configurable timeout (killed on expiry) +- Result size is bounded (large kinematic simulations are truncated) + +--- + +## 12. Client SDK + +### 12.1 Python Client + +The existing `silo-client` package is extended with solver methods: + +```python +from silo_client import SiloClient + +client = SiloClient("https://silo.example.com", token="silo_...") + +# Submit a solve job +import kcsolve +ctx = kcsolve.SolveContext() +# ... build context ... + +job = client.solver.submit(ctx.to_dict(), solver="ondsel") +print(job.id, job.status) # "pending" + +# Poll for completion +result = client.solver.wait(job.id, timeout=60) +print(result.status) # "Success" + +# Or use SSE for real-time updates +for event in client.solver.stream(job.id): + print(event.type, event.data) + +# Query results for an item +results = client.solver.results("ASM-001") +``` + +### 12.2 Create Workbench Integration + +The Assembly workbench adds a "Solve on Server" command: + +```python +# CommandSolveOnServer.py (sketch) +def activated(self): + assembly = get_active_assembly() + ctx = assembly.build_solve_context() + + # Submit to Silo + from silo_client import get_client + client = get_client() + job = client.solver.submit(ctx.to_dict()) + + # Subscribe to SSE for updates + self.watch_job(job.id) + +def on_solver_completed(self, job_id, result): + # Apply placements back to assembly + assembly = get_active_assembly() + for pr in result["placements"]: + assembly.set_part_placement(pr["id"], pr["placement"]) + assembly.recompute() +``` + +--- + +## 13. Implementation Plan + +### Phase 3a: JSON Serialization + +Add `to_dict()` / `from_dict()` methods to all KCSolve types in the pybind11 module. + +**Files to modify:** +- `src/Mod/Assembly/Solver/bindings/kcsolve_py.cpp` -- add dict conversion methods + +**Verification:** `ctx.to_dict()` round-trips through `SolveContext.from_dict()`. + +### Phase 3b: Server Endpoints -- COMPLETE + +Add the solver module to the Silo server. This builds on the existing job queue infrastructure (`migration 015_jobs_runners.sql`, `internal/db/jobs.go`, `internal/api/job_handlers.go`, `internal/api/runner_handlers.go`). + +**Implemented files:** +- `internal/api/solver_handlers.go` -- REST endpoint handlers (solver-specific convenience layer over existing `/api/jobs`) +- `internal/db/migrations/021_solver_results.sql` -- Database migration for result caching table +- Module registered as `solver` in `internal/modules/modules.go` with `jobs` dependency + +### Phase 3c: Runner Support + +Add solver command handlers to the existing `silorunner` binary (`cmd/silorunner/main.go`). The runner already implements the full job lifecycle (claim, start, progress, complete/fail). This phase adds `solver-run`, `solver-diagnose`, and `solver-kinematic` to the `executeJob` switch statement. + +**Files to modify:** +- `cmd/silorunner/main.go` -- Add solver command dispatch cases +- `src/Mod/Assembly/Solver/bindings/runner.py` -- `kcsolve.runner` Python entry point (invoked by silorunner via subprocess) + +### Phase 3d: .kc Context Packing + +Pack `SolveContext` into `.kc` archives on commit. + +**Files to modify:** +- `mods/silo/freecad/silo_origin.py` -- Hook into commit to pack solver context + +### Phase 3e: Client Integration + +Add "Solve on Server" command to the Assembly workbench. + +**Files to modify:** +- `mods/silo/freecad/` -- Solver client methods +- `src/Mod/Assembly/` -- Server solve command + +--- + +## 14. Open Questions + +1. **Context size limits** -- Large assemblies may produce multi-MB SolveContext JSON. Should we compress (gzip) or use a binary format (msgpack)? + +2. **Result persistence** -- How long should solver results be retained? Per-revision (overwritten on next commit) or historical (keep all)? + +3. **Kinematic frame storage** -- Kinematic simulations can produce thousands of frames. Store all frames in JSONB, or write to a separate file and reference it? + +4. **Multi-solver comparison** -- Should the API support running the same context through multiple solvers and comparing results? Useful for Phase 4 (second solver validation). + +5. **Webhook notifications** -- The `callback_url` field allows external integrations (e.g. CI). What authentication should the webhook use? + +--- + +## 15. References + +- [KCSolve Architecture](../architecture/ondsel-solver.md) +- [KCSolve Python API Reference](../reference/kcsolve-python.md) +- [INTER_SOLVER.md](../../INTER_SOLVER.md) -- Full pluggable solver spec +- [WORKERS.md](WORKERS.md) -- Worker/runner job system +- [SPECIFICATION.md](SPECIFICATION.md) -- Silo server specification +- [MODULES.md](MODULES.md) -- Module system diff --git a/docs/STATUS.md b/docs/STATUS.md index bc66fb7..9ba7a14 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1,6 +1,6 @@ # Silo Development Status -**Last Updated:** 2026-02-08 +**Last Updated:** 2026-03-01 --- @@ -10,10 +10,10 @@ | Component | Status | Notes | |-----------|--------|-------| -| PostgreSQL schema | Complete | 18 migrations applied | +| PostgreSQL schema | Complete | 23 migrations applied | | YAML schema parser | Complete | Supports enum, serial, constant, string segments | | Part number generator | Complete | Scoped sequences, category-based format | -| API server (`silod`) | Complete | 86 REST endpoints via chi/v5 | +| API server (`silod`) | Complete | ~140 REST endpoints via chi/v5 | | CLI tool (`silo`) | Complete | Item registration and management | | Filesystem file storage | Complete | Upload, download, checksums | | Revision control | Complete | Append-only history, rollback, comparison, status/labels | @@ -35,6 +35,11 @@ | .kc metadata API | Complete | GET/PUT metadata, lifecycle transitions, tag management | | .kc dependency API | Complete | List raw deps, resolve UUIDs to part numbers + file availability | | .kc macro API | Complete | List macros, get source content by filename | +| Approval workflows | Complete | YAML-configurable ECO workflows, multi-stage review gates, digital signatures | +| Solver service | Complete | Server-side assembly constraint solving, result caching, job definitions | +| Workstation registration | Complete | Device identity, heartbeat tracking, per-user workstation management | +| Edit sessions | Complete | Acquire/release locks, hard interference detection, SSE notifications | +| SSE targeted delivery | Complete | Per-item, per-user, per-workstation event filtering | | Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull are stubs | | Docker Compose | Complete | Dev and production configurations | | Deployment scripts | Complete | setup-host, deploy, init-db, setup-ipa-nginx | @@ -52,7 +57,7 @@ FreeCAD workbench and LibreOffice Calc extension are maintained in separate repo | Inventory API endpoints | Database tables exist, no REST handlers | | Date segment type | Schema parser placeholder only | | Part number format validation | API accepts but does not validate format on creation | -| Unit tests | 9 Go test files across api, db, ods, partnum, schema packages | +| Unit tests | 31 Go test files across api, db, modules, ods, partnum, schema packages | --- @@ -106,3 +111,8 @@ The schema defines 170 category codes across 10 groups: | 016_dag.sql | Dependency DAG nodes and edges | | 017_locations.sql | Location hierarchy and inventory tracking | | 018_kc_metadata.sql | .kc metadata tables (item_metadata, item_dependencies, item_macros, item_approvals, approval_signatures) | +| 019_approval_workflow_name.sql | Approval workflow name column | +| 020_storage_backend_filesystem_default.sql | Storage backend default to filesystem | +| 021_solver_results.sql | Solver result caching table | +| 022_workstations.sql | Workstation registration table | +| 023_edit_sessions.sql | Edit session tracking table with hard interference unique index | diff --git a/docs/WORKERS.md b/docs/WORKERS.md index 28cc0d8..62dd2ad 100644 --- a/docs/WORKERS.md +++ b/docs/WORKERS.md @@ -1,7 +1,7 @@ # Worker System Specification -**Status:** Draft -**Last Updated:** 2026-02-13 +**Status:** Implemented +**Last Updated:** 2026-03-01 ---