136 Commits

Author SHA1 Message Date
Forbes
68c9acea5c feat(sessions): edit session acquire, release, and query endpoints
- Add 023_edit_sessions.sql migration with unique index on (item_id, context_level, object_id) for hard interference
- Add EditSessionRepository with Acquire, Release, ReleaseForWorkstation, GetByID, ListForItem, ListForUser, TouchHeartbeat, ExpireStale, GetConflict
- Add 4 handlers: acquire (POST), release (DELETE), list by item (GET), list by user (GET)
- Acquire auto-computes dependency_cone from DAG forward cone when available
- Hard interference returns 409 with holder info (username, workstation, context_level, object_id, acquired_at)
- Publish edit.session_acquired and edit.session_released via item-scoped SSE
- Add /api/edit-sessions (user scope) and /api/items/{pn}/edit-sessions (item scope) routes

Closes #163
2026-03-01 13:40:18 -06:00
Forbes
a669327042 Merge branch 'feat/sse-per-connection-filtering' into feat/edit-sessions 2026-03-01 13:37:44 -06:00
Forbes
e7da3ee94d feat(sse): per-connection filtering with user and workstation context
- Extend sseClient with userID, workstationID, and item filter set
- Update Subscribe() to accept userID and workstationID params
- Add WatchItem/UnwatchItem/IsWatchingItem methods on sseClient
- Add PublishToItem, PublishToWorkstation, PublishToUser targeted delivery
- Targeted events get IDs but skip history ring buffer (real-time only)
- Update HandleEvents to pass auth user ID and workstation_id query param
- Touch workstation last_seen on SSE connect
- Existing Publish() broadcast unchanged; all current callers unaffected
- Add 5 new tests for targeted delivery and item watch lifecycle

Closes #162
2026-03-01 10:04:01 -06:00
Forbes
a851630d85 feat(sessions): workstation table, registration API, and module scaffold
- Add 022_workstations.sql migration (UUID PK, user_id FK, UNIQUE(user_id, name))
- Add Sessions module (depends on Auth, default enabled) with config toggle
- Add WorkstationRepository with Upsert, GetByID, ListByUser, Touch, Delete
- Add workstation handlers: register (POST upsert), list (GET), delete (DELETE)
- Add /api/workstations routes gated by sessions module
- Wire WorkstationRepository into Server struct
- Update module tests for new Sessions module

Closes #161
2026-03-01 09:56:43 -06:00
e5cae28a8c Merge pull request 'feat(api): solver service Phase 3b — server endpoints and job definitions' (#160) from feat/solver-service into main
Reviewed-on: #160
2026-02-20 18:15:20 +00:00
Forbes
5f144878d6 feat(api): solver service Phase 3b — server endpoints, job definitions, and result cache
Add server-side solver service module with REST API endpoints, database
schema, job definitions, and runner result caching.

New files:
- migrations/021_solver_results.sql: solver_results table with upsert constraint
- internal/db/solver_results.go: SolverResultRepository (Upsert, GetByItem, GetByItemRevision)
- internal/api/solver_handlers.go: solver API handlers and maybeCacheSolverResult hook
- jobdefs/assembly-solve.yaml: manual solve job definition
- jobdefs/assembly-validate.yaml: auto-validate on revision creation
- jobdefs/assembly-kinematic.yaml: manual kinematic simulation job

Modified:
- internal/config/config.go: SolverConfig struct with max_context_size_mb, default_timeout
- internal/modules/modules.go, loader.go: register solver module (depends on jobs)
- internal/db/jobs.go: ListSolverJobs helper with definition_name prefix filter
- internal/api/handlers.go: wire SolverResultRepository into Server
- internal/api/routes.go: /api/solver/* routes + /api/items/{partNumber}/solver/results
- internal/api/runner_handlers.go: async result cache hook on job completion

API endpoints:
- POST   /api/solver/jobs          — submit solver job (editor)
- GET    /api/solver/jobs          — list solver jobs with filters
- GET    /api/solver/jobs/{id}     — get solver job status
- POST   /api/solver/jobs/{id}/cancel — cancel solver job (editor)
- GET    /api/solver/solvers       — registry of available solvers
- GET    /api/items/{pn}/solver/results — cached results for item

Also fixes pre-existing test compilation errors (missing workflows param
in NewServer calls across 6 test files).
2026-02-20 12:08:34 -06:00
ed1ac45e12 Merge pull request 'feat(api): approvals + ECO workflows; refactor(storage): remove MinIO' (#154) from feat/approval-workflows into main
Reviewed-on: #154
2026-02-19 20:57:15 +00:00
Forbes
88d1ab1f97 refactor(storage): remove MinIO backend, filesystem-only storage
Remove the MinIO/S3 storage backend entirely. The filesystem backend is
fully implemented, already used in production, and a migrate-storage tool
exists for any remaining MinIO deployments to migrate beforehand.

Changes:
- Delete MinIO client implementation (internal/storage/storage.go)
- Delete migrate-storage tool (cmd/migrate-storage, scripts/migrate-storage.sh)
- Remove MinIO service, volumes, and env vars from all Docker Compose files
- Simplify StorageConfig: remove Endpoint, AccessKey, SecretKey, Bucket,
  UseSSL, Region fields; add SILO_STORAGE_ROOT_DIR env override
- Change all SQL COALESCE defaults from 'minio' to 'filesystem'
- Add migration 020 to update column defaults to 'filesystem'
- Remove minio-go/v7 dependency (go mod tidy)
- Update all config examples, setup scripts, docs, and tests
2026-02-19 14:36:22 -06:00
Forbes
12ecffdabe feat(api): approvals + ECO workflow API with YAML-configurable workflows
- Add internal/workflow/ package for YAML workflow definitions (Load, LoadAll, Validate)
- Add internal/db/item_approvals.go repository (Create, AddSignature, GetWithSignatures, ListByItemWithSignatures, UpdateState, UpdateSignature)
- Add internal/api/approval_handlers.go with 4 endpoints:
  - GET /{partNumber}/approvals (list approvals with signatures)
  - POST /{partNumber}/approvals (create ECO with workflow + signers)
  - POST /{partNumber}/approvals/{id}/sign (approve or reject)
  - GET /workflows (list available workflow definitions)
- Rule-driven state transitions: any_reject and all_required_approve
- Pack approvals into silo/approvals.json on .kc checkout
- Add WorkflowsConfig to config, load workflows at startup
- Migration 019: add workflow_name column to item_approvals
- Example workflows: engineering-change.yaml, quick-review.yaml
- 7 workflow tests, all passing

Closes #145
2026-02-18 19:38:20 -06:00
e260c175bf Merge pull request 'docs: update documentation for .kc file integration (Phases 1-4)' (#153) from docs/kc-integration into main
Reviewed-on: #153
2026-02-19 01:11:23 +00:00
Forbes
bae06da1a1 docs: update documentation for .kc file integration (Phases 1-4)
- SPECIFICATION.md: add 8 KC endpoints to Section 11.1, new Section 11.3
  documenting .kc format, extraction pipeline, packing, lifecycle state
  machine, and all response shapes. Update endpoint count 78 → 86.
- ROADMAP.md: mark .kc Format Spec as Complete in Tier 0 table
- STATUS.md: add KC features to core systems table, update migration
  list through 018, update endpoint count
- MODULES.md: add metadata, dependencies, and macros endpoints to
  core module listing
2026-02-18 19:10:56 -06:00
161c1c1e62 Merge pull request 'feat(api): macro indexing from .kc files and read-only API' (#152) from feat/kc-macros into main
Reviewed-on: #152
2026-02-19 01:06:00 +00:00
df0fc13193 Merge branch 'main' into feat/kc-macros 2026-02-19 01:05:51 +00:00
Forbes
6e6c9c2c75 feat(api): macro indexing from .kc files and read-only API
- Add MacroFile type to internal/kc and extract silo/macros/* files
  from .kc ZIP archives on commit
- Create ItemMacroRepository with ReplaceForItem, ListByItem, and
  GetByFilename methods
- Add GET /{partNumber}/macros (list) and
  GET /{partNumber}/macros/{filename} (source content) endpoints
- Index macros in extractKCMetadata with SSE broadcast
- List endpoint omits content for lightweight responses

Closes #144
2026-02-18 19:03:44 -06:00
98be1fa78c Merge pull request 'feat(api): item dependency extraction, indexing, and resolve endpoints' (#151) from feat/kc-dependencies into main
Reviewed-on: #151
2026-02-19 00:55:55 +00:00
f8b8eda973 Merge branch 'main' into feat/kc-dependencies 2026-02-19 00:55:40 +00:00
Forbes
cffcf56085 feat(api): item dependency extraction, indexing, and resolve endpoints
- Add Dependency type to internal/kc and extract silo/dependencies.json
  from .kc files on commit
- Create ItemDependencyRepository with ReplaceForRevision, ListByItem,
  and Resolve (LEFT JOIN against items table)
- Add GET /{partNumber}/dependencies and
  GET /{partNumber}/dependencies/resolve endpoints
- Index dependencies in extractKCMetadata with SSE broadcast
- Pack real dependency data into .kc files on checkout
- Update PackInput.Dependencies from []any to []Dependency

Closes #143
2026-02-18 18:53:40 -06:00
1a34455ad5 Merge pull request 'feat(kc): checkout packing + ETag caching (Phase 2)' (#150) from feat/kc-checkout-packing into main
Reviewed-on: #150
2026-02-18 23:06:17 +00:00
Forbes
c216d64702 feat(kc): checkout packing + ETag caching (Phase 2)
Implements issue #142 — .kc checkout pipeline that repacks silo/ entries
with current DB state before serving downloads.

When a client downloads a .kc file via GET /api/items/{pn}/file/{rev},
the server now:
1. Reads the file from storage into memory
2. Checks for silo/ directory (plain .fcstd files bypass packing)
3. Repacks silo/ entries with current item_metadata + revision history
4. Streams the repacked ZIP to the client

New files:
- internal/kc/pack.go: Pack() replaces silo/ entries in ZIP, preserving
  all non-silo entries (FreeCAD files, thumbnails) with original
  compression and timestamps. HasSiloDir() for lightweight detection.
- internal/api/pack_handlers.go: packKCFile server helper, computeETag,
  canSkipRepack lazy optimization.

ETag caching:
- ETag computed from revision_number + metadata.updated_at
- If-None-Match support returns 304 Not Modified before reading storage
- Cache-Control: private, must-revalidate

Lazy packing optimization:
- Skips repack if revision_hash matches and metadata unchanged since upload

Phase 2 packs: manifest.json, metadata.json, history.json,
dependencies.json (empty []). Approvals, macros, jobs deferred to
Phase 3-5.

Closes #142
2026-02-18 17:01:26 -06:00
28f133411e Merge pull request 'feat(kc): commit extraction pipeline + metadata API (Phase 1)' (#149) from feat/kc-extraction-pipeline into main
Reviewed-on: #149
2026-02-18 22:39:59 +00:00
6528df0461 Merge branch 'main' into feat/kc-extraction-pipeline 2026-02-18 22:39:49 +00:00
Forbes
dd010331c0 feat(kc): commit extraction pipeline + metadata API (Phase 1)
Implements issue #141 — .kc server-side metadata integration Phase 1.

When a .kc file is uploaded, the server extracts silo/manifest.json and
silo/metadata.json from the ZIP archive and indexes them into the
item_metadata table. Plain .fcstd files continue to work unchanged.
Extraction is best-effort: failures are logged but do not block the upload.

New packages:
- internal/kc: ZIP extraction library (Extract, Manifest, Metadata types)
- internal/db: ItemMetadataRepository (Get, Upsert, UpdateFields,
  UpdateLifecycle, SetTags)

New API endpoints under /api/items/{partNumber}:
- GET    /metadata           — read indexed metadata (viewer)
- PUT    /metadata           — merge fields into JSONB (editor)
- PATCH  /metadata/lifecycle — transition lifecycle state (editor)
- PATCH  /metadata/tags      — add/remove tags (editor)

SSE events: metadata.updated, metadata.lifecycle, metadata.tags

Lifecycle transitions (Phase 1): draft→review→released→obsolete,
review→draft (reject).

Closes #141
2026-02-18 16:37:39 -06:00
628cd1d252 Merge pull request 'feat(db): .kc metadata database migration' (#148) from feat/kc-metadata-migration into main
Reviewed-on: #148
2026-02-18 21:05:15 +00:00
Forbes
8d777e83bb feat(db): .kc metadata database migration (#140)
Add migration 018_kc_metadata.sql with all tables needed for .kc
server-side metadata indexing:

- item_metadata: indexed manifest + metadata fields from silo/
  directory (tags, lifecycle_state, fields JSONB, manifest info)
- item_dependencies: CAD-extracted assembly dependencies
  (complements existing relationships table)
- item_approvals + approval_signatures: ECO workflow state
- item_macros: registered macros from silo/macros/

Also adds docs/KC_SERVER.md specification document.

Closes #140
2026-02-18 15:04:03 -06:00
d96ba8d394 Merge pull request 'docs: replace MinIO with filesystem storage throughout' (#139) from update-silo-fs-docs into main
Reviewed-on: #139
2026-02-18 20:46:51 +00:00
Forbes
56c76940ed docs: replace MinIO with filesystem storage throughout
Remove all MinIO/S3 references from documentation and deployment
configs. Silo now uses local filesystem storage exclusively.

Updated files:
- docs/CONFIGURATION.md: storage section now documents filesystem backend
- docs/DEPLOYMENT.md: architecture diagram, external services, troubleshooting
- docs/INSTALL.md: remove MinIO setup, update architecture diagrams
- docs/SPECIFICATION.md: architecture, technology stack, file storage strategy
- docs/STATUS.md: storage backend status
- docs/GAP_ANALYSIS.md: file handling references
- docs/ROADMAP.md: file storage appendix entries
- deployments/config.prod.yaml: filesystem backend config
- deployments/systemd/silod.env.example: remove MinIO credential vars
2026-02-18 14:45:00 -06:00
9dabaf5796 Merge pull request 'feat(scripts): remote migrate-storage script for MinIO to filesystem migration' (#138) from feat-remote-migrate-storage into main
Reviewed-on: #138
2026-02-18 20:33:38 +00:00
Forbes
3bb335397c feat(scripts): remote migrate-storage script for MinIO to filesystem migration
Adds scripts/migrate-storage.sh that follows the same deploy.sh pattern:
cross-compiles the migrate-storage binary locally, uploads it to the
target host via SCP, then runs it over SSH using credentials from
/etc/silo/silod.env.

Usage: ./scripts/migrate-storage.sh <silo-host> <psql-host> <minio-host> [flags...]
2026-02-18 14:29:46 -06:00
344a0cd0a0 Merge pull request 'feat(storage): add MinIO to filesystem migration tool' (#137) from feat/migrate-storage-tool into main
Reviewed-on: #137
2026-02-18 20:16:17 +00:00
forbes
f5b03989ff feat(storage): add MinIO to filesystem migration tool
Standalone binary (cmd/migrate-storage) that downloads all files from
MinIO and writes them to the local filesystem for decommissioning MinIO.

Queries revision files, item file attachments, and item thumbnails from
the database, then downloads each from MinIO preserving the object key
structure as filesystem paths. Supports --dry-run, --verbose, atomic
writes via temp+rename, and idempotent re-runs (skips existing files
with matching size).
2026-02-18 14:12:32 -06:00
8cd92a4025 Merge pull request 'feat(api): direct multipart upload endpoints for filesystem backend' (#136) from feat-direct-upload into main
Reviewed-on: #136
2026-02-17 19:05:39 +00:00
ffa01ebeb7 feat(api): direct multipart upload endpoints for filesystem backend
Add three new endpoints that bypass the MinIO presigned URL flow:
- POST /api/items/{pn}/files/upload — multipart file upload
- POST /api/items/{pn}/thumbnail/upload — multipart thumbnail upload
- GET /api/items/{pn}/files/{fileId}/download — stream file download

Rewrite frontend upload flow: files are held in browser memory on drop
and uploaded directly after item creation via multipart POST. The old
presign+associate endpoints remain for MinIO backward compatibility.

Closes #129
2026-02-17 13:04:44 -06:00
9181673554 Merge pull request 'feat(db): add storage backend metadata columns' (#135) from feat-file-storage-metadata into main
Reviewed-on: #135
2026-02-17 18:32:05 +00:00
8cef4fa55f feat(db): add storage backend metadata columns
Add storage_backend columns to track which backend (minio or filesystem)
holds each file, enabling dual-running during migration.

Migration 017_file_storage_metadata.sql:
- item_files.storage_backend TEXT NOT NULL DEFAULT 'minio'
- revisions.file_storage_backend TEXT NOT NULL DEFAULT 'minio'

DB repository changes:
- Revision struct: add FileStorageBackend field
- ItemFile struct: add StorageBackend field
- All INSERT queries include the new columns
- All SELECT queries read them (COALESCE for pre-migration compat)
- CreateRevisionFromExisting copies the backend from source revision
- Default to 'minio' when field is empty (backward compat)

Existing rows default to 'minio'. New uploads will write 'filesystem'
when the filesystem backend is active.

Closes #128
2026-02-17 12:30:20 -06:00
7a9dd057a5 Merge pull request 'feat(storage): FileStore interface abstraction + filesystem backend' (#134) from feat-storage-interface-filesystem into main
Reviewed-on: #134
2026-02-17 17:55:09 +00:00
9f347e7898 feat(storage): implement filesystem backend
Implement FilesystemStore satisfying the FileStore interface for local
filesystem storage, replacing MinIO for simpler deployments.

- Atomic writes via temp file + os.Rename (no partial files)
- SHA-256 checksum computed on Put via io.MultiWriter
- Get/GetVersion return os.File (GetVersion ignores versionID)
- Delete is idempotent (no error if file missing)
- Copy uses same atomic write pattern
- PresignPut returns ErrPresignNotSupported
- Ping verifies root directory is writable
- Wire NewFilesystemStore in main.go backend switch
- 14 unit tests covering all methods including atomicity

Closes #127
2026-02-17 11:49:42 -06:00
b531617e39 feat(storage): define FileStore interface and refactor to use it
Extract a FileStore interface from the concrete *storage.Storage MinIO
wrapper so the API layer is storage-backend agnostic.

- Define FileStore interface in internal/storage/interface.go
- Add Exists method to MinIO Storage (via StatObject)
- Add compile-time interface satisfaction check
- Change Server.storage and ServerState.storage to FileStore interface
- Update NewServer and NewServerState signatures
- Add Backend and FilesystemConfig fields to StorageConfig
- Add backend selection switch in main.go (minio/filesystem/unknown)
- Update config.example.yaml with backend field

The nil-interface pattern is preserved: when storage is unconfigured,
store remains a true nil FileStore (not a typed nil pointer), so all
existing if s.storage == nil checks continue to work correctly.

Closes #126
2026-02-17 11:49:35 -06:00
906277149e Merge pull request 'feat(web): read-write configuration from admin UI' (#124) from feat-admin-config-ui into main
Reviewed-on: #124
2026-02-15 23:12:04 +00:00
Forbes
fc4826f576 feat(web): read-write configuration from admin UI
Convert all module settings from read-only to editable fields in the
admin settings page:

- Core: host, port, base_url (read-only stays read-only)
- Schemas: directory, default (count stays read-only)
- Database: host, port, name, user, password, sslmode (dropdown),
  max_connections
- Storage: endpoint, bucket, use_ssl (checkbox), region
- Auth: local/ldap/oidc sub-sections with enabled checkboxes,
  connection fields, and secret fields (password input for redacted)

New field components: SelectField (dropdown), CheckboxField (toggle).
Redacted fields now render as password inputs with placeholder.
Auth uses nested key handling to send sub-section objects.

Backend already persists overrides and flags restart-required changes.

Closes #117
2026-02-15 13:33:48 -06:00
fbfc955ccc Merge pull request 'feat(modules): SSE settings.changed event broadcast' (#123) from feat-sse-settings-changed into main
Reviewed-on: #123
2026-02-15 19:14:36 +00:00
e0295e7180 Merge branch 'main' into feat-sse-settings-changed 2026-02-15 19:14:26 +00:00
Forbes
7fec219152 feat(modules): SSE settings.changed event broadcast and UI reactions
Add useSSE hook that connects to /api/events with automatic reconnect
and exponential backoff. On settings.changed events:

- Refresh module state so sidebar nav items show/hide immediately
- Show dismissable toast when another admin updates settings

The backend already publishes settings.changed in HandleUpdateModuleSettings.

Closes #101
2026-02-15 13:11:04 -06:00
fa069eb05c Merge pull request 'feat(web): move edit/delete buttons into tab bar on item detail' (#122) from feat-move-edit-delete-buttons into main
Reviewed-on: #122
2026-02-15 19:03:59 +00:00
Forbes
8735c8341b feat(web): move edit/delete buttons into tab bar on item detail
Relocate Edit and Delete buttons from the header row into the tab bar,
grouping them with tab navigation to reduce mouse travel. Adds Pencil
and Trash2 icons for quick visual recognition.

Header now only shows part number, type badge, and close button.

Closes #119
2026-02-15 12:59:40 -06:00
7a172ce34c Merge pull request 'feat(web): favicon, narrow settings, scrollable token list' (#121) from feat-ui-tweaks into main
Reviewed-on: #121
2026-02-15 18:47:03 +00:00
Forbes
da65d4bc1a feat(web): favicon, narrow settings, scrollable token list
- Add kindred-logo.svg as site favicon (#115)
- Narrow settings page to 66% max-width, centered (#116)
- Add max-height and scroll to API token table (#118)

Closes #115, closes #116, closes #118
2026-02-15 12:38:20 -06:00
57d5a786d0 Merge pull request 'feat(web): collapsible left sidebar, remove top nav bar' (#120) from feat-sidebar-nav into main
Reviewed-on: #120
2026-02-15 18:33:09 +00:00
Forbes
42a901f39c feat(web): collapsible left sidebar, remove top nav bar
- Replace top header with left sidebar navigation
- Sidebar shows module-aware nav items filtered by /api/modules
- Collapsible: expanded shows icon+label, collapsed shows icon only
- Toggle with Ctrl+J or collapse button, state persisted in localStorage
- Keyboard navigable: Arrow Up/Down, Enter to navigate, Escape to collapse
- Bottom section: density toggle, user info with role badge, logout
- Add useModules hook for fetching module state
- Add sidebar density variables to theme.css

Closes #113, closes #114
2026-02-15 12:32:52 -06:00
666cc2b23b Merge pull request 'feat(jobs): wire auto-triggering on bom_changed events' (#112) from feat-job-auto-trigger into main
Reviewed-on: #112
2026-02-15 15:44:42 +00:00
Forbes
747bae8354 feat(jobs): wire auto-triggering on bom_changed events, add module guard
- Add IsEnabled("jobs") guard to triggerJobs() to skip when module disabled
- Fire bom_changed trigger from HandleAddBOMEntry, HandleUpdateBOMEntry,
  HandleDeleteBOMEntry (matching existing HandleMergeBOM pattern)
- Add 4 integration tests: revision trigger, BOM trigger, filter mismatch,
  module disabled
- Fix AppShell overflow: hidden -> auto so Settings page scrolls
- Clean old frontend assets in deploy script before extracting

Closes #107
2026-02-15 09:43:05 -06:00
71603bb6d7 Merge pull request 'feat: location hierarchy CRUD API' (#106) from feat-location-crud into main
Reviewed-on: #106
2026-02-15 09:16:52 +00:00
Forbes
4ef912cf4b feat: location hierarchy CRUD API
Add LocationRepository with CRUD operations, hierarchy traversal
(children, subtree by path prefix), and inventory-safe deletion.

Endpoints:
  GET    /api/locations          — list all or ?tree={path} for subtree
  POST   /api/locations          — create (auto-resolves parent_id, depth)
  GET    /api/locations/{path..} — get by hierarchical path
  PUT    /api/locations/{path..} — update name, type, metadata
  DELETE /api/locations/{path..} — delete (rejects if inventory exists)

Uses chi wildcard routes to support multi-segment paths like
/api/locations/lab/shelf-a/bin-3.

Includes 10 handler integration tests covering CRUD, nesting,
validation, duplicates, tree queries, and delete-not-found.

Closes #81
2026-02-15 03:15:54 -06:00
decb32c3e7 Merge pull request 'feat(web): admin settings page — module cards, toggles, config forms' (#105) from feat-admin-settings-api into main
Reviewed-on: #105
2026-02-15 09:09:17 +00:00
Forbes
0be39065ac feat(web): admin settings page with module cards, toggles, config forms
Add admin-only Module Configuration section to the Settings page.
Each module gets a collapsible card with enable/disable toggle,
status badge, module-specific config fields, save and test
connectivity buttons.

- AdminModules: fetches GET /api/modules + GET /api/admin/settings,
  renders Infrastructure and Features groups, restart banner
- ModuleCard: collapsible card with toggle, status badge, field
  layouts per module, save (PUT) and test (POST) actions
- TypeScript types for ModuleInfo, ModulesResponse, admin settings
  API response shapes

Ref: #100
2026-02-15 03:01:33 -06:00
Forbes
101d04ab6f test(api): admin settings handler tests
- TestGetAllSettings — all module keys present, secrets redacted
- TestGetModuleSettings — single module response
- TestGetModuleSettings_Unknown — 404 for unknown module
- TestToggleModule — disable projects, verify registry state
- TestToggleModule_DependencyError — enable dag without jobs, expect 400
- TestToggleRequiredModule — disable core, expect 400
- TestTestConnectivity_Database — ping database, expect success
- TestTestConnectivity_NotTestable — core module, expect 400
2026-02-15 02:51:00 -06:00
Forbes
8167d9c216 feat(api): admin settings API endpoints
Add four admin-only endpoints under /api/admin/settings:

- GET  /                — full config (secrets redacted)
- GET  /{module}        — single module config
- PUT  /{module}        — toggle modules + persist config overrides
- POST /{module}/test   — test external connectivity (database, storage)

PUT publishes a settings.changed SSE event. Config overrides are
persisted for future hot-reload support; changes to database/storage/
server/schemas namespaces return restart_required: true.

Wires SettingsRepository into Server struct.

Closes #99
2026-02-15 02:51:00 -06:00
Forbes
319a739adb feat(db): add SettingsRepository for module state and config overrides
Provides CRUD operations on the module_state and settings_overrides
tables (created in migration 016).

- GetModuleStates / SetModuleState — upsert module enabled/disabled
- GetOverrides / SetOverride / DeleteOverride — JSONB config overrides

Part of #99
2026-02-15 02:51:00 -06:00
e20252a993 Merge pull request 'feat: module system — registry, middleware, and discovery endpoint' (#102) from feat-module-system into main
Reviewed-on: #102
2026-02-14 20:05:42 +00:00
Forbes
138ce16010 fix: remove unreachable code in testutil.findProjectRoot 2026-02-14 14:02:48 -06:00
Forbes
690ad73161 feat(modules): public GET /api/modules discovery endpoint
Add HandleGetModules returning module state, metadata, and
public config (auth providers, Create URI scheme). No auth
required — clients call this pre-login.

Register at /api/modules before the auth middleware.

Ref #97
2026-02-14 14:02:11 -06:00
Forbes
b8abd8859d feat(modules): RequireModule middleware to gate route groups
Add RequireModule middleware that returns 404 with
{"error":"module '<id>' is not enabled"} when a module is disabled.

Wrap route groups:
- projects → RequireModule("projects")
- audit → RequireModule("audit")
- integrations/odoo → RequireModule("odoo")
- jobs, job-definitions, runners → RequireModule("jobs")
- /api/runner (runner-facing) → RequireModule("jobs")
- dag → RequireModule("dag") (extracted into sub-route)

Ref #98
2026-02-14 14:01:32 -06:00
Forbes
4fd4013360 feat(modules): wire registry into server startup
Add modules.Registry and config.Config fields to Server struct.
Create registry in main.go, load state from YAML+DB, log all
module states at startup.

Conditionally start job/runner sweeper goroutines only when the
jobs module is enabled.

Update all 5 test files to pass registry to NewServer.

Ref #95, #96
2026-02-14 14:00:24 -06:00
Forbes
3adc155b14 feat(modules): config loader refactor — YAML → DB → env pipeline
Add ModulesConfig and ModuleToggle types to config.go for explicit
module enable/disable in YAML.

Add LoadState() that merges state from three sources:
1. Backward-compat YAML fields (auth.enabled, odoo.enabled)
2. Explicit modules.* YAML toggles (override compat)
3. Database module_state table (highest precedence)

Validates dependency chain after loading. 5 loader tests.

Ref #95
2026-02-14 13:58:26 -06:00
Forbes
9d8afa5981 feat(modules): module registry with metadata, dependencies, and defaults
In-memory registry for 10 modules (3 required, 7 optional).
SetEnabled validates dependency chains: cannot enable a module
whose dependencies are disabled, cannot disable a module that
others depend on.

9 unit tests covering default state, toggling, dependency
validation, and error cases.

Ref #96
2026-02-14 13:57:32 -06:00
Forbes
f91cf2bc6f feat(modules): settings_overrides and module_state migration
Add migration 016 with two tables for the module system:
- settings_overrides: dotted-path config overrides set via admin UI
- module_state: per-module enabled/disabled state

Update testutil.TruncateAll to include new tables.

Ref #94
2026-02-14 13:56:26 -06:00
ef44523ae8 Merge pull request 'fix(web): standardize typography and spacing to style guide' (#93) from fix-web-style-guide into main
Reviewed-on: #93
2026-02-14 19:37:04 +00:00
Forbes
ba92dd363c fix(web): align all spacing values to 4px grid
Standardize all spacing to multiples of 4px (0.25rem):
- 0.15rem (2.4px) → 0.25rem (4px)
- 0.35rem (5.6px) → 0.25rem (4px)
- 0.375rem (6px) → 0.25rem (4px) for borderRadius
- 0.4rem (6.4px) → 0.5rem (8px)
- 0.6rem (9.6px) → 0.5rem (8px)

Updated theme.css density variables, silo-base.css focus ring,
and all TSX component inline styles.

Closes #71
2026-02-14 13:36:22 -06:00
Forbes
c7857fdfc9 fix(web): standardize font sizes to style guide scale
Map fontWeight: 700 → 600 in non-title contexts (LoginPage, FileDropZone).
Align FileDropZone badge padding to 4px grid.

Closes #70
2026-02-14 13:36:07 -06:00
defb3af56f Merge pull request 'feat: dependency DAG and YAML-defined compute jobs' (#92) from feat-dag-workers into main
Reviewed-on: #92
2026-02-14 19:27:18 +00:00
Forbes
6d7a85cfac docs: add DAG client integration contract for silo-mod and runners 2026-02-14 13:24:36 -06:00
Forbes
22c778f8b0 test: add DAG handler, job handler, and runner token tests 2026-02-14 13:23:21 -06:00
Forbes
ad4224aa8f feat: add silorunner binary with job poll/claim/execute lifecycle 2026-02-14 13:21:21 -06:00
Forbes
b6ac5133c3 feat: add auto-trigger hooks for revision and BOM changes 2026-02-14 13:20:15 -06:00
Forbes
2732554cd2 feat: add job, runner, and DAG API handlers with routes 2026-02-14 13:19:02 -06:00
Forbes
df073709ce feat: add DAG API handlers for graph queries and sync 2026-02-14 13:16:19 -06:00
Forbes
0eb891667b feat: add runner authentication middleware and identity context 2026-02-14 13:14:36 -06:00
Forbes
1952dea00c feat: wire job definitions, DAG/job repos, and background sweepers 2026-02-14 13:13:54 -06:00
Forbes
6becfd82d4 feat: add job and runner repository with atomic claim 2026-02-14 13:11:41 -06:00
Forbes
671a0aeefe feat: add DAG repository with graph queries and dirty propagation 2026-02-14 13:09:41 -06:00
Forbes
f60c25983b feat: add YAML job definition parser and example definitions
New package internal/jobdef mirrors the schema package pattern:
- Load/LoadAll/Validate for YAML job definitions
- Supports trigger types: revision_created, bom_changed, manual, schedule
- Supports scope types: item, assembly, project
- Supports compute types: validate, rebuild, diff, export, custom
- Defaults: timeout=600s, max_retries=1, priority=100

Example definitions in jobdefs/:
- assembly-validate.yaml: incremental validation on revision_created
- part-export-step.yaml: STEP export on manual trigger

11 unit tests, all passing.
2026-02-14 13:06:24 -06:00
Forbes
83e0d6821c feat: add database migrations for DAG and worker system
Migration 014: dag_nodes, dag_edges, dag_cross_edges tables for the
feature-level dependency graph with validation state tracking.

Migration 015: runners, job_definitions, jobs, job_log tables for the
async compute job system with PostgreSQL-backed work queue.

Update TruncateAll in testutil to include new tables.
2026-02-14 13:04:41 -06:00
Forbes
9a8b3150ff docs: add DAG and worker system specifications
DAG.md describes the two-tier dependency graph (BOM DAG + feature DAG),
node/edge data model, validation states, dirty propagation, forward/backward
cone queries, DAG sync payload format, and REST API.

WORKERS.md describes the general-purpose async compute job system: YAML job
definitions, job lifecycle (pending→claimed→running→completed/failed),
runner registration and authentication, claim semantics (SELECT FOR UPDATE
SKIP LOCKED), timeout enforcement, SSE events, and REST API.
2026-02-14 13:03:48 -06:00
376fa3db31 Merge pull request 'test: add test coverage for DB, file handlers, CSV/ODS, and API endpoints' (#86) from test-coverage-batch into main
Reviewed-on: #86
2026-02-14 14:50:38 +00:00
Forbes
257e3d99ac test(api): add revision, schema, audit, and auth handler tests (#78)
Revision tests (8):
- List, get, create, update status/labels, compare, rollback
- Not-found paths for missing items and revisions

Schema tests (4):
- List schemas, get by name, form descriptor, not-found

Audit tests (4):
- Completeness summary (empty + with items), item detail, not-found

Auth tests (4):
- Get current user (authenticated + unauthenticated)
- Auth config response
- Token lifecycle: create, list, revoke
2026-02-13 15:22:28 -06:00
Forbes
384b137148 test(api): add CSV and ODS import/export handler tests (#77)
CSV tests:
- Export empty/with items, template generation
- Import dry-run (preview without creating), commit (items created)
- BOM CSV export with parent/child relationships

ODS tests:
- Export with items (verify ODS content type and ZIP magic bytes)
- Template generation per schema
- Project sheet export with item associations
2026-02-13 15:20:20 -06:00
Forbes
7c838bdf5e test(api): add file handler tests and fix createItemDirect helper (#76)
- Test ListItemFiles, DeleteItemFile with real DB
- Test cross-item file deletion guard (404)
- Test storage-unavailable paths: presign, upload, associate, thumbnail (503)
- Fix createItemDirect: StandardCost moved to revision properties
2026-02-13 15:18:46 -06:00
Forbes
c9b081b8f8 test(db): add edge-case tests for items, revisions, projects, and files (#75)
- Duplicate part number constraint (PG 23505)
- Hard delete, pagination, search filtering
- Revision status/labels update, compare, rollback
- Project-item association by code, list by project filter
- Item file CRUD: create, list, get, delete
2026-02-13 15:17:38 -06:00
bc1149d4ba Merge pull request 'fix(web): style guide batch 3 — icons, font scale, spacing' (#85) from fix-style-guide-batch-3 into main
Reviewed-on: #85
2026-02-13 20:59:59 +00:00
Forbes
07c4aa1c28 fix(web): align spacing values to style guide grid (#71)
- Replace 0.3rem padding/margin/gap with 0.25rem (xs)
- Replace 0.2rem margins with 0.25rem (xs)
- Replace 0.1rem padding with 0.15rem (badge spec)
- Replace 0.6rem margins/padding with 0.5rem (sm)
- Fix borderRadius 0.3rem to 0.375rem (6px per style guide)
- Preserve style-guide-specified values: 0.35rem button gap, 0.4rem cell padding, 0.45rem input padding
2026-02-13 14:37:40 -06:00
Forbes
679b730e74 fix(web): standardize font sizes to style guide scale (#70) 2026-02-13 14:33:11 -06:00
Forbes
b53ce94274 feat(web): install lucide-react and replace unicode icons (#67) 2026-02-13 13:44:48 -06:00
8316ac085c Merge pull request 'fix: style guide batch 2 — buttons, tables, transitions, inputs, validation' (#84) from fix-style-guide-batch-2 into main
Reviewed-on: #84
2026-02-13 19:27:56 +00:00
Forbes
d5f1b4e587 feat(partnum): implement part number validation (#80)
Implement Generator.Validate() to check part numbers against schemas:
- Split by separator, verify segment count
- Constant: must equal expected value
- Enum: must be in allowed values map
- String: length, case, pattern constraints
- Serial: length + numeric-only check
- Date: length matches expected format output

Add belt-and-suspenders call in HandleCreateItem after Generate().
Add 9 validation tests (all pass alongside 10 existing tests).

Closes #80
2026-02-13 13:26:13 -06:00
Forbes
f4a1c8004b feat(web): add input focus states via silo-base.css (#72)
Create silo-base.css with .silo-input hover/focus pseudo-classes:
- hover: border-color overlay0
- focus: border-color mauve + 0.2rem mauve box-shadow

Applied className='silo-input' to form inputs in:
CreateItemPane, EditItemPane, ProjectsPage, SchemasPage,
SettingsPage, LoginPage

Closes #72
2026-02-13 13:24:39 -06:00
Forbes
a9614e704e fix(web): standardize transition durations to 0.15s ease (#73)
All inline transition properties now use 'all 0.15s ease'.

Files: AppShell, CategoryPicker, AuditDetailPanel,
AuditSummaryBar, AuditTable, FileDropZone

Closes #73
2026-02-13 13:23:08 -06:00
Forbes
289d488469 fix(web): table header color overlay1 + selected row mauve tint (#69)
- All thStyle objects: color subtext1/subtext0 → overlay1
- Selected row bg: surface1 → rgba(mauve, 0.08)

Files: ItemTable, BOMTab, ProjectsPage, SchemasPage,
SettingsPage, AuditTable

Closes #69
2026-02-13 13:22:30 -06:00
Forbes
2585305590 fix(web): standardize button borderRadius, fontSize, fontWeight (#68)
All button style objects now use:
- borderRadius: 0.375rem
- fontSize: 0.75rem
- fontWeight: 500

Files: CreateItemPane, EditItemPane, DeleteItemPane, BOMTab,
CategoryPicker, ProjectsPage, SchemasPage, LoginPage,
ItemsToolbar, SettingsPage, ImportItemsPane, ItemTable

Closes #68
2026-02-13 13:21:54 -06:00
65063c9ee7 Merge pull request 'fix: style guide compliance batch 1 + date segment type' (#83) from fix-style-guide-batch-1 into main
Reviewed-on: #83
2026-02-13 19:13:35 +00:00
Forbes
1f7960db50 feat: implement date segment type for part number generation
Fixes #79

Implement the date segment type in the part number generator. Uses Go's
time.Format with the segment's Value field as the layout string.

- Default format: 20060102 (YYYYMMDD) when no Value is specified
- Custom formats via Value field: "0601" (YYMM), "2006" (YYYY), etc.
- Always uses UTC time
- Add 3 tests: default format, custom YYMM format, year-only format
2026-02-13 13:10:57 -06:00
Forbes
648c659e2b fix(web): use system font stack per style guide
Fixes #66

Remove Inter and Roboto from font-family. The style guide specifies
system fonts only: -apple-system, BlinkMacSystemFont, Segoe UI,
system-ui, sans-serif.
2026-02-13 13:09:56 -06:00
Forbes
d4ea6d2739 fix(web): align item type badge colors with style guide
Fixes #65

- Part: blue → green (--ctp-green)
- Assembly: green → mauve (--ctp-mauve)
- Document: yellow → blue (--ctp-blue)
- Add purchased (--ctp-peach) and phantom (--ctp-overlay1)
- Keep tooling as red (--ctp-red)
2026-02-13 13:09:41 -06:00
e20da25405 Merge pull request 'docs: integrate root ROADMAP.md into docs/ROADMAP.md' (#64) from docs-integrate-roadmap into main
Reviewed-on: #64
2026-02-13 18:45:04 +00:00
Forbes
30bb3ee56e docs: integrate root ROADMAP.md into docs/ROADMAP.md
Merge the tactical root ROADMAP.md (SOLIDWORKS PDM gap analysis, Phase 1-6
timelines, project inventory) with the visionary docs/ROADMAP.md (modular
platform architecture, .kc format, dependency tiers) into a single unified
document at docs/ROADMAP.md.

- Adopt dependency tier structure (Tier 0-6) as the organizing spine
- Add Status column to each tier table from project inventory
- Replace Phase 1-6 calendar timelines with Near-Term Priorities section
- Migrate 9 SOLIDWORKS PDM comparison tables to docs/GAP_ANALYSIS.md
- Migrate Feature Comparison Matrix to docs/GAP_ANALYSIS.md
- Preserve Phase 1 task checklists and success metrics as appendices
- Update README.md link from ROADMAP.md to docs/ROADMAP.md
- Delete root ROADMAP.md
2026-02-13 12:44:25 -06:00
a517a95912 Merge branch 'production' into main 2026-02-12 15:28:43 +00:00
6f1504021c Merge pull request 'docs: add consolidated installation guide and helper scripts' (#62) from install-guide-and-scripts into main
Reviewed-on: #62
2026-02-12 15:02:25 +00:00
d93770c551 Merge branch 'main' into install-guide-and-scripts 2026-02-12 15:02:13 +00:00
606316204d docs: add consolidated installation guide
Add docs/INSTALL.md with two installation paths:
- Option A: Docker Compose (all-in-one with PostgreSQL, MinIO,
  OpenLDAP, and optional nginx)
- Option B: Daemon install (systemd with external services, links to
  setup instructions for PostgreSQL, MinIO, FreeIPA, nginx)

Includes LDAP user/group management instructions, verification steps,
and upgrade procedures for both paths.

Update README.md Quick Start to point to INSTALL.md, add to docs table.
Add redirect banner to DEPLOYMENT.md for first-time users.
Add comments to docker-compose.prod.yaml noting unsupported env vars.
2026-02-12 08:59:10 -06:00
3d9ef9e99e refactor(scripts): parameterize hostnames in deployment scripts
- setup-host.sh: add SILO_DB_HOST and SILO_MINIO_HOST env var
  overrides, update Go version from 1.23 to 1.24, expand generated
  silod.env template with session secret and admin password fields
- deploy.sh: add SILO_DEPLOY_TARGET and SILO_DB_HOST env var
  overrides for target host and database host
- setup-ipa-nginx.sh: replace hardcoded hostname with SILO_HOSTNAME
  env var (default: silo.example.internal), parameterize SILO_PORT,
  use variable substitution in nginx config template

All scripts retain backward-compatible defaults.
2026-02-12 08:59:01 -06:00
fb13795ef7 feat(deployments): add all-in-one Docker Compose stack with OpenLDAP
Add docker-compose.allinone.yaml with five services:
- PostgreSQL 16 with auto-applied migrations
- MinIO for S3-compatible file storage
- OpenLDAP (bitnami/openldap:2.6) with memberOf overlay and
  preconfigured silo-admins/silo-users/silo-viewers groups
- Silo API server built from Dockerfile
- Nginx reverse proxy (optional, via --profile nginx)

Add scripts/setup-docker.sh interactive helper that generates
deployments/.env and deployments/config.docker.yaml with random
credentials. Supports --non-interactive for CI.

Add deployments/ldap/ LDIF init scripts for memberOf overlay and
Silo role groups. Add deployments/nginx/ reverse proxy configs.
2026-02-12 08:58:55 -06:00
1c1cd144dc fix(deployments): fix broken config mount and expand env example
- Fix docker-compose.yaml: mount config.dev.yaml instead of nonexistent
  configs/config.yaml
- Add deployments/config.dev.yaml with Docker service names and dev
  defaults for zero-setup make docker-up
- Expand .env.example with all SILO_* and LDAP_* variables
- Update config.example.yaml hostnames to localhost with Docker comments
- Add deployments/config.docker.yaml to .gitignore (generated file)
2026-02-12 08:58:39 -06:00
460b0f37fd Merge pull request 'docs: replace kindred.internal with example.internal in all docs and config' (#61) from docs-remove-internal-hostnames into main
Reviewed-on: #61
2026-02-11 17:49:40 +00:00
73195be6a1 docs: fix stale migration and endpoint counts, add missing endpoints
- Update migration count from 11 to 13 across all docs (012_bom_source,
  013_move_cost_sourcing_to_props)
- Update endpoint count from 75 to 78 across all docs
- Add 3 missing endpoints to SPECIFICATION.md section 11.1:
  GET /api/events (SSE), GET /api/items/by-uuid/{uuid},
  POST /api/items/{pn}/bom/merge
- Add migrations 012 and 013 to STATUS.md table
- Fix migration 010 description (sourcing_link and standard_cost moved
  to revision properties in 013)
2026-02-11 11:37:53 -06:00
127836f7ce docs: replace kindred.internal with example.internal in all docs and config
Replace all references to internal hostnames (silo.kindred.internal,
psql.kindred.internal, minio.kindred.internal, ipa.kindred.internal,
keycloak.kindred.internal) with example.internal equivalents.

Replace gitea.kindred.internal and git.kindred.internal with the public
git.kindred-systems.com instance. Also fix stale silo-0062 repo name
in setup-host.sh and DEPLOYMENT.md.
2026-02-11 11:20:45 -06:00
a258152175 Merge pull request 'docs: update specs for schema-driven form descriptor API' (#60) from issue-docs-form-descriptor into main
Reviewed-on: #60
2026-02-11 16:41:16 +00:00
efb3ccdfb5 Merge branch 'main' into issue-docs-form-descriptor 2026-02-11 16:41:07 +00:00
a80e99e500 docs: update specs for schema-driven form descriptor API
- frontend-spec.md: rewrite CreateItemPane spec for dynamic form
  rendering from form descriptor, replace CategoryPicker three-column
  spec with multi-stage domain/subcategory picker, replace useCategories
  hook with useFormDescriptor, update form sections to dynamic field
  groups, mark hierarchical categories as implemented, remove
  sourcing_link/standard_cost from item-level DB columns, update types
  and implementation order
- SPECIFICATION.md: rename /api/schemas/{name}/properties endpoint to
  /api/schemas/{name}/form
2026-02-11 10:40:09 -06:00
485675b020 Merge pull request 'issue-dedup-sourcing-cost' (#59) from issue-dedup-sourcing-cost into main
Reviewed-on: #59
2026-02-11 16:28:50 +00:00
beaf091d62 Merge branch 'main' into issue-dedup-sourcing-cost 2026-02-11 16:28:42 +00:00
4edaa35c49 feat: schema-driven form descriptor API and dynamic form rendering
- Add ui section to kindred-rd.yaml with category_picker (multi-stage),
  item_fields, field_groups, category_field_groups, and field_overrides
- Add UIConfig structs to Go schema parser with full YAML/JSON tags
- Add ValidateUI() to validate field references against property schemas
- Add ValuesByDomain() helper to auto-derive subcategory picker stages
- Implement GET /api/schemas/{name}/form endpoint that returns resolved
  form descriptor with field metadata, widget hints, and category picker
- Replace GET /api/schemas/{name}/properties route with /form
- Add FormDescriptor TypeScript types
- Create useFormDescriptor hook (replaces useCategories)
- Rewrite CreateItemPane to render all sections dynamically from descriptor
- Update CategoryPicker with multi-stage domain/subcategory selection
- Delete useCategories.ts (superseded by useFormDescriptor)
2026-02-11 10:14:00 -06:00
b3c748ef10 refactor: move sourcing_link and standard_cost from item columns to revision properties
- Add migration 013 to copy sourcing_link/standard_cost values into
  current revision properties JSONB and drop the columns from items table
- Remove SourcingLink/StandardCost from Go Item struct and all DB queries
  (items.go, audit_queries.go, projects.go)
- Remove from API request/response structs and handlers
- Update CSV/ODS/BOM export/import to read these from revision properties
- Update audit handlers to score as regular property fields
- Remove from frontend Item type and hardcoded form fields
- MainTab now reads sourcing_link/standard_cost from item.properties
- CreateItemPane/EditItemPane no longer have dedicated fields for these;
  they will be rendered as schema-driven property fields
2026-02-11 09:50:31 -06:00
a49680b274 Merge pull request 'main' (#58) from main into production
Reviewed-on: #58
2026-02-10 00:45:44 +00:00
701a5c21ce Merge pull request 'fix(sse): disable read deadline for long-lived connections' (#56) from fix-sse-read-deadline into main
Reviewed-on: #56
2026-02-09 17:39:09 +00:00
Forbes
f7aa673d2c fix(sse): disable read deadline for long-lived SSE connections
The server's ReadTimeout (15s) was closing SSE connections shortly after
they were established, causing a rapid connect/disconnect loop. The handler
already disabled WriteTimeout but not ReadTimeout.
2026-02-08 22:52:42 -06:00
2157b40d06 Merge pull request 'feat(web): BOM merge resolution UI (#47)' (#55) from issue-47-bom-merge-ui into main
Reviewed-on: #55
2026-02-09 02:09:02 +00:00
Forbes
25c42bd70b feat(web): add BOM merge resolution UI with source badges and dropdown
- Add source badges (assembly=teal, manual=blue) to BOM display rows
- Add info banner when assembly-sourced entries exist
- Change source input from text field to select dropdown
- Add merge response types to types.ts

Closes #47
2026-02-08 19:56:33 -06:00
8d88f77ff6 Merge pull request 'feat: expose file attachment stats as item properties' (#54) from issue-37-file-stats into main
Reviewed-on: #54
2026-02-09 01:26:17 +00:00
Forbes
50985ed805 feat: expose file attachment stats as item properties (#37)
Add file_count and files_total_size to item API responses, computed
via batch query on item_files table (no migration needed).

- Add BatchGetFileStats() to audit_queries.go (follows BatchCheckBOM pattern)
- Add file stats to ItemResponse, HandleListItems, HandleGetItem, HandleGetItemByUUID
- Add 'Files' column to ItemTable (default visible in vertical mode)
- Add has_files computed field to audit completeness scoring (weight 1 for manufactured)
2026-02-08 19:25:46 -06:00
32bc00caef Merge pull request 'main' (#53) from main into production
Reviewed-on: #53
2026-02-09 01:23:28 +00:00
9be6f45f09 Merge pull request 'chore(docs): delete stale documentation files' (#52) from issue-31-delete-dead-docs into main
Reviewed-on: #52
2026-02-09 01:22:02 +00:00
ef05aec619 Merge branch 'main' into issue-31-delete-dead-docs 2026-02-09 01:21:52 +00:00
64075d88b5 Merge pull request 'feat(api): add POST /api/items/{partNumber}/bom/merge endpoint' (#51) from issue-45-bom-merge into main
Reviewed-on: #51
2026-02-09 01:21:44 +00:00
eac64f863b Merge branch 'main' into issue-45-bom-merge 2026-02-09 01:21:38 +00:00
aa414adc43 Merge pull request 'feat(db): add source column to relationships table' (#50) from issue-44-bom-source into main
Reviewed-on: #50
2026-02-09 01:21:30 +00:00
Forbes
08e84703d5 chore(docs): delete stale REPOSITORY_STATUS.md (#31)
Generated 2026-01-31, references HTML templates and 8 migrations
that are now outdated. Superseded by STATUS.md and SPECIFICATION.md.

API.md and silo-spec.md were already deleted in earlier commits.
2026-02-08 19:17:53 -06:00
Forbes
fbe4f3a36c feat(api): add POST /api/items/{partNumber}/bom/merge endpoint (#45)
Add BOM merge endpoint for syncing assembly-derived BOM entries from
FreeCAD's silo-mod plugin.

Merge rules:
- Added: entries in request but not in server BOM are auto-created
  with source='assembly'
- Quantity changed: existing entries with different quantity are
  auto-updated
- Unchanged: same part and quantity are skipped
- Unreferenced: assembly-sourced entries in server BOM but not in
  request are flagged as warnings (never auto-deleted)
- Manual entries are silently ignored in unreferenced detection

Also emits SSE 'bom.merged' event on successful merge (#46).
2026-02-08 19:15:27 -06:00
2b7a9ae73a Merge pull request 'main' (#41) from main into production
Reviewed-on: #41
2026-02-08 22:02:18 +00:00
181 changed files with 24575 additions and 2688 deletions

View File

@@ -1,12 +1,22 @@
# Silo Environment Configuration
# Copy this file to .env and update values as needed
# Copy to .env (or deployments/.env) and update values as needed.
# For automated setup, run: ./scripts/setup-docker.sh
# PostgreSQL
POSTGRES_PASSWORD=silodev
# MinIO
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
# OpenLDAP
LDAP_ADMIN_PASSWORD=ldapadmin
LDAP_USERS=siloadmin
LDAP_PASSWORDS=siloadmin
# Silo API (optional overrides)
# SILO_SERVER_PORT=8080
# Silo Authentication
SILO_SESSION_SECRET=change-me-in-production
SILO_ADMIN_USERNAME=admin
SILO_ADMIN_PASSWORD=admin
# Optional: OIDC (Keycloak)
# SILO_OIDC_CLIENT_SECRET=
# Optional: LDAP service account
# SILO_LDAP_BIND_PASSWORD=

2
.gitignore vendored
View File

@@ -1,6 +1,7 @@
# Binaries
/silo
/silod
/migrate-storage
*.exe
*.dll
*.so
@@ -29,6 +30,7 @@ Thumbs.db
# Config with secrets
config.yaml
*.env
deployments/config.docker.yaml
# Python
__pycache__/

View File

@@ -11,6 +11,7 @@
build: web-build
go build -o silo ./cmd/silo
go build -o silod ./cmd/silod
go build -o silorunner ./cmd/silorunner
# Run the API server locally
run:
@@ -30,7 +31,7 @@ test-integration:
# Clean build artifacts
clean:
rm -f silo silod
rm -f silo silod silorunner
rm -f *.out
rm -rf web/dist
@@ -67,7 +68,7 @@ db-shell:
docker-build:
docker build -t silo:latest -f build/package/Dockerfile .
# Start the full stack (postgres + minio + silo)
# Start the full stack (postgres + silo)
docker-up:
docker compose -f deployments/docker-compose.yaml up -d
@@ -94,9 +95,6 @@ docker-logs-silo:
docker-logs-postgres:
docker compose -f deployments/docker-compose.yaml logs -f postgres
docker-logs-minio:
docker compose -f deployments/docker-compose.yaml logs -f minio
# Show running containers
docker-ps:
docker compose -f deployments/docker-compose.yaml ps
@@ -166,7 +164,7 @@ help:
@echo ""
@echo "Docker:"
@echo " docker-build - Build Docker image"
@echo " docker-up - Start full stack (postgres + minio + silo)"
@echo " docker-up - Start full stack (postgres + silo)"
@echo " docker-down - Stop the stack"
@echo " docker-clean - Stop and remove volumes (deletes data)"
@echo " docker-logs - View all logs"

View File

@@ -25,7 +25,7 @@ silo/
│ ├── silo/ # CLI tool
│ └── silod/ # API server
├── internal/
│ ├── api/ # HTTP handlers and routes (75 endpoints)
│ ├── api/ # HTTP handlers and routes (78 endpoints)
│ ├── auth/ # Authentication (local, LDAP, OIDC)
│ ├── config/ # Configuration loading
│ ├── db/ # PostgreSQL repositories
@@ -34,7 +34,7 @@ silo/
│ ├── ods/ # ODS spreadsheet library
│ ├── partnum/ # Part number generation
│ ├── schema/ # YAML schema parsing
│ ├── storage/ # MinIO file storage
│ ├── storage/ # Filesystem storage
│ └── testutil/ # Test helpers
├── web/ # React SPA (Vite + TypeScript)
│ └── src/
@@ -53,15 +53,20 @@ silo/
## Quick Start
```bash
# Docker Compose (quickest)
cp config.example.yaml config.yaml
# Edit config.yaml with your database, MinIO, and auth settings
make docker-up
See the **[Installation Guide](docs/INSTALL.md)** for complete setup instructions.
# Or manual setup
psql -h localhost -U silo -d silo -f migrations/*.sql
go run ./cmd/silod -config config.yaml
**Docker Compose (quickest — includes PostgreSQL, OpenLDAP, and Silo):**
```bash
./scripts/setup-docker.sh
docker compose -f deployments/docker-compose.allinone.yaml up -d
```
**Development (local Go + Docker services):**
```bash
make docker-up # Start PostgreSQL in Docker
make run # Run silo locally with Go
```
When auth is enabled, a default admin account is created on first startup using the credentials in `config.yaml` under `auth.local.default_admin_username` and `auth.local.default_admin_password`.
@@ -104,15 +109,16 @@ The server provides the REST API and ODS endpoints consumed by these clients.
| Document | Description |
|----------|-------------|
| [docs/INSTALL.md](docs/INSTALL.md) | Installation guide (Docker Compose and daemon) |
| [docs/SPECIFICATION.md](docs/SPECIFICATION.md) | Full design specification and API reference |
| [docs/STATUS.md](docs/STATUS.md) | Implementation status |
| [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) | Production deployment guide |
| [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) | Production deployment and operations guide |
| [docs/CONFIGURATION.md](docs/CONFIGURATION.md) | Configuration reference (all `config.yaml` options) |
| [docs/AUTH.md](docs/AUTH.md) | Authentication system design |
| [docs/AUTH_USER_GUIDE.md](docs/AUTH_USER_GUIDE.md) | User guide for login, tokens, and roles |
| [docs/GAP_ANALYSIS.md](docs/GAP_ANALYSIS.md) | Gap analysis and revision control roadmap |
| [docs/COMPONENT_AUDIT.md](docs/COMPONENT_AUDIT.md) | Component audit tool design |
| [ROADMAP.md](ROADMAP.md) | Feature roadmap and SOLIDWORKS PDM comparison |
| [docs/ROADMAP.md](docs/ROADMAP.md) | Platform roadmap, dependency tiers, and gap summary |
| [frontend-spec.md](frontend-spec.md) | React SPA frontend specification |
## License

View File

@@ -1,536 +0,0 @@
# Silo Roadmap
**Version:** 1.1
**Date:** February 2026
**Purpose:** Project inventory, SOLIDWORKS PDM gap analysis, and development roadmap
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Current Project Inventory](#current-project-inventory)
3. [SOLIDWORKS PDM Gap Analysis](#solidworks-pdm-gap-analysis)
4. [Feature Roadmap](#feature-roadmap)
5. [Implementation Phases](#implementation-phases)
---
## Executive Summary
Silo is an R&D-oriented item database and part management system. It provides configurable part number generation, revision tracking, BOM management, and file versioning through MinIO storage. CAD integration (FreeCAD workbench, LibreOffice Calc extension) is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)).
This document compares Silo's current capabilities against SOLIDWORKS PDM—the industry-leading product data management solution—to identify gaps and prioritize future development.
### Key Differentiators
| Aspect | Silo | SOLIDWORKS PDM |
|--------|------|----------------|
| **Target CAD** | FreeCAD / Kindred Create (open source) | SOLIDWORKS (proprietary) |
| **Part Numbering** | Schema-as-configuration (YAML) | Fixed format with some customization |
| **Licensing** | Open source / Kindred Proprietary | Commercial ($3,000-$10,000+ per seat) |
| **Storage** | PostgreSQL + MinIO (S3-compatible) | SQL Server + File Archive |
| **Philosophy** | R&D-oriented, lightweight | Enterprise-grade, comprehensive |
---
## Current Project Inventory
### Implemented Features (MVP Complete)
#### Core Database System
- PostgreSQL schema with 11 migrations
- UUID-based identifiers throughout
- Soft delete support via `archived_at` timestamps
- Atomic sequence generation for part numbers
#### Part Number Generation
- YAML schema parser with validation
- Segment types: `string`, `enum`, `serial`, `constant`
- Scope templates for serial counters (e.g., `{category}`, `{project}`)
- Format templates for custom output
#### Item Management
- Full CRUD operations for items
- Item types: part, assembly, drawing, document, tooling, purchased, electrical, software
- Custom properties via JSONB storage
- Project tagging with many-to-many relationships
#### Revision Control
- Append-only revision history
- Revision metadata: properties, file reference, checksum, comment
- Status tracking: draft, review, released, obsolete
- Labels/tags per revision
- Revision comparison (diff)
- Rollback functionality
#### File Management
- MinIO integration with versioning
- File upload/download via REST API
- SHA256 checksums for integrity
- Storage path: `items/{partNumber}/rev{N}.FCStd`
#### Bill of Materials (BOM)
- Relationship types: component, alternate, reference
- Multi-level BOM (recursive expansion with configurable depth)
- Where-used queries (reverse parent lookup)
- BOM CSV and ODS export/import with cycle detection
- Reference designators for electronics
- Quantity tracking with units
- Revision-specific child linking
#### Project Management
- Project CRUD operations
- Unique project codes (2-10 characters)
- Item-to-project tagging
- Project-filtered queries
#### Data Import/Export
- CSV export with configurable properties
- CSV import with dry-run validation
- ODS spreadsheet import/export (items, BOMs, project sheets)
- Template generation for import formatting
#### API & Web Interface
- REST API with 75 endpoints
- Authentication: local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak
- Role-based access control (admin > editor > viewer)
- API token management (SHA-256 hashed)
- Session management (PostgreSQL-backed, 24h lifetime)
- CSRF protection (nosurf on web forms)
- Middleware: logging, CORS, recovery, request ID
- Web UI — React SPA (Vite + TypeScript, Catppuccin Mocha theme)
- Fuzzy search
- Health and readiness probes
#### Audit & Completeness
- Audit logging (database table with user/action/resource tracking)
- Item completeness scoring with weighted fields
- Category-specific property validation
- Tier classification (critical/low/partial/good/complete)
#### Configuration
- YAML configuration with environment variable overrides
- Multi-schema support
- Docker Compose deployment ready
### Partially Implemented
| Feature | Status | Notes |
|---------|--------|-------|
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull sync operations are stubs |
| Date segment type | Not started | Schema parser placeholder exists |
| Part number validation | Not started | API accepts but doesn't validate format |
| Location hierarchy CRUD | Schema only | Tables exist, no API endpoints |
| Inventory tracking | Schema only | Tables exist, no API endpoints |
| Unit tests | Partial | 9 Go test files across api, db, ods, partnum, schema packages |
### Infrastructure Status
| Component | Status |
|-----------|--------|
| PostgreSQL | Running (psql.kindred.internal) |
| MinIO | Configured in Docker Compose |
| Silo API Server | Builds successfully |
| Docker Compose | Complete (dev and production) |
| systemd service | Unit file and env template ready |
| Deployment scripts | setup-host, deploy, init-db, setup-ipa-nginx |
---
## SOLIDWORKS PDM Gap Analysis
This section compares Silo's capabilities against SOLIDWORKS PDM features. Gaps are categorized by priority and implementation complexity.
### Legend
- **Silo Status:** Full / Partial / None
- **Priority:** Critical / High / Medium / Low
- **Complexity:** Simple / Moderate / Complex
---
### 1. Version Control & Revision Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Check-in/check-out | Full pessimistic locking | None | High | Moderate |
| Version history | Complete with branching | Full (linear) | - | - |
| Revision labels | A, B, C or custom schemes | Full (custom labels) | - | - |
| Rollback/restore | Full | Full | - | - |
| Compare revisions | Visual + metadata diff | Metadata diff only | Medium | Complex |
| Get Latest Revision | One-click retrieval | Partial (API only) | Medium | Simple |
**Gap Analysis:**
Silo lacks pessimistic locking (check-out), which is critical for multi-user CAD environments where file merging is impractical. Visual diff comparison would require FreeCAD integration for CAD file visualization.
---
### 2. Workflow Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Custom workflows | Full visual designer | None | Critical | Complex |
| State transitions | Configurable with permissions | Basic (status field only) | Critical | Complex |
| Parallel approvals | Multiple approvers required | None | High | Complex |
| Automatic transitions | Timer/condition-based | None | Medium | Moderate |
| Email notifications | On state change | None | High | Moderate |
| ECO process | Built-in change management | None | High | Complex |
| Child state conditions | Block parent if children invalid | None | Medium | Moderate |
**Gap Analysis:**
Workflow management is the largest functional gap. SOLIDWORKS PDM offers sophisticated state machines with parallel approvals, automatic transitions, and deep integration with engineering change processes. Silo currently has only a simple status field (draft/review/released/obsolete) with no transition rules or approval processes.
---
### 3. User Management & Security
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| User authentication | Windows AD, LDAP | Full (local, LDAP, OIDC) | - | - |
| Role-based permissions | Granular per folder/state | Partial (3-tier role model) | Medium | Moderate |
| Group management | Full | None | Medium | Moderate |
| Folder permissions | Read/write/delete per folder | None | Medium | Moderate |
| State permissions | Actions allowed per state | None | High | Moderate |
| Audit trail | Complete action logging | Full | - | - |
| Private files | Pre-check-in visibility control | None | Low | Simple |
**Gap Analysis:**
Authentication is implemented with three backends (local, LDAP/FreeIPA, OIDC/Keycloak) and a 3-tier role model (admin > editor > viewer). Audit logging captures user actions. Remaining gaps: group management, folder-level permissions, and state-based permission rules.
---
### 4. Search & Discovery
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Metadata search | Full with custom cards | Partial (API query params + fuzzy) | High | Moderate |
| Full-text content search | iFilters for Office, CAD | None | Medium | Complex |
| Quick search | Toolbar with history | Partial (fuzzy search API) | Medium | Simple |
| Saved searches | User-defined favorites | None | Medium | Simple |
| Advanced operators | AND, OR, NOT, wildcards | None | Medium | Simple |
| Multi-variable search | Search across multiple fields | None | Medium | Simple |
| Where-used search | Find all assemblies using part | Full | - | - |
**Gap Analysis:**
Silo has API-level filtering, fuzzy search, and where-used queries. Remaining gaps: saved searches, advanced search operators, and a richer search UI. Content search (searching within CAD files) is not planned for the server.
---
### 5. BOM Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Single-level BOM | Yes | Full | - | - |
| Multi-level BOM | Indented/exploded views | Full (recursive, configurable depth) | - | - |
| BOM comparison | Between revisions | None | Medium | Moderate |
| BOM export | Excel, XML, ERP formats | Full (CSV, ODS) | - | - |
| BOM import | Bulk BOM loading | Full (CSV with upsert) | - | - |
| Calculated BOMs | Quantities rolled up | None | Medium | Moderate |
| Reference designators | Full support | Full | - | - |
| Alternate parts | Substitute tracking | Full | - | - |
**Gap Analysis:**
Multi-level BOM retrieval (recursive CTE with configurable depth) and BOM export (CSV, ODS) are implemented. BOM import supports CSV with upsert and cycle detection. Remaining gap: BOM comparison between revisions.
---
### 6. CAD Integration
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Native CAD add-in | Deep SOLIDWORKS integration | FreeCAD workbench (silo-mod) | Medium | Complex |
| Property mapping | Bi-directional sync | Planned (silo-mod) | Medium | Moderate |
| Task pane | Embedded in CAD UI | Auth dock panel (silo-mod) | Medium | Complex |
| Lightweight components | Handle without full load | N/A | - | - |
| Drawing/model linking | Automatic association | Manual | Medium | Moderate |
| Multi-CAD support | Third-party formats | FreeCAD only | Low | - |
**Gap Analysis:**
CAD integration is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The Silo server provides the REST API endpoints consumed by those clients.
---
### 7. External Integrations
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| ERP integration | SAP, Dynamics, etc. | Partial (Odoo stubs) | Medium | Complex |
| API access | Full COM/REST API | Full REST API (75 endpoints) | - | - |
| Dispatch scripts | Automation without coding | None | Medium | Moderate |
| Task scheduler | Background processing | None | Medium | Moderate |
| Email system | SMTP integration | None | High | Simple |
| Web portal | Browser access | Full (React SPA + auth) | - | - |
**Gap Analysis:**
Silo has a comprehensive REST API (75 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation.
---
### 8. Reporting & Analytics
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Standard reports | Inventory, usage, activity | None | Medium | Moderate |
| Custom reports | User-defined queries | None | Medium | Moderate |
| Dashboard | Visual KPIs | None | Low | Moderate |
| Export formats | PDF, Excel, CSV | CSV and ODS | Medium | Simple |
**Gap Analysis:**
Reporting capabilities are absent. Basic reports (item counts, revision activity, where-used) would provide immediate value.
---
### 9. File Handling
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| File versioning | Automatic | Full (MinIO) | - | - |
| File preview | Thumbnails, 3D preview | None | Medium | Complex |
| File conversion | PDF, DXF generation | None | Medium | Complex |
| Replication | Multi-site sync | None | Low | Complex |
| File copy with refs | Copy tree with references | None | Medium | Moderate |
**Gap Analysis:**
File storage works well. Thumbnail generation and file preview would significantly improve the web UI experience. Automatic conversion to PDF/DXF is valuable for sharing with non-CAD users.
---
### Gap Summary by Priority
#### Completed (Previously Critical/High)
1. ~~**User authentication**~~ - Implemented: local, LDAP, OIDC
2. ~~**Role-based permissions**~~ - Implemented: 3-tier role model (admin/editor/viewer)
3. ~~**Audit trail**~~ - Implemented: audit_log table with completeness scoring
4. ~~**Where-used search**~~ - Implemented: reverse parent lookup API
5. ~~**Multi-level BOM API**~~ - Implemented: recursive expansion with configurable depth
6. ~~**BOM export**~~ - Implemented: CSV and ODS formats
#### Critical Gaps (Required for Team Use)
1. **Workflow engine** - State machines with transitions and approvals
2. **Check-out locking** - Pessimistic locking for CAD files
#### High Priority Gaps (Significant Value)
1. **Email notifications** - Alert users on state changes
2. **Web UI search** - Advanced search interface with saved searches
3. **Folder/state permissions** - Granular access control beyond role model
#### Medium Priority Gaps (Nice to Have)
1. **Saved searches** - Frequently used queries
2. **File preview/thumbnails** - Visual browsing
3. **Reporting** - Activity and inventory reports
4. **Scheduled tasks** - Background automation
5. **BOM comparison** - Revision diff for assemblies
---
## Feature Roadmap
### Phase 1: Foundation (Current - Q2 2026)
*Complete MVP and stabilize core functionality*
| Feature | Description | Status |
|---------|-------------|--------|
| MinIO integration | File upload/download with versioning and checksums | Complete |
| Revision control | Rollback, comparison, status/labels | Complete |
| CSV import/export | Dry-run validation, template generation | Complete |
| ODS import/export | Items, BOMs, project sheets, templates | Complete |
| Project management | CRUD, many-to-many item tagging | Complete |
| Multi-level BOM | Recursive expansion, where-used, export | Complete |
| Authentication | Local, LDAP, OIDC with role-based access | Complete |
| Audit logging | Action logging, completeness scoring | Complete |
| Unit tests | Core API and database operations | Not Started |
| Date segment type | Support date-based part number segments | Not Started |
| Part number validation | Validate format on creation | Not Started |
| Location CRUD API | Expose location hierarchy via REST | Not Started |
| Inventory API | Expose inventory operations via REST | Not Started |
### Phase 2: Multi-User (Q2-Q3 2026)
*Enable team collaboration*
| Feature | Description | Status |
|---------|-------------|--------|
| LDAP authentication | Integrate with FreeIPA/Active Directory | **Complete** |
| OIDC authentication | Keycloak / OpenID Connect | **Complete** |
| Audit logging | Record all user actions with timestamps | **Complete** |
| Session management | Token-based and session-based API authentication | **Complete** |
| User/group management | Create, assign, manage users and groups | Not Started |
| Folder permissions | Read/write/delete per folder hierarchy | Not Started |
| Check-out locking | Pessimistic locks with timeout | Not Started |
### Phase 3: Workflow Engine (Q3-Q4 2026)
*Implement engineering change processes*
| Feature | Description | Complexity |
|---------|-------------|------------|
| Workflow designer | YAML-defined state machines | Complex |
| State transitions | Configurable transition rules | Complex |
| Transition permissions | Who can execute which transitions | Moderate |
| Single approvals | Basic approval workflow | Moderate |
| Parallel approvals | Multi-approver gates | Complex |
| Automatic transitions | Timer and condition-based | Complex |
| Email notifications | SMTP integration for alerts | Simple |
| Child state conditions | Block parent transitions | Moderate |
### Phase 4: Search & Discovery (Q4 2026 - Q1 2027)
*Improve findability and navigation*
| Feature | Description | Status |
|---------|-------------|--------|
| Where-used queries | Find parent assemblies | **Complete** |
| Fuzzy search | Quick search across items | **Complete** |
| Advanced search UI | Web interface with filters | Not Started |
| Search operators | AND, OR, NOT, wildcards | Not Started |
| Saved searches | User favorites | Not Started |
| Content search | Search within file content | Not Started |
### Phase 5: BOM & Reporting (Q1-Q2 2027)
*Enhanced BOM management and analytics*
| Feature | Description | Status |
|---------|-------------|--------|
| Multi-level BOM API | Recursive assembly retrieval | **Complete** |
| BOM export | CSV and ODS formats | **Complete** |
| BOM import | CSV with upsert and cycle detection | **Complete** |
| BOM comparison | Diff between revisions | Not Started |
| Standard reports | Activity, inventory, usage | Not Started |
| Custom queries | User-defined report builder | Not Started |
| Dashboard | Visual KPIs and metrics | Not Started |
### Phase 6: Advanced Features (Q2-Q4 2027)
*Enterprise capabilities*
| Feature | Description | Complexity |
|---------|-------------|------------|
| File preview | Thumbnail generation | Complex |
| File conversion | Auto-generate PDF/DXF | Complex |
| ERP integration | Adapter framework | Complex |
| Task scheduler | Background job processing | Moderate |
| Webhooks | Event notifications to external systems | Moderate |
| API rate limiting | Protect against abuse | Simple |
---
## Implementation Phases
### Phase 1 Detailed Tasks
#### 1.1 MinIO Integration -- COMPLETE
- [x] MinIO service configured in Docker Compose
- [x] File upload via REST API
- [x] File download via REST API (latest and by revision)
- [x] SHA256 checksums on upload
#### 1.2 Authentication & Authorization -- COMPLETE
- [x] Local authentication (bcrypt)
- [x] LDAP/FreeIPA authentication
- [x] OIDC/Keycloak authentication
- [x] Role-based access control (admin/editor/viewer)
- [x] API token management (SHA-256 hashed)
- [x] Session management (PostgreSQL-backed)
- [x] CSRF protection (nosurf)
- [x] Audit logging (database table)
#### 1.3 Multi-level BOM & Export -- COMPLETE
- [x] Recursive BOM expansion with configurable depth
- [x] Where-used reverse lookup
- [x] BOM CSV export/import with cycle detection
- [x] BOM ODS export
- [x] ODS item export/import/template
#### 1.4 Unit Test Suite
- [ ] Database connection and transaction tests
- [ ] Item CRUD operation tests
- [ ] Revision creation and retrieval tests
- [ ] Part number generation tests
- [ ] File upload/download tests
- [ ] CSV import/export tests
- [ ] API endpoint tests
#### 1.5 Missing Segment Types
- [ ] Implement date segment type
- [ ] Add strftime-style format support
#### 1.6 Location & Inventory APIs
- [ ] `GET /api/locations` - List locations
- [ ] `POST /api/locations` - Create location
- [ ] `GET /api/locations/{path}` - Get location
- [ ] `DELETE /api/locations/{path}` - Delete location
- [ ] `GET /api/inventory/{partNumber}` - Get inventory
- [ ] `POST /api/inventory/{partNumber}/adjust` - Adjust quantity
- [ ] `POST /api/inventory/{partNumber}/move` - Move between locations
---
## Success Metrics
### Phase 1 (Foundation)
- All existing tests pass
- File upload/download works end-to-end
- FreeCAD users can checkout, modify, commit parts
### Phase 2 (Multi-User)
- 5+ concurrent users supported
- No data corruption under concurrent access
- Audit log captures all modifications
### Phase 3 (Workflow)
- Engineering change process completable in Silo
- Email notifications delivered reliably
- Workflow state visible in web UI
### Phase 4+ (Advanced)
- Search returns results in <2 seconds
- Where-used queries complete in <5 seconds
- BOM export matches assembly structure
---
## References
### SOLIDWORKS PDM Documentation
- [SOLIDWORKS PDM Product Page](https://www.solidworks.com/product/solidworks-pdm)
- [What's New in SOLIDWORKS PDM 2025](https://blogs.solidworks.com/solidworksblog/2024/10/whats-new-in-solidworks-pdm-2025.html)
- [Top 5 Enhancements in SOLIDWORKS PDM 2024](https://blogs.solidworks.com/solidworksblog/2023/10/top-5-enhancements-in-solidworks-pdm-2024.html)
- [SOLIDWORKS PDM Workflow Transitions](https://help.solidworks.com/2023/english/EnterprisePDM/Admin/c_workflow_transition.htm)
- [Ultimate Guide to SOLIDWORKS PDM Permissions](https://www.goengineer.com/blog/ultimate-guide-to-solidworks-pdm-permissions)
- [Searching in SOLIDWORKS PDM](https://help.solidworks.com/2021/english/EnterprisePDM/fileexplorer/c_searches.htm)
- [SOLIDWORKS PDM API Getting Started](https://3dswym.3dexperience.3ds.com/wiki/solidworks-news-info/getting-started-with-the-solidworks-pdm-api-solidpractices_gBCYaM75RgORBcpSO1m_Mw)
### Silo Documentation
- [Specification](docs/SPECIFICATION.md)
- [Development Status](docs/STATUS.md)
- [Deployment Guide](docs/DEPLOYMENT.md)
- [Gap Analysis](docs/GAP_ANALYSIS.md)
---
## Appendix: Feature Comparison Matrix
| Category | Feature | SW PDM Standard | SW PDM Pro | Silo Current | Silo Planned |
|----------|---------|-----------------|------------|--------------|--------------|
| **Version Control** | Check-in/out | Yes | Yes | No | Phase 2 |
| | Version history | Yes | Yes | Yes | - |
| | Rollback | Yes | Yes | Yes | - |
| | Revision labels/status | Yes | Yes | Yes | - |
| | Revision comparison | Yes | Yes | Yes (metadata) | - |
| **Workflow** | Custom workflows | Limited | Yes | No | Phase 3 |
| | Parallel approval | No | Yes | No | Phase 3 |
| | Notifications | No | Yes | No | Phase 3 |
| **Security** | User auth | Windows | Windows/LDAP | Yes (local, LDAP, OIDC) | - |
| | Permissions | Basic | Granular | Partial (role-based) | Phase 2 |
| | Audit trail | Basic | Full | Yes | - |
| **Search** | Metadata search | Yes | Yes | Partial (API + fuzzy) | Phase 4 |
| | Content search | No | Yes | No | Phase 4 |
| | Where-used | Yes | Yes | Yes | - |
| **BOM** | Single-level | Yes | Yes | Yes | - |
| | Multi-level | Yes | Yes | Yes (recursive) | - |
| | BOM export | Yes | Yes | Yes (CSV, ODS) | - |
| **Data** | CSV import/export | Yes | Yes | Yes | - |
| | ODS import/export | No | No | Yes | - |
| | Project management | Yes | Yes | Yes | - |
| **Integration** | API | Limited | Full | Full REST (75) | - |
| | ERP connectors | No | Yes | Partial (Odoo stubs) | Phase 6 |
| | Web access | No | Yes | Yes (React SPA + auth) | - |
| **Files** | Versioning | Yes | Yes | Yes | - |
| | Preview | Yes | Yes | No | Phase 6 |
| | Multi-site | No | Yes | No | Not Planned |

View File

@@ -66,7 +66,7 @@ Token subcommands:
silo token revoke <id> Revoke a token
Environment variables for API access:
SILO_API_URL Base URL of the Silo server (e.g., https://silo.kindred.internal)
SILO_API_URL Base URL of the Silo server (e.g., https://silo.example.internal)
SILO_API_TOKEN API token for authentication
Examples:

View File

@@ -3,6 +3,7 @@ package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"net/http"
@@ -13,12 +14,16 @@ import (
"github.com/alexedwards/scs/pgxstore"
"github.com/alexedwards/scs/v2"
"github.com/kindredsystems/silo/internal/api"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/config"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/jobdef"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/storage"
"github.com/kindredsystems/silo/internal/workflow"
"github.com/rs/zerolog"
)
@@ -40,7 +45,6 @@ func main() {
Str("host", cfg.Server.Host).
Int("port", cfg.Server.Port).
Str("database", cfg.Database.Host).
Str("storage", cfg.Storage.Endpoint).
Msg("starting silo server")
// Connect to database
@@ -60,23 +64,15 @@ func main() {
defer database.Close()
logger.Info().Msg("connected to database")
// Connect to storage (optional - may be externally managed)
var store *storage.Storage
if cfg.Storage.Endpoint != "" {
store, err = storage.Connect(ctx, storage.Config{
Endpoint: cfg.Storage.Endpoint,
AccessKey: cfg.Storage.AccessKey,
SecretKey: cfg.Storage.SecretKey,
Bucket: cfg.Storage.Bucket,
UseSSL: cfg.Storage.UseSSL,
Region: cfg.Storage.Region,
})
if err != nil {
logger.Warn().Err(err).Msg("failed to connect to storage - file operations disabled")
store = nil
} else {
logger.Info().Msg("connected to storage")
// Connect to storage (optional — requires root_dir to be set)
var store storage.FileStore
if cfg.Storage.Filesystem.RootDir != "" {
s, fsErr := storage.NewFilesystemStore(cfg.Storage.Filesystem.RootDir)
if fsErr != nil {
logger.Fatal().Err(fsErr).Msg("failed to initialize filesystem storage")
}
store = s
logger.Info().Str("root", cfg.Storage.Filesystem.RootDir).Msg("connected to filesystem storage")
} else {
logger.Info().Msg("storage not configured - file operations disabled")
}
@@ -178,6 +174,67 @@ func main() {
}
}
// Load job definitions (optional — directory may not exist yet)
var jobDefs map[string]*jobdef.Definition
if _, err := os.Stat(cfg.Jobs.Directory); err == nil {
jobDefs, err = jobdef.LoadAll(cfg.Jobs.Directory)
if err != nil {
logger.Fatal().Err(err).Str("directory", cfg.Jobs.Directory).Msg("failed to load job definitions")
}
logger.Info().Int("count", len(jobDefs)).Msg("loaded job definitions")
} else {
jobDefs = make(map[string]*jobdef.Definition)
logger.Info().Str("directory", cfg.Jobs.Directory).Msg("job definitions directory not found, skipping")
}
// Upsert job definitions into database
jobRepo := db.NewJobRepository(database)
for _, def := range jobDefs {
defJSON, _ := json.Marshal(def)
var defMap map[string]any
json.Unmarshal(defJSON, &defMap)
rec := &db.JobDefinitionRecord{
Name: def.Name,
Version: def.Version,
TriggerType: def.Trigger.Type,
ScopeType: def.Scope.Type,
ComputeType: def.Compute.Type,
RunnerTags: def.Runner.Tags,
TimeoutSeconds: def.Timeout,
MaxRetries: def.MaxRetries,
Priority: def.Priority,
Definition: defMap,
Enabled: true,
}
if err := jobRepo.UpsertDefinition(ctx, rec); err != nil {
logger.Fatal().Err(err).Str("name", def.Name).Msg("failed to upsert job definition")
}
}
// Load approval workflow definitions (optional — directory may not exist yet)
var workflows map[string]*workflow.Workflow
if _, err := os.Stat(cfg.Workflows.Directory); err == nil {
workflows, err = workflow.LoadAll(cfg.Workflows.Directory)
if err != nil {
logger.Fatal().Err(err).Str("directory", cfg.Workflows.Directory).Msg("failed to load workflow definitions")
}
logger.Info().Int("count", len(workflows)).Msg("loaded workflow definitions")
} else {
workflows = make(map[string]*workflow.Workflow)
logger.Info().Str("directory", cfg.Workflows.Directory).Msg("workflows directory not found, skipping")
}
// Initialize module registry
registry := modules.NewRegistry()
if err := modules.LoadState(registry, cfg, database.Pool()); err != nil {
logger.Fatal().Err(err).Msg("failed to load module state")
}
for _, m := range registry.All() {
logger.Info().Str("module", m.ID).Bool("enabled", registry.IsEnabled(m.ID)).
Bool("required", m.Required).Msg("module")
}
// Create SSE broker and server state
broker := api.NewBroker(logger)
serverState := api.NewServerState(logger, store, broker)
@@ -190,9 +247,32 @@ func main() {
// Create API server
server := api.NewServer(logger, database, schemas, cfg.Schemas.Directory, store,
authService, sessionManager, oidcBackend, &cfg.Auth, broker, serverState)
authService, sessionManager, oidcBackend, &cfg.Auth, broker, serverState,
jobDefs, cfg.Jobs.Directory, registry, cfg, workflows)
router := api.NewRouter(server, logger)
// Start background sweepers for job/runner timeouts (only when jobs module enabled)
if registry.IsEnabled(modules.Jobs) {
go func() {
ticker := time.NewTicker(time.Duration(cfg.Jobs.JobTimeoutCheck) * time.Second)
defer ticker.Stop()
for range ticker.C {
if n, err := jobRepo.TimeoutExpiredJobs(ctx); err != nil {
logger.Error().Err(err).Msg("job timeout sweep failed")
} else if n > 0 {
logger.Info().Int64("count", n).Msg("timed out expired jobs")
}
if n, err := jobRepo.ExpireStaleRunners(ctx, time.Duration(cfg.Jobs.RunnerTimeout)*time.Second); err != nil {
logger.Error().Err(err).Msg("runner expiry sweep failed")
} else if n > 0 {
logger.Info().Int64("count", n).Msg("expired stale runners")
}
}
}()
logger.Info().Msg("job/runner sweepers started")
}
// Create HTTP server
addr := fmt.Sprintf("%s:%d", cfg.Server.Host, cfg.Server.Port)
httpServer := &http.Server{

330
cmd/silorunner/main.go Normal file
View File

@@ -0,0 +1,330 @@
// Command silorunner is a compute worker that polls the Silo server for jobs
// and executes them using Headless Create with silo-mod installed.
package main
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/rs/zerolog"
"gopkg.in/yaml.v3"
)
// RunnerConfig holds runner configuration.
type RunnerConfig struct {
ServerURL string `yaml:"server_url"`
Token string `yaml:"token"`
Name string `yaml:"name"`
Tags []string `yaml:"tags"`
PollInterval int `yaml:"poll_interval"` // seconds, default 5
CreatePath string `yaml:"create_path"` // path to Headless Create binary
}
func main() {
configPath := flag.String("config", "runner.yaml", "Path to runner config file")
flag.Parse()
logger := zerolog.New(os.Stdout).With().Timestamp().Str("component", "silorunner").Logger()
// Load config
cfg, err := loadConfig(*configPath)
if err != nil {
logger.Fatal().Err(err).Msg("failed to load config")
}
if cfg.ServerURL == "" {
logger.Fatal().Msg("server_url is required")
}
if cfg.Token == "" {
logger.Fatal().Msg("token is required")
}
if cfg.Name == "" {
hostname, _ := os.Hostname()
cfg.Name = "runner-" + hostname
}
if cfg.PollInterval <= 0 {
cfg.PollInterval = 5
}
logger.Info().
Str("server", cfg.ServerURL).
Str("name", cfg.Name).
Strs("tags", cfg.Tags).
Int("poll_interval", cfg.PollInterval).
Msg("starting runner")
client := &http.Client{Timeout: 30 * time.Second}
// Graceful shutdown
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
// Heartbeat goroutine
go func() {
ticker := time.NewTicker(30 * time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
if err := heartbeat(client, cfg); err != nil {
logger.Error().Err(err).Msg("heartbeat failed")
}
case <-quit:
return
}
}
}()
// Initial heartbeat
if err := heartbeat(client, cfg); err != nil {
logger.Warn().Err(err).Msg("initial heartbeat failed")
}
// Poll loop
ticker := time.NewTicker(time.Duration(cfg.PollInterval) * time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
job, definition, err := claimJob(client, cfg)
if err != nil {
logger.Error().Err(err).Msg("claim failed")
continue
}
if job == nil {
continue
}
jobID, _ := job["id"].(string)
defName, _ := job["definition_name"].(string)
logger.Info().Str("job_id", jobID).Str("definition", defName).Msg("claimed job")
// Start the job
if err := startJob(client, cfg, jobID); err != nil {
logger.Error().Err(err).Str("job_id", jobID).Msg("failed to start job")
continue
}
// Execute the job
executeJob(logger, client, cfg, jobID, job, definition)
case <-quit:
logger.Info().Msg("shutting down")
return
}
}
}
func loadConfig(path string) (*RunnerConfig, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("reading config: %w", err)
}
data = []byte(os.ExpandEnv(string(data)))
var cfg RunnerConfig
if err := yaml.Unmarshal(data, &cfg); err != nil {
return nil, fmt.Errorf("parsing config: %w", err)
}
return &cfg, nil
}
func heartbeat(client *http.Client, cfg *RunnerConfig) error {
req, err := http.NewRequest("POST", cfg.ServerURL+"/api/runner/heartbeat", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+cfg.Token)
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf("heartbeat: %d %s", resp.StatusCode, string(body))
}
return nil
}
func claimJob(client *http.Client, cfg *RunnerConfig) (map[string]any, map[string]any, error) {
req, err := http.NewRequest("POST", cfg.ServerURL+"/api/runner/claim", nil)
if err != nil {
return nil, nil, err
}
req.Header.Set("Authorization", "Bearer "+cfg.Token)
resp, err := client.Do(req)
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusNoContent {
return nil, nil, nil // No jobs available
}
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, nil, fmt.Errorf("claim: %d %s", resp.StatusCode, string(body))
}
var result struct {
Job map[string]any `json:"job"`
Definition map[string]any `json:"definition"`
}
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return nil, nil, fmt.Errorf("decoding claim response: %w", err)
}
return result.Job, result.Definition, nil
}
func startJob(client *http.Client, cfg *RunnerConfig, jobID string) error {
req, err := http.NewRequest("POST", cfg.ServerURL+"/api/runner/jobs/"+jobID+"/start", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+cfg.Token)
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return fmt.Errorf("start: %d %s", resp.StatusCode, string(body))
}
return nil
}
func reportProgress(client *http.Client, cfg *RunnerConfig, jobID string, progress int, message string) {
body, _ := json.Marshal(map[string]any{
"progress": progress,
"message": message,
})
req, _ := http.NewRequest("PUT", cfg.ServerURL+"/api/runner/jobs/"+jobID+"/progress", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+cfg.Token)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return
}
resp.Body.Close()
}
func completeJob(client *http.Client, cfg *RunnerConfig, jobID string, result map[string]any) error {
body, _ := json.Marshal(map[string]any{"result": result})
req, err := http.NewRequest("POST", cfg.ServerURL+"/api/runner/jobs/"+jobID+"/complete", bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+cfg.Token)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
respBody, _ := io.ReadAll(resp.Body)
return fmt.Errorf("complete: %d %s", resp.StatusCode, string(respBody))
}
return nil
}
func failJob(client *http.Client, cfg *RunnerConfig, jobID string, errMsg string) {
body, _ := json.Marshal(map[string]string{"error": errMsg})
req, _ := http.NewRequest("POST", cfg.ServerURL+"/api/runner/jobs/"+jobID+"/fail", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+cfg.Token)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return
}
resp.Body.Close()
}
func appendLog(client *http.Client, cfg *RunnerConfig, jobID, level, message string) {
body, _ := json.Marshal(map[string]string{
"level": level,
"message": message,
})
req, _ := http.NewRequest("POST", cfg.ServerURL+"/api/runner/jobs/"+jobID+"/log", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+cfg.Token)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return
}
resp.Body.Close()
}
// executeJob dispatches the job based on its compute command.
// For now, this is a stub that demonstrates the lifecycle.
// Real execution will shell out to Headless Create with silo-mod.
func executeJob(logger zerolog.Logger, client *http.Client, cfg *RunnerConfig, jobID string, job, definition map[string]any) {
defName, _ := job["definition_name"].(string)
// Extract compute config from definition
var command string
if definition != nil {
if compute, ok := definition["compute"].(map[string]any); ok {
command, _ = compute["command"].(string)
}
}
appendLog(client, cfg, jobID, "info", fmt.Sprintf("starting execution: %s (command: %s)", defName, command))
reportProgress(client, cfg, jobID, 10, "preparing")
switch command {
case "create-validate", "create-export", "create-dag-extract", "create-thumbnail":
if cfg.CreatePath == "" {
failJob(client, cfg, jobID, "create_path not configured")
return
}
appendLog(client, cfg, jobID, "info", fmt.Sprintf("would execute: %s --console with silo-mod", cfg.CreatePath))
reportProgress(client, cfg, jobID, 50, "executing")
// TODO: Actual Create execution:
// 1. Download item file from Silo API
// 2. Shell out: create --console -e "from silo.runner import <entry>; <entry>(...)"
// 3. Parse output JSON
// 4. Upload results / sync DAG
// For now, complete with a placeholder result.
reportProgress(client, cfg, jobID, 90, "finalizing")
if err := completeJob(client, cfg, jobID, map[string]any{
"status": "placeholder",
"message": "Create execution not yet implemented - runner lifecycle verified",
"command": command,
}); err != nil {
logger.Error().Err(err).Str("job_id", jobID).Msg("failed to complete job")
} else {
logger.Info().Str("job_id", jobID).Msg("job completed (placeholder)")
}
default:
failJob(client, cfg, jobID, fmt.Sprintf("unknown compute command: %s", command))
logger.Warn().Str("job_id", jobID).Str("command", command).Msg("unknown compute command")
}
}

View File

@@ -8,21 +8,18 @@ server:
# read_only: false # Reject all write operations; toggle at runtime with SIGUSR1
database:
host: "psql.kindred.internal"
host: "localhost" # Use "postgres" for Docker Compose
port: 5432
name: "silo"
user: "silo"
password: "" # Use SILO_DB_PASSWORD env var
sslmode: "require"
sslmode: "require" # Use "disable" for Docker Compose (internal network)
max_connections: 10
storage:
endpoint: "minio.kindred.internal:9000"
access_key: "" # Use SILO_MINIO_ACCESS_KEY env var
secret_key: "" # Use SILO_MINIO_SECRET_KEY env var
bucket: "silo-files"
use_ssl: true
region: "us-east-1"
backend: "filesystem"
filesystem:
root_dir: "/opt/silo/data" # Override with SILO_STORAGE_ROOT_DIR env var
schemas:
# Directory containing YAML schema files
@@ -53,7 +50,7 @@ auth:
# LDAP / FreeIPA
ldap:
enabled: false
url: "ldaps://ipa.kindred.internal"
url: "ldaps://ipa.example.internal"
base_dn: "dc=kindred,dc=internal"
user_search_dn: "cn=users,cn=accounts,dc=kindred,dc=internal"
# Optional service account for user search (omit for direct user bind)
@@ -77,10 +74,10 @@ auth:
# OIDC / Keycloak
oidc:
enabled: false
issuer_url: "https://keycloak.kindred.internal/realms/silo"
issuer_url: "https://keycloak.example.internal/realms/silo"
client_id: "silo"
client_secret: "" # Use SILO_OIDC_CLIENT_SECRET env var
redirect_url: "https://silo.kindred.internal/auth/callback"
redirect_url: "https://silo.example.internal/auth/callback"
scopes: ["openid", "profile", "email"]
# Map Keycloak realm roles to Silo roles
admin_role: "silo-admin"
@@ -90,4 +87,4 @@ auth:
# CORS origins (locked down when auth is enabled)
cors:
allowed_origins:
- "https://silo.kindred.internal"
- "https://silo.example.internal"

View File

@@ -0,0 +1,32 @@
# Silo Development Configuration
# Used by deployments/docker-compose.yaml — works with zero setup via `make docker-up`.
# For production Docker installs, run scripts/setup-docker.sh instead.
server:
host: "0.0.0.0"
port: 8080
base_url: "http://localhost:8080"
database:
host: "postgres"
port: 5432
name: "silo"
user: "silo"
password: "${POSTGRES_PASSWORD:-silodev}"
sslmode: "disable"
max_connections: 10
storage:
backend: "filesystem"
filesystem:
root_dir: "/var/lib/silo/data"
schemas:
directory: "/etc/silo/schemas"
default: "kindred-rd"
freecad:
uri_scheme: "silo"
auth:
enabled: false

View File

@@ -1,7 +1,7 @@
# Silo Production Configuration
# Single-binary deployment: silod serves API + React SPA
#
# Layout on silo.kindred.internal:
# Layout on silo.example.internal:
# /opt/silo/bin/silod - server binary
# /opt/silo/web/dist/ - built React frontend (served automatically)
# /opt/silo/schemas/ - part number schemas
@@ -10,18 +10,16 @@
#
# Credentials via environment variables (set in /etc/silo/silod.env):
# SILO_DB_PASSWORD
# SILO_MINIO_ACCESS_KEY
# SILO_MINIO_SECRET_KEY
# SILO_SESSION_SECRET
# SILO_ADMIN_PASSWORD
server:
host: "0.0.0.0"
port: 8080
base_url: "https://silo.kindred.internal"
base_url: "https://silo.example.internal"
database:
host: "psql.kindred.internal"
host: "psql.example.internal"
port: 5432
name: "silo"
user: "silo"
@@ -30,12 +28,9 @@ database:
max_connections: 20
storage:
endpoint: "minio.kindred.internal:9000"
access_key: "" # Set via SILO_MINIO_ACCESS_KEY
secret_key: "" # Set via SILO_MINIO_SECRET_KEY
bucket: "silo-files"
use_ssl: true
region: "us-east-1"
backend: "filesystem"
filesystem:
root_dir: "/opt/silo/data"
schemas:
directory: "/opt/silo/schemas"
@@ -53,7 +48,7 @@ auth:
default_admin_password: "" # Set via SILO_ADMIN_PASSWORD
ldap:
enabled: true
url: "ldaps://ipa.kindred.internal"
url: "ldaps://ipa.example.internal"
base_dn: "dc=kindred,dc=internal"
user_search_dn: "cn=users,cn=accounts,dc=kindred,dc=internal"
user_attr: "uid"
@@ -73,4 +68,4 @@ auth:
enabled: false
cors:
allowed_origins:
- "https://silo.kindred.internal"
- "https://silo.example.internal"

View File

@@ -0,0 +1,149 @@
# Silo All-in-One Stack
# PostgreSQL + OpenLDAP + Silo API + Nginx (optional)
#
# Quick start:
# ./scripts/setup-docker.sh
# docker compose -f deployments/docker-compose.allinone.yaml up -d
#
# With nginx reverse proxy:
# docker compose -f deployments/docker-compose.allinone.yaml --profile nginx up -d
#
# View logs:
# docker compose -f deployments/docker-compose.allinone.yaml logs -f
#
# Stop:
# docker compose -f deployments/docker-compose.allinone.yaml down
#
# Stop and delete data:
# docker compose -f deployments/docker-compose.allinone.yaml down -v
services:
# ---------------------------------------------------------------------------
# PostgreSQL 16
# ---------------------------------------------------------------------------
postgres:
image: postgres:16-alpine
container_name: silo-postgres
restart: unless-stopped
environment:
POSTGRES_DB: silo
POSTGRES_USER: silo
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Run ./scripts/setup-docker.sh first}
volumes:
- postgres_data:/var/lib/postgresql/data
- ../migrations:/docker-entrypoint-initdb.d:ro
healthcheck:
test: ["CMD-SHELL", "pg_isready -U silo -d silo"]
interval: 5s
timeout: 5s
retries: 5
networks:
- silo-net
# ---------------------------------------------------------------------------
# OpenLDAP (user directory for LDAP authentication)
# ---------------------------------------------------------------------------
openldap:
image: bitnami/openldap:2.6
container_name: silo-openldap
restart: unless-stopped
environment:
LDAP_ROOT: "dc=silo,dc=local"
LDAP_ADMIN_USERNAME: "admin"
LDAP_ADMIN_PASSWORD: ${LDAP_ADMIN_PASSWORD:?Run ./scripts/setup-docker.sh first}
LDAP_USERS: ${LDAP_USERS:-siloadmin}
LDAP_PASSWORDS: ${LDAP_PASSWORDS:?Run ./scripts/setup-docker.sh first}
LDAP_GROUP: "silo-users"
LDAP_USER_OU: "users"
LDAP_GROUP_OU: "groups"
volumes:
- openldap_data:/bitnami/openldap
- ./ldap:/docker-entrypoint-initdb.d:ro
ports:
- "1389:1389" # LDAP access for debugging (remove in hardened setups)
healthcheck:
test:
[
"CMD-SHELL",
"ldapsearch -x -H ldap://localhost:1389 -b dc=silo,dc=local -D cn=admin,dc=silo,dc=local -w $${LDAP_ADMIN_PASSWORD} '(objectClass=organization)' >/dev/null 2>&1",
]
interval: 10s
timeout: 5s
retries: 5
networks:
- silo-net
# ---------------------------------------------------------------------------
# Silo API Server
# ---------------------------------------------------------------------------
silo:
build:
context: ..
dockerfile: build/package/Dockerfile
container_name: silo-api
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
openldap:
condition: service_healthy
env_file:
- .env
environment:
# These override values in config.docker.yaml via the Go config loader's
# direct env var support (see internal/config/config.go).
SILO_DB_HOST: postgres
SILO_DB_NAME: silo
SILO_DB_USER: silo
SILO_DB_PASSWORD: ${POSTGRES_PASSWORD}
ports:
- "${SILO_PORT:-8080}:8080"
volumes:
- silo_data:/var/lib/silo/data
- ../schemas:/etc/silo/schemas:ro
- ./config.docker.yaml:/etc/silo/config.yaml:ro
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/health"]
interval: 10s
timeout: 5s
retries: 3
start_period: 15s
networks:
- silo-net
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
# ---------------------------------------------------------------------------
# Nginx reverse proxy (optional — enable with --profile nginx)
# ---------------------------------------------------------------------------
nginx:
image: nginx:alpine
container_name: silo-nginx
restart: unless-stopped
profiles:
- nginx
depends_on:
silo:
condition: service_healthy
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf:ro
# Uncomment to mount TLS certificates:
# - /path/to/cert.pem:/etc/nginx/ssl/cert.pem:ro
# - /path/to/key.pem:/etc/nginx/ssl/key.pem:ro
networks:
- silo-net
volumes:
postgres_data:
silo_data:
openldap_data:
networks:
silo-net:
driver: bridge

View File

@@ -1,10 +1,8 @@
# Production Docker Compose for Silo
# Uses external PostgreSQL (psql.kindred.internal) and MinIO (minio.kindred.internal)
# Uses external PostgreSQL (psql.example.internal) and filesystem storage
#
# Usage:
# export SILO_DB_PASSWORD=<your-password>
# export SILO_MINIO_ACCESS_KEY=<your-access-key>
# export SILO_MINIO_SECRET_KEY=<your-secret-key>
# docker compose -f docker-compose.prod.yaml up -d
services:
@@ -15,23 +13,15 @@ services:
container_name: silod
restart: unless-stopped
environment:
# Database connection (psql.kindred.internal)
SILO_DB_HOST: psql.kindred.internal
SILO_DB_PORT: 5432
# Database connection (psql.example.internal)
# Supported as direct env var overrides in the Go config loader:
SILO_DB_HOST: psql.example.internal
SILO_DB_NAME: silo
SILO_DB_USER: silo
SILO_DB_PASSWORD: ${SILO_DB_PASSWORD:?Database password required}
SILO_DB_SSLMODE: require
# MinIO storage (minio.kindred.internal)
SILO_MINIO_ENDPOINT: minio.kindred.internal:9000
SILO_MINIO_ACCESS_KEY: ${SILO_MINIO_ACCESS_KEY:?MinIO access key required}
SILO_MINIO_SECRET_KEY: ${SILO_MINIO_SECRET_KEY:?MinIO secret key required}
SILO_MINIO_BUCKET: silo-files
SILO_MINIO_USE_SSL: "true"
# Server settings
SILO_SERVER_BASE_URL: ${SILO_BASE_URL:-http://silo.kindred.internal:8080}
# Note: SILO_DB_PORT and SILO_DB_SSLMODE are NOT supported as direct
# env var overrides. Set these in config.yaml instead, or use ${VAR}
# syntax in the YAML file. See docs/CONFIGURATION.md for details.
ports:
- "8080:8080"
volumes:

View File

@@ -19,26 +19,6 @@ services:
networks:
- silo-network
minio:
image: minio/minio:RELEASE.2023-05-04T21-44-30Z
container_name: silo-minio
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: ${MINIO_ACCESS_KEY:-silominio}
MINIO_ROOT_PASSWORD: ${MINIO_SECRET_KEY:-silominiosecret}
volumes:
- minio_data:/data
ports:
- "9000:9000"
- "9001:9001"
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 10s
timeout: 5s
retries: 5
networks:
- silo-network
silo:
build:
context: ..
@@ -47,19 +27,12 @@ services:
depends_on:
postgres:
condition: service_healthy
minio:
condition: service_healthy
environment:
SILO_DB_HOST: postgres
SILO_DB_PORT: 5432
SILO_DB_NAME: silo
SILO_DB_USER: silo
SILO_DB_PASSWORD: ${POSTGRES_PASSWORD:-silodev}
SILO_MINIO_ENDPOINT: minio:9000
SILO_MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY:-silominio}
SILO_MINIO_SECRET_KEY: ${MINIO_SECRET_KEY:-silominiosecret}
SILO_MINIO_BUCKET: silo-files
SILO_MINIO_USE_SSL: "false"
SILO_SESSION_SECRET: ${SILO_SESSION_SECRET:-change-me-in-production}
SILO_OIDC_CLIENT_SECRET: ${SILO_OIDC_CLIENT_SECRET:-}
SILO_LDAP_BIND_PASSWORD: ${SILO_LDAP_BIND_PASSWORD:-}
@@ -68,8 +41,9 @@ services:
ports:
- "8080:8080"
volumes:
- silo_data:/var/lib/silo/data
- ../schemas:/etc/silo/schemas:ro
- ../configs/config.yaml:/etc/silo/config.yaml:ro
- ./config.dev.yaml:/etc/silo/config.yaml:ro
healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:8080/health"]
interval: 10s
@@ -80,7 +54,7 @@ services:
volumes:
postgres_data:
minio_data:
silo_data:
networks:
silo-network:

View File

@@ -0,0 +1,36 @@
# Enable the memberOf overlay for OpenLDAP.
# When a user is added to a groupOfNames, their entry automatically
# gets a memberOf attribute pointing to the group DN.
# This is required for Silo's LDAP role mapping.
#
# Loaded automatically by bitnami/openldap from /docker-entrypoint-initdb.d/
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: memberof
dn: olcOverlay=memberof,olcDatabase={2}mdb,cn=config
changetype: add
objectClass: olcOverlayConfig
objectClass: olcMemberOf
olcOverlay: memberof
olcMemberOfRefInt: TRUE
olcMemberOfDangling: ignore
olcMemberOfGroupOC: groupOfNames
olcMemberOfMemberAD: member
olcMemberOfMemberOfAD: memberOf
# Enable refint overlay to maintain referential integrity
# (removes memberOf when a user is removed from a group)
dn: cn=module{0},cn=config
changetype: modify
add: olcModuleLoad
olcModuleLoad: refint
dn: olcOverlay=refint,olcDatabase={2}mdb,cn=config
changetype: add
objectClass: olcOverlayConfig
objectClass: olcRefintConfig
olcOverlay: refint
olcRefintAttribute: memberOf member

View File

@@ -0,0 +1,34 @@
# Create Silo role groups for LDAP-based access control.
# These groups map to Silo roles via auth.ldap.role_mapping in config.yaml.
#
# Group hierarchy:
# silo-admins -> admin role (full access)
# silo-users -> editor role (create/modify items)
# silo-viewers -> viewer role (read-only)
#
# The initial LDAP user (set via LDAP_USERS env var) is added to silo-admins.
# Additional users can be added with ldapadd or ldapmodify.
#
# Loaded automatically by bitnami/openldap from /docker-entrypoint-initdb.d/
# Note: This runs after the default tree is created (users/groups OUs exist).
# Admin group — initial user is a member
dn: cn=silo-admins,ou=groups,dc=silo,dc=local
objectClass: groupOfNames
cn: silo-admins
description: Silo administrators (full access)
member: cn=siloadmin,ou=users,dc=silo,dc=local
# Editor group
dn: cn=silo-users,ou=groups,dc=silo,dc=local
objectClass: groupOfNames
cn: silo-users
description: Silo editors (create and modify items)
member: cn=placeholder,ou=users,dc=silo,dc=local
# Viewer group
dn: cn=silo-viewers,ou=groups,dc=silo,dc=local
objectClass: groupOfNames
cn: silo-viewers
description: Silo viewers (read-only access)
member: cn=placeholder,ou=users,dc=silo,dc=local

View File

@@ -0,0 +1,44 @@
# Silo Nginx Reverse Proxy — HTTP only (no TLS)
#
# Use this when TLS is terminated by an external load balancer or when
# running on a trusted internal network without HTTPS.
upstream silo_backend {
server silo:8080;
}
server {
listen 80;
listen [::]:80;
server_name _;
location / {
proxy_pass http://silo_backend;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# SSE support
proxy_set_header Connection "";
proxy_buffering off;
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 300s;
# File uploads (CAD files can be large)
client_max_body_size 100M;
}
location /nginx-health {
access_log off;
return 200 "OK\n";
add_header Content-Type text/plain;
}
}

View File

@@ -0,0 +1,103 @@
# Silo Nginx Reverse Proxy (Docker)
#
# HTTP reverse proxy with optional HTTPS. To enable TLS:
# 1. Uncomment the ssl server block below
# 2. Mount your certificate and key in docker-compose:
# volumes:
# - /path/to/cert.pem:/etc/nginx/ssl/cert.pem:ro
# - /path/to/key.pem:/etc/nginx/ssl/key.pem:ro
# 3. Uncomment the HTTP-to-HTTPS redirect in the port 80 block
upstream silo_backend {
server silo:8080;
}
# HTTP server
server {
listen 80;
listen [::]:80;
server_name _;
# Uncomment the next line to redirect all HTTP traffic to HTTPS
# return 301 https://$host$request_uri;
location / {
proxy_pass http://silo_backend;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
# SSE support
proxy_set_header Connection "";
proxy_buffering off;
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 300s;
# File uploads (CAD files can be large)
client_max_body_size 100M;
}
# Health check endpoint for monitoring
location /nginx-health {
access_log off;
return 200 "OK\n";
add_header Content-Type text/plain;
}
}
# Uncomment for HTTPS (mount certs in docker-compose volumes)
# server {
# listen 443 ssl http2;
# listen [::]:443 ssl http2;
# server_name _;
#
# ssl_certificate /etc/nginx/ssl/cert.pem;
# ssl_certificate_key /etc/nginx/ssl/key.pem;
#
# ssl_protocols TLSv1.2 TLSv1.3;
# ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305;
# ssl_prefer_server_ciphers off;
# ssl_session_timeout 1d;
# ssl_session_cache shared:SSL:10m;
# ssl_session_tickets off;
#
# # Security headers
# add_header X-Frame-Options "SAMEORIGIN" always;
# add_header X-Content-Type-Options "nosniff" always;
# add_header Referrer-Policy "strict-origin-when-cross-origin" always;
#
# location / {
# proxy_pass http://silo_backend;
# proxy_http_version 1.1;
#
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto $scheme;
# proxy_set_header X-Forwarded-Host $host;
# proxy_set_header X-Forwarded-Port $server_port;
#
# proxy_set_header Connection "";
# proxy_buffering off;
#
# proxy_connect_timeout 60s;
# proxy_send_timeout 60s;
# proxy_read_timeout 300s;
#
# client_max_body_size 100M;
# }
#
# location /nginx-health {
# access_log off;
# return 200 "OK\n";
# add_header Content-Type text/plain;
# }
# }

View File

@@ -2,14 +2,10 @@
# Copy to /etc/silo/silod.env and fill in values
# Permissions: chmod 600 /etc/silo/silod.env
# Database credentials (psql.kindred.internal)
# Database credentials (psql.example.internal)
# Database: silo, User: silo
SILO_DB_PASSWORD=
# MinIO credentials (minio.kindred.internal)
# User: silouser
SILO_MINIO_ACCESS_KEY=silouser
SILO_MINIO_SECRET_KEY=
# Authentication
# Session secret (required when auth is enabled)
@@ -23,4 +19,4 @@ SILO_ADMIN_PASSWORD=
# SILO_LDAP_BIND_PASSWORD=
# Optional: Override server base URL
# SILO_SERVER_BASE_URL=http://silo.kindred.internal:8080
# SILO_SERVER_BASE_URL=http://silo.example.internal:8080

View File

@@ -27,6 +27,7 @@ NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadWritePaths=/opt/silo/data
ReadOnlyPaths=/etc/silo /opt/silo
# Resource limits

View File

@@ -38,7 +38,7 @@ API tokens allow the FreeCAD plugin, scripts, and CI pipelines to authenticate w
### Creating a Token (CLI)
```sh
export SILO_API_URL=https://silo.kindred.internal
export SILO_API_URL=https://silo.example.internal
export SILO_API_TOKEN=silo_<your-existing-token>
silo token create --name "CI pipeline"
@@ -140,7 +140,7 @@ auth:
ldap:
enabled: true
url: "ldaps://ipa.kindred.internal"
url: "ldaps://ipa.example.internal"
base_dn: "dc=kindred,dc=internal"
user_search_dn: "cn=users,cn=accounts,dc=kindred,dc=internal"
user_attr: "uid"
@@ -170,10 +170,10 @@ auth:
oidc:
enabled: true
issuer_url: "https://keycloak.kindred.internal/realms/silo"
issuer_url: "https://keycloak.example.internal/realms/silo"
client_id: "silo"
client_secret: "" # Set via SILO_OIDC_CLIENT_SECRET
redirect_url: "https://silo.kindred.internal/auth/callback"
redirect_url: "https://silo.example.internal/auth/callback"
scopes: ["openid", "profile", "email"]
admin_role: "silo-admin"
editor_role: "silo-editor"
@@ -186,7 +186,7 @@ auth:
auth:
cors:
allowed_origins:
- "https://silo.kindred.internal"
- "https://silo.example.internal"
```
## Environment Variables
@@ -254,4 +254,4 @@ UPDATE users SET password_hash = '<bcrypt-hash>', is_active = true WHERE usernam
- Verify the token is set in FreeCAD preferences or `SILO_API_TOKEN`
- Check the API URL points to the correct server
- Test with curl: `curl -H "Authorization: Bearer silo_..." https://silo.kindred.internal/api/items`
- Test with curl: `curl -H "Authorization: Bearer silo_..." https://silo.example.internal/api/items`

View File

@@ -73,25 +73,27 @@ database:
---
## Storage (MinIO/S3)
## Storage (Filesystem)
| Key | Type | Default | Env Override | Description |
|-----|------|---------|-------------|-------------|
| `storage.endpoint` | string | — | `SILO_MINIO_ENDPOINT` | MinIO/S3 endpoint (`host:port`) |
| `storage.access_key` | string | — | `SILO_MINIO_ACCESS_KEY` | Access key |
| `storage.secret_key` | string | — | `SILO_MINIO_SECRET_KEY` | Secret key |
| `storage.bucket` | string | — | — | S3 bucket name (created automatically if missing) |
| `storage.use_ssl` | bool | `false` | — | Use HTTPS for MinIO connections |
| `storage.region` | string | `"us-east-1"` | — | S3 region |
Files are stored on the local filesystem under a configurable root directory.
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `storage.backend` | string | `"filesystem"` | Storage backend (`filesystem`) |
| `storage.filesystem.root_dir` | string | — | Root directory for file storage (required) |
```yaml
storage:
endpoint: "localhost:9000"
access_key: "" # use SILO_MINIO_ACCESS_KEY env var
secret_key: "" # use SILO_MINIO_SECRET_KEY env var
bucket: "silo-files"
use_ssl: false
region: "us-east-1"
backend: "filesystem"
filesystem:
root_dir: "/opt/silo/data"
```
Ensure the directory exists and is writable by the `silo` user:
```bash
sudo mkdir -p /opt/silo/data
sudo chown silo:silo /opt/silo/data
```
---
@@ -264,9 +266,6 @@ All environment variable overrides. These take precedence over values in `config
| `SILO_DB_NAME` | `database.name` | PostgreSQL database name |
| `SILO_DB_USER` | `database.user` | PostgreSQL user |
| `SILO_DB_PASSWORD` | `database.password` | PostgreSQL password |
| `SILO_MINIO_ENDPOINT` | `storage.endpoint` | MinIO endpoint |
| `SILO_MINIO_ACCESS_KEY` | `storage.access_key` | MinIO access key |
| `SILO_MINIO_SECRET_KEY` | `storage.secret_key` | MinIO secret key |
| `SILO_SESSION_SECRET` | `auth.session_secret` | Session cookie signing secret |
| `SILO_ADMIN_USERNAME` | `auth.local.default_admin_username` | Default admin username |
| `SILO_ADMIN_PASSWORD` | `auth.local.default_admin_password` | Default admin password |
@@ -296,11 +295,9 @@ database:
sslmode: "disable"
storage:
endpoint: "localhost:9000"
access_key: "minioadmin"
secret_key: "minioadmin"
bucket: "silo-files"
use_ssl: false
backend: "filesystem"
filesystem:
root_dir: "./data"
schemas:
directory: "./schemas"

246
docs/DAG.md Normal file
View File

@@ -0,0 +1,246 @@
# Dependency DAG Specification
**Status:** Draft
**Last Updated:** 2026-02-13
---
## 1. Purpose
The Dependency DAG is a server-side graph that tracks how features, constraints, and assembly relationships depend on each other. It enables three capabilities described in [MULTI_USER_EDITS.md](MULTI_USER_EDITS.md):
1. **Interference detection** -- comparing dependency cones of concurrent edit sessions to classify conflicts as none, soft, or hard before the user encounters them.
2. **Incremental validation** -- marking changed nodes dirty and propagating only through the affected subgraph, using input-hash memoization to stop early when inputs haven't changed.
3. **Structured merge safety** -- walking the DAG to determine whether concurrent edits share upstream dependencies, deciding if auto-merge is safe or manual review is required.
---
## 2. Two-Tier Model
Silo maintains two levels of dependency graph:
### 2.1 BOM DAG (existing)
The assembly-to-part relationship graph already stored in the `relationships` table. Each row represents a parent item containing a child item with a quantity and relationship type (`component`, `alternate`, `reference`). This graph is queried via `GetBOM`, `GetExpandedBOM`, `GetWhereUsed`, and `HasCycle` in `internal/db/relationships.go`.
The BOM DAG is **not modified** by this specification. It continues to serve its existing purpose.
### 2.2 Feature DAG (new)
A finer-grained graph stored in `dag_nodes` and `dag_edges` tables. Each node represents a feature within a single item's revision -- a sketch, pad, fillet, pocket, constraint, body, or part-level container. Edges represent "depends on" relationships: if Pad003 depends on Sketch001, an edge runs from Sketch001 to Pad003.
The feature DAG is populated by clients (silo-mod) when users save, or by runners after compute jobs. Silo stores and queries it but does not generate it -- the Create client has access to the feature tree and is the authoritative source.
### 2.3 Cross-Item Edges
Assembly constraints often reference geometry on child parts (e.g., "mate Face6 of PartA to Face2 of PartB"). These cross-item dependencies are stored in `dag_cross_edges`, linking a node in one item to a node in another. Each cross-edge optionally references the `relationships` row that establishes the BOM connection.
---
## 3. Data Model
### 3.1 dag_nodes
| Column | Type | Description |
|--------|------|-------------|
| `id` | UUID | Primary key |
| `item_id` | UUID | FK to `items.id` |
| `revision_number` | INTEGER | Revision this DAG snapshot belongs to |
| `node_key` | TEXT | Feature name from Create (e.g., `Sketch001`, `Pad003`, `Body`) |
| `node_type` | TEXT | One of: `sketch`, `pad`, `pocket`, `fillet`, `chamfer`, `constraint`, `body`, `part`, `datum`, `mirror`, `pattern`, `boolean` |
| `properties_hash` | TEXT | SHA-256 of the node's parametric inputs (sketch coordinates, fillet radius, constraint values). Used for memoization -- if the hash hasn't changed, validation can skip this node. |
| `validation_state` | TEXT | One of: `clean`, `dirty`, `validating`, `failed` |
| `validation_msg` | TEXT | Error message when `validation_state = 'failed'` |
| `metadata` | JSONB | Type-specific data (sketch coords, feature params, constraint definitions) |
| `created_at` | TIMESTAMPTZ | Row creation time |
| `updated_at` | TIMESTAMPTZ | Last state change |
**Uniqueness:** `(item_id, revision_number, node_key)` -- one node per feature per revision.
### 3.2 dag_edges
| Column | Type | Description |
|--------|------|-------------|
| `id` | UUID | Primary key |
| `source_node_id` | UUID | FK to `dag_nodes.id` -- the upstream node |
| `target_node_id` | UUID | FK to `dag_nodes.id` -- the downstream node that depends on source |
| `edge_type` | TEXT | `depends_on` (default), `references`, `constrains` |
| `metadata` | JSONB | Optional edge metadata |
**Direction convention:** An edge from A to B means "B depends on A". A is upstream, B is downstream. Forward-cone traversal from A walks edges where A is the source.
**Uniqueness:** `(source_node_id, target_node_id, edge_type)`.
**Constraint:** `source_node_id != target_node_id` (no self-edges).
### 3.3 dag_cross_edges
| Column | Type | Description |
|--------|------|-------------|
| `id` | UUID | Primary key |
| `source_node_id` | UUID | FK to `dag_nodes.id` -- node in item A |
| `target_node_id` | UUID | FK to `dag_nodes.id` -- node in item B |
| `relationship_id` | UUID | FK to `relationships.id` (nullable) -- the BOM entry connecting the two items |
| `edge_type` | TEXT | `assembly_ref` (default) |
| `metadata` | JSONB | Reference details (face ID, edge ID, etc.) |
**Uniqueness:** `(source_node_id, target_node_id)`.
---
## 4. Validation States
Each node has a `validation_state` that tracks whether its computed geometry is current:
| State | Meaning |
|-------|---------|
| `clean` | Node's geometry matches its `properties_hash`. No recompute needed. |
| `dirty` | An upstream change has propagated to this node. Recompute required. |
| `validating` | A compute job is currently revalidating this node. |
| `failed` | Recompute failed. `validation_msg` contains the error. |
### 4.1 State Transitions
```
clean → dirty (upstream change detected, or MarkDirty called)
dirty → validating (compute job claims this node)
validating → clean (recompute succeeded, properties_hash updated)
validating → failed (recompute produced an error)
failed → dirty (upstream change detected, retry possible)
dirty → clean (properties_hash matches previous -- memoization shortcut)
```
### 4.2 Dirty Propagation
When a node is marked dirty, all downstream nodes in its forward cone are also marked dirty. This is done atomically in a single recursive CTE:
```sql
WITH RECURSIVE forward_cone AS (
SELECT $1::uuid AS node_id
UNION
SELECT e.target_node_id
FROM dag_edges e
JOIN forward_cone fc ON fc.node_id = e.source_node_id
)
UPDATE dag_nodes SET validation_state = 'dirty', updated_at = now()
WHERE id IN (SELECT node_id FROM forward_cone)
AND validation_state = 'clean';
```
### 4.3 Memoization
Before marking a node dirty, the system can compare the new `properties_hash` against the stored value. If they match, the change did not affect this node's inputs, and propagation stops. This is the memoization boundary described in MULTI_USER_EDITS.md Section 5.2.
---
## 5. Graph Queries
### 5.1 Forward Cone
Returns all nodes downstream of a given node -- everything that would be affected if the source node changes. Used for interference detection: if two users' forward cones overlap, there is potential interference.
```sql
WITH RECURSIVE forward_cone AS (
SELECT target_node_id AS node_id
FROM dag_edges WHERE source_node_id = $1
UNION
SELECT e.target_node_id
FROM dag_edges e
JOIN forward_cone fc ON fc.node_id = e.source_node_id
)
SELECT n.* FROM dag_nodes n JOIN forward_cone fc ON n.id = fc.node_id;
```
### 5.2 Backward Cone
Returns all nodes upstream of a given node -- everything the target node depends on.
### 5.3 Dirty Subgraph
Returns all nodes for a given item where `validation_state != 'clean'`, along with their edges. This is the input to an incremental validation job.
### 5.4 Cycle Detection
Before adding an edge, check that it would not create a cycle. Uses the same recursive ancestor-walk pattern as `HasCycle` in `internal/db/relationships.go`.
---
## 6. DAG Sync
Clients push the full feature DAG to Silo via `PUT /api/items/{partNumber}/dag`. The sync payload is a JSON document:
```json
{
"revision": 3,
"nodes": [
{
"key": "Sketch001",
"type": "sketch",
"properties_hash": "a1b2c3...",
"metadata": {
"coordinates": [[0, 0], [10, 0], [10, 5]],
"constraints": ["horizontal", "vertical"]
}
},
{
"key": "Pad003",
"type": "pad",
"properties_hash": "d4e5f6...",
"metadata": {
"length": 15.0,
"direction": [0, 0, 1]
}
}
],
"edges": [
{
"source": "Sketch001",
"target": "Pad003",
"type": "depends_on"
}
]
}
```
The server processes this within a single transaction:
1. Upsert all nodes (matched by `item_id + revision_number + node_key`).
2. Replace all edges for this item/revision.
3. Compare new `properties_hash` values against stored values to detect changes.
4. Mark changed nodes and their forward cones dirty.
5. Publish `dag.updated` SSE event.
---
## 7. Interference Detection
When a user registers an edit context (MULTI_USER_EDITS.md Section 3.1), the server:
1. Looks up the node(s) being edited by `node_key` within the item's current revision.
2. Computes the forward cone for those nodes.
3. Compares the cone against all active edit sessions' cones.
4. Classifies interference:
- **No overlap** → no interference, fully concurrent.
- **Overlap, different objects** → soft interference, visual indicator via SSE.
- **Same object, same edit type** → hard interference, edit blocked.
---
## 8. REST API
All endpoints are under `/api/items/{partNumber}` and require authentication.
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/dag` | viewer | Get full feature DAG for current revision |
| `GET` | `/dag/forward-cone/{nodeKey}` | viewer | Get forward dependency cone |
| `GET` | `/dag/dirty` | viewer | Get dirty subgraph |
| `PUT` | `/dag` | editor | Sync full feature tree (from client or runner) |
| `POST` | `/dag/mark-dirty/{nodeKey}` | editor | Manually mark a node and its cone dirty |
---
## 9. References
- [MULTI_USER_EDITS.md](MULTI_USER_EDITS.md) -- Full multi-user editing specification
- [WORKERS.md](WORKERS.md) -- Worker/runner system that executes validation jobs
- [ROADMAP.md](ROADMAP.md) -- Tier 0 Dependency DAG entry

View File

@@ -0,0 +1,395 @@
# DAG Client Integration Contract
**Status:** Draft
**Last Updated:** 2026-02-13
This document describes what silo-mod and Headless Create runners need to implement to integrate with the Silo dependency DAG and worker system.
---
## 1. Overview
The DAG system has two client-side integration points:
1. **silo-mod workbench** (desktop) -- pushes DAG data to Silo on save or revision create.
2. **silorunner + silo-mod** (headless) -- extracts DAGs, validates features, and exports geometry as compute jobs.
Both share the same Python codebase in the silo-mod repository. Desktop users call the code interactively; runners call it headlessly via `create --console`.
---
## 2. DAG Sync Payload
Clients push feature trees to Silo via:
```
PUT /api/items/{partNumber}/dag
Authorization: Bearer <user_token or runner_token>
Content-Type: application/json
```
### 2.1 Request Body
```json
{
"revision_number": 3,
"nodes": [
{
"node_key": "Sketch001",
"node_type": "sketch",
"properties_hash": "a1b2c3d4e5f6...",
"metadata": {
"label": "Base Profile",
"constraint_count": 12
}
},
{
"node_key": "Pad001",
"node_type": "pad",
"properties_hash": "f6e5d4c3b2a1...",
"metadata": {
"label": "Main Extrusion",
"length": 25.0
}
}
],
"edges": [
{
"source_key": "Sketch001",
"target_key": "Pad001",
"edge_type": "depends_on"
}
]
}
```
### 2.2 Field Reference
**Nodes:**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `node_key` | string | yes | Unique within item+revision. Use Create's internal object name (e.g. `Sketch001`, `Pad003`). |
| `node_type` | string | yes | One of: `sketch`, `pad`, `pocket`, `fillet`, `chamfer`, `constraint`, `body`, `part`, `datum`. |
| `properties_hash` | string | no | SHA-256 hex digest of the node's parametric inputs. Used for memoization. |
| `validation_state` | string | no | One of: `clean`, `dirty`, `validating`, `failed`. Defaults to `clean`. |
| `metadata` | object | no | Arbitrary key-value pairs for display or debugging. |
**Edges:**
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `source_key` | string | yes | The node that is depended upon. |
| `target_key` | string | yes | The node that depends on the source. |
| `edge_type` | string | no | One of: `depends_on` (default), `references`, `constrains`. |
**Direction convention:** Edges point from dependency to dependent. If Pad001 depends on Sketch001, the edge is `source_key: "Sketch001"`, `target_key: "Pad001"`.
### 2.3 Response
```json
{
"synced": true,
"node_count": 15,
"edge_count": 14
}
```
---
## 3. Computing properties_hash
The `properties_hash` enables memoization -- if a node's inputs haven't changed since the last validation, it can be skipped. Computing it:
```python
import hashlib
import json
def compute_properties_hash(feature_obj):
"""Hash the parametric inputs of a Create feature."""
inputs = {}
if feature_obj.TypeId == "Sketcher::SketchObject":
# Hash geometry + constraints
inputs["geometry_count"] = feature_obj.GeometryCount
inputs["constraint_count"] = feature_obj.ConstraintCount
inputs["geometry"] = str(feature_obj.Shape.exportBrep())
elif feature_obj.TypeId == "PartDesign::Pad":
inputs["length"] = feature_obj.Length.Value
inputs["type"] = str(feature_obj.Type)
inputs["reversed"] = feature_obj.Reversed
inputs["sketch"] = feature_obj.Profile[0].Name
# ... other feature types
canonical = json.dumps(inputs, sort_keys=True)
return hashlib.sha256(canonical.encode()).hexdigest()
```
The exact inputs per feature type are determined by what parametric values affect the feature's geometry. Include anything that, if changed, would require recomputation.
---
## 4. Feature Tree Walking
To extract the DAG from a Create document:
```python
import FreeCAD
def extract_dag(doc):
"""Walk a Create document and return nodes + edges."""
nodes = []
edges = []
for obj in doc.Objects:
# Skip non-feature objects
if not hasattr(obj, "TypeId"):
continue
node_type = classify_type(obj.TypeId)
if node_type is None:
continue
nodes.append({
"node_key": obj.Name,
"node_type": node_type,
"properties_hash": compute_properties_hash(obj),
"metadata": {
"label": obj.Label,
"type_id": obj.TypeId,
}
})
# Walk dependencies via InList (objects this one depends on)
for dep in obj.InList:
if hasattr(dep, "TypeId") and classify_type(dep.TypeId):
edges.append({
"source_key": dep.Name,
"target_key": obj.Name,
"edge_type": "depends_on",
})
return nodes, edges
def classify_type(type_id):
"""Map Create TypeIds to DAG node types."""
mapping = {
"Sketcher::SketchObject": "sketch",
"PartDesign::Pad": "pad",
"PartDesign::Pocket": "pocket",
"PartDesign::Fillet": "fillet",
"PartDesign::Chamfer": "chamfer",
"PartDesign::Body": "body",
"Part::Feature": "part",
"Sketcher::SketchConstraint": "constraint",
}
return mapping.get(type_id)
```
---
## 5. When to Push DAG Data
Push the DAG to Silo in these scenarios:
| Event | Trigger | Who |
|-------|---------|-----|
| User saves in silo-mod | On save callback | Desktop silo-mod workbench |
| User creates a revision | After `POST /api/items/{pn}/revisions` succeeds | Desktop silo-mod workbench |
| Runner extracts DAG | After `create-dag-extract` job completes | silorunner via `PUT /api/runner/jobs/{id}/dag` |
| Runner validates | After `create-validate` job, push updated validation states | silorunner via `PUT /api/runner/jobs/{id}/dag` |
---
## 6. Runner Entry Points
silo-mod must provide these Python entry points for headless invocation:
### 6.1 silo.runner.dag_extract
Extracts the feature DAG from a Create file and writes it as JSON.
```python
# silo/runner.py
def dag_extract(input_path, output_path):
"""
Extract feature DAG from a Create file.
Args:
input_path: Path to the .kc (Kindred Create) file.
output_path: Path to write the JSON output.
Output JSON format:
{
"nodes": [...], // Same format as DAG sync payload
"edges": [...]
}
"""
doc = FreeCAD.openDocument(input_path)
nodes, edges = extract_dag(doc)
with open(output_path, 'w') as f:
json.dump({"nodes": nodes, "edges": edges}, f)
FreeCAD.closeDocument(doc.Name)
```
### 6.2 silo.runner.validate
Rebuilds all features and reports pass/fail per node.
```python
def validate(input_path, output_path):
"""
Validate a Create file by rebuilding all features.
Output JSON format:
{
"valid": true/false,
"nodes": [
{
"node_key": "Pad001",
"state": "clean", // or "failed"
"message": null, // error message if failed
"properties_hash": "..."
}
]
}
"""
doc = FreeCAD.openDocument(input_path)
doc.recompute()
results = []
all_valid = True
for obj in doc.Objects:
if not hasattr(obj, "TypeId"):
continue
node_type = classify_type(obj.TypeId)
if node_type is None:
continue
state = "clean"
message = None
if hasattr(obj, "isValid") and not obj.isValid():
state = "failed"
message = f"Feature {obj.Label} failed to recompute"
all_valid = False
results.append({
"node_key": obj.Name,
"state": state,
"message": message,
"properties_hash": compute_properties_hash(obj),
})
with open(output_path, 'w') as f:
json.dump({"valid": all_valid, "nodes": results}, f)
FreeCAD.closeDocument(doc.Name)
```
### 6.3 silo.runner.export
Exports geometry to STEP, IGES, or other formats.
```python
def export(input_path, output_path, format="step"):
"""
Export a Create file to an external format.
Args:
input_path: Path to the .kc file.
output_path: Path to write the exported file.
format: Export format ("step", "iges", "stl", "obj").
"""
doc = FreeCAD.openDocument(input_path)
import Part
shapes = [obj.Shape for obj in doc.Objects if hasattr(obj, "Shape")]
compound = Part.makeCompound(shapes)
format_map = {
"step": "STEP",
"iges": "IGES",
"stl": "STL",
"obj": "OBJ",
}
Part.export([compound], output_path)
FreeCAD.closeDocument(doc.Name)
```
---
## 7. Headless Invocation
The `silorunner` binary shells out to Create (with silo-mod installed):
```bash
# DAG extraction
create --console -e "from silo.runner import dag_extract; dag_extract('/tmp/job/part.kc', '/tmp/job/dag.json')"
# Validation
create --console -e "from silo.runner import validate; validate('/tmp/job/part.kc', '/tmp/job/result.json')"
# Export
create --console -e "from silo.runner import export; export('/tmp/job/part.kc', '/tmp/job/output.step', 'step')"
```
**Prerequisites:** The runner host must have:
- Headless Create installed (Kindred's fork of FreeCAD)
- silo-mod installed as a Create addon (so `from silo.runner import ...` works)
- No display server required -- `--console` mode is headless
---
## 8. Validation Result Handling
After a runner completes a `create-validate` job, it should:
1. Read the result JSON.
2. Push updated validation states via `PUT /api/runner/jobs/{jobID}/dag`:
```json
{
"revision_number": 3,
"nodes": [
{"node_key": "Sketch001", "node_type": "sketch", "validation_state": "clean", "properties_hash": "abc..."},
{"node_key": "Pad001", "node_type": "pad", "validation_state": "failed", "properties_hash": "def..."}
],
"edges": [
{"source_key": "Sketch001", "target_key": "Pad001"}
]
}
```
3. Complete the job via `POST /api/runner/jobs/{jobID}/complete` with the summary result.
---
## 9. SSE Events
Clients should listen for these events on `GET /api/events`:
| Event | Payload | When |
|-------|---------|------|
| `dag.updated` | `{item_id, part_number, revision_number, node_count, edge_count}` | After any DAG sync |
| `dag.validated` | `{item_id, part_number, valid, failed_count}` | After validation completes |
| `job.created` | `{job_id, definition_name, trigger, item_id}` | Job auto-triggered or manually created |
| `job.claimed` | `{job_id, runner_id, runner}` | Runner claims a job |
| `job.progress` | `{job_id, progress, message}` | Runner reports progress |
| `job.completed` | `{job_id, runner_id}` | Job finishes successfully |
| `job.failed` | `{job_id, runner_id, error}` | Job fails |
| `job.cancelled` | `{job_id, cancelled_by}` | Job cancelled by user |
---
## 10. Cross-Item Edges
For assembly constraints that reference geometry in child parts (e.g. a mate constraint between two parts), use the `dag_cross_edges` table. These edges bridge the BOM DAG and the feature DAG.
Cross-item edges are **not** included in the standard `PUT /dag` sync. They will be managed through a dedicated endpoint in a future iteration once the assembly constraint model in Create/silo-mod is finalized.
For now, the DAG sync covers intra-item dependencies only. Assembly-level interference detection uses the BOM DAG (`relationships` table) combined with per-item feature DAGs.

View File

@@ -1,6 +1,10 @@
# Silo Production Deployment Guide
This guide covers deploying Silo to a dedicated VM using external PostgreSQL and MinIO services.
> **First-time setup?** See the [Installation Guide](INSTALL.md) for step-by-step
> instructions. This document covers ongoing maintenance and operations for an
> existing deployment.
This guide covers deploying Silo to a dedicated VM using external PostgreSQL and local filesystem storage.
## Table of Contents
@@ -17,33 +21,30 @@ This guide covers deploying Silo to a dedicated VM using external PostgreSQL and
```
┌─────────────────────────────────────────────────────────────────┐
│ silo.kindred.internal │
│ silo.example.internal │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ silod │ │
│ │ (Silo API Server) │ │
│ │ :8080 │ │
│ │ Files: /opt/silo/data │ │
│ └───────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────┐ ┌─────────────────────────────────┐
│ psql.kindred.internal │ │ minio.kindred.internal
│ PostgreSQL 16 │ │ MinIO S3 │
│ :5432 │ │ :9000 (API) │
│ │ │ :9001 (Console) │
└─────────────────────────┘ └─────────────────────────────────┘
┌─────────────────────────┐
│ psql.example.internal │
│ PostgreSQL 16 │
│ :5432 │
└─────────────────────────┘
```
## External Services
The following external services are already configured:
| Service | Host | Database | User |
|---------|------|----------|------|
| PostgreSQL | psql.example.internal:5432 | silo | silo |
| Service | Host | Database/Bucket | User |
|---------|------|-----------------|------|
| PostgreSQL | psql.kindred.internal:5432 | silo | silo |
| MinIO | minio.kindred.internal:9000 | silo-files | silouser |
Migrations have been applied to the database.
Files are stored on the local filesystem at `/opt/silo/data`. Migrations have been applied to the database.
---
@@ -53,10 +54,10 @@ For a fresh VM, run these commands:
```bash
# 1. SSH to the target host
ssh root@silo.kindred.internal
ssh root@silo.example.internal
# 2. Download and run setup script
curl -fsSL https://gitea.kindred.internal/kindred/silo-0062/raw/branch/main/scripts/setup-host.sh | bash
curl -fsSL https://git.kindred-systems.com/kindred/silo/raw/branch/main/scripts/setup-host.sh | bash
# 3. Configure credentials
nano /etc/silo/silod.env
@@ -69,16 +70,16 @@ nano /etc/silo/silod.env
## Initial Setup
Run the setup script once on `silo.kindred.internal` to prepare the host:
Run the setup script once on `silo.example.internal` to prepare the host:
```bash
# Option 1: If you have the repo locally
scp scripts/setup-host.sh root@silo.kindred.internal:/tmp/
ssh root@silo.kindred.internal 'bash /tmp/setup-host.sh'
scp scripts/setup-host.sh root@silo.example.internal:/tmp/
ssh root@silo.example.internal 'bash /tmp/setup-host.sh'
# Option 2: Direct on the host
ssh root@silo.kindred.internal
curl -fsSL https://git.kindred.internal/kindred/silo/raw/branch/main/scripts/setup-host.sh -o /tmp/setup-host.sh
ssh root@silo.example.internal
curl -fsSL https://git.kindred-systems.com/kindred/silo/raw/branch/main/scripts/setup-host.sh -o /tmp/setup-host.sh
bash /tmp/setup-host.sh
```
@@ -100,24 +101,18 @@ sudo nano /etc/silo/silod.env
Fill in the values:
```bash
# Database credentials (psql.kindred.internal)
# Database credentials (psql.example.internal)
SILO_DB_PASSWORD=your-database-password
# MinIO credentials (minio.kindred.internal)
SILO_MINIO_ACCESS_KEY=silouser
SILO_MINIO_SECRET_KEY=your-minio-secret-key
```
### Verify External Services
Before deploying, verify connectivity to external services:
Before deploying, verify connectivity to PostgreSQL:
```bash
# Test PostgreSQL
psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1'
# Test MinIO
curl -I http://minio.kindred.internal:9000/minio/health/live
psql -h psql.example.internal -U silo -d silo -c 'SELECT 1'
```
---
@@ -129,7 +124,7 @@ curl -I http://minio.kindred.internal:9000/minio/health/live
To deploy or update Silo, run the deploy script on the target host:
```bash
ssh root@silo.kindred.internal
ssh root@silo.example.internal
/opt/silo/src/scripts/deploy.sh
```
@@ -165,7 +160,7 @@ sudo /opt/silo/src/scripts/deploy.sh --status
You can override the git repository URL and branch:
```bash
export SILO_REPO_URL=https://git.kindred.internal/kindred/silo.git
export SILO_REPO_URL=https://git.kindred-systems.com/kindred/silo.git
export SILO_BRANCH=main
sudo -E /opt/silo/src/scripts/deploy.sh
```
@@ -179,6 +174,7 @@ sudo -E /opt/silo/src/scripts/deploy.sh
| File | Purpose |
|------|---------|
| `/opt/silo/bin/silod` | Server binary |
| `/opt/silo/data/` | File storage root |
| `/opt/silo/src/` | Git repository checkout |
| `/etc/silo/config.yaml` | Server configuration |
| `/etc/silo/silod.env` | Environment variables (secrets) |
@@ -238,7 +234,7 @@ sudo journalctl -u silod --since "2024-01-15 10:00:00"
# Basic health check
curl http://localhost:8080/health
# Full readiness check (includes DB and MinIO)
# Full readiness check (includes DB)
curl http://localhost:8080/ready
```
@@ -247,7 +243,7 @@ curl http://localhost:8080/ready
To update to the latest version:
```bash
ssh root@silo.kindred.internal
ssh root@silo.example.internal
/opt/silo/src/scripts/deploy.sh
```
@@ -269,7 +265,7 @@ When new migrations are added, run them manually:
ls -la /opt/silo/src/migrations/
# Run a specific migration
psql -h psql.kindred.internal -U silo -d silo -f /opt/silo/src/migrations/008_new_feature.sql
psql -h psql.example.internal -U silo -d silo -f /opt/silo/src/migrations/008_new_feature.sql
```
---
@@ -303,35 +299,17 @@ psql -h psql.kindred.internal -U silo -d silo -f /opt/silo/src/migrations/008_ne
1. Test network connectivity:
```bash
nc -zv psql.kindred.internal 5432
nc -zv psql.example.internal 5432
```
2. Test credentials:
```bash
source /etc/silo/silod.env
PGPASSWORD=$SILO_DB_PASSWORD psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1'
PGPASSWORD=$SILO_DB_PASSWORD psql -h psql.example.internal -U silo -d silo -c 'SELECT 1'
```
3. Check `pg_hba.conf` on PostgreSQL server allows connections from this host.
### Connection Refused to MinIO
1. Test network connectivity:
```bash
nc -zv minio.kindred.internal 9000
```
2. Test with curl:
```bash
curl -I http://minio.kindred.internal:9000/minio/health/live
```
3. Check SSL settings in config match MinIO setup:
```yaml
storage:
use_ssl: true # or false
```
### Health Check Fails
```bash
@@ -340,8 +318,10 @@ curl -v http://localhost:8080/health
curl -v http://localhost:8080/ready
# If ready fails but health passes, check external services
psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1'
curl http://minio.kindred.internal:9000/minio/health/live
psql -h psql.example.internal -U silo -d silo -c 'SELECT 1'
# Check file storage directory
ls -la /opt/silo/data
```
### Build Fails
@@ -391,14 +371,14 @@ This script:
getcert list
```
2. The silo config is already updated to use `https://silo.kindred.internal` as base URL. Restart silo:
2. The silo config is already updated to use `https://silo.example.internal` as base URL. Restart silo:
```bash
sudo systemctl restart silod
```
3. Test the setup:
```bash
curl https://silo.kindred.internal/health
curl https://silo.example.internal/health
```
### Certificate Management
@@ -422,7 +402,7 @@ For clients to trust the Silo HTTPS certificate, they need the IPA CA:
```bash
# Download CA cert
curl -o /tmp/ipa-ca.crt https://ipa.kindred.internal/ipa/config/ca.crt
curl -o /tmp/ipa-ca.crt https://ipa.example.internal/ipa/config/ca.crt
# Ubuntu/Debian
sudo cp /tmp/ipa-ca.crt /usr/local/share/ca-certificates/ipa-ca.crt
@@ -456,10 +436,9 @@ sudo systemctl reload nginx
- [ ] `/etc/silo/silod.env` has mode 600 (`chmod 600`)
- [ ] Database password is strong and unique
- [ ] MinIO credentials are specific to silo (not admin)
- [ ] SSL/TLS enabled for PostgreSQL (`sslmode: require`)
- [ ] SSL/TLS enabled for MinIO (`use_ssl: true`) if available
- [ ] HTTPS enabled via nginx reverse proxy
- [ ] File storage directory (`/opt/silo/data`) owned by `silo` user with mode 750
- [ ] Silod listens on localhost only (`host: 127.0.0.1`)
- [ ] Firewall allows only ports 80, 443 (not 8080)
- [ ] Service runs as non-root `silo` user

View File

@@ -1,13 +1,15 @@
# Silo Gap Analysis and Revision Control Roadmap
# Silo Gap Analysis
**Date:** 2026-02-08
**Date:** 2026-02-13
**Status:** Analysis Complete (Updated)
---
## Executive Summary
This document analyzes the current state of the Silo project against its specification, identifies documentation and feature gaps, and outlines a roadmap for enhanced revision control capabilities.
This document analyzes the current state of the Silo project against its specification and against SOLIDWORKS PDM (the industry-leading product data management solution). It identifies documentation gaps, feature gaps, and outlines a roadmap for enhanced revision control capabilities.
See [ROADMAP.md](ROADMAP.md) for the platform roadmap and dependency tier structure.
---
@@ -25,7 +27,7 @@ This document analyzes the current state of the Silo project against its specifi
| `docs/AUTH.md` | Authentication system design | Current |
| `docs/AUTH_USER_GUIDE.md` | User guide for login, tokens, and roles | Current |
| `docs/GAP_ANALYSIS.md` | Revision control roadmap | Current |
| `ROADMAP.md` | Feature roadmap and SOLIDWORKS PDM comparison | Current |
| `docs/ROADMAP.md` | Platform roadmap and dependency tiers | Current |
| `frontend-spec.md` | React SPA frontend specification | Current |
### 1.2 Documentation Gaps (Priority Order)
@@ -74,7 +76,7 @@ This document analyzes the current state of the Silo project against its specifi
| Append-only revision history | Complete | `internal/db/items.go` |
| Sequential revision numbering | Complete | Database trigger |
| Property snapshots (JSONB) | Complete | `revisions.properties` |
| File versioning (MinIO) | Complete | `internal/storage/` |
| File storage (filesystem) | Complete | `internal/storage/` |
| SHA256 checksums | Complete | Captured on upload |
| Revision comments | Complete | `revisions.comment` |
| User attribution | Complete | `revisions.created_by` |
@@ -91,7 +93,7 @@ CREATE TABLE revisions (
revision_number INTEGER NOT NULL,
properties JSONB NOT NULL DEFAULT '{}',
file_key TEXT,
file_version TEXT, -- MinIO version ID
file_version TEXT, -- storage version ID
file_checksum TEXT, -- SHA256
file_size BIGINT,
thumbnail_key TEXT,
@@ -281,7 +283,7 @@ Effort: Medium | Priority: Low | Risk: Low
**Changes:**
- Add thumbnail generation on file upload
- Store in MinIO at `thumbnails/{part_number}/rev{n}.png`
- Store at `thumbnails/{part_number}/rev{n}.png`
- Expose via `GET /api/items/{pn}/thumbnail/{rev}`
---
@@ -365,7 +367,7 @@ internal/
handlers.go # Items, schemas, projects, revisions
middleware.go # Auth middleware
odoo_handlers.go # Odoo integration endpoints
routes.go # Route registration (75 endpoints)
routes.go # Route registration (78 endpoints)
search.go # Fuzzy search
auth/
auth.go # Auth service: local, LDAP, OIDC
@@ -375,7 +377,7 @@ internal/
relationships.go # BOM repository
projects.go # Project repository
storage/
storage.go # MinIO file storage helpers
storage.go # File storage helpers
migrations/
001_initial.sql # Core schema
...
@@ -450,3 +452,163 @@ GET /api/releases/{name} # Get release details
POST /api/releases/{name}/items # Add items to release
GET /api/items/{pn}/thumbnail/{rev} # Get thumbnail
```
---
## Appendix C: SOLIDWORKS PDM Comparison
This section compares Silo's capabilities against SOLIDWORKS PDM features. Gaps are categorized by priority and implementation complexity.
**Legend:** Silo Status = Full / Partial / None | Priority = Critical / High / Medium / Low | Complexity = Simple / Moderate / Complex
### C.1 Version Control & Revision Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Check-in/check-out | Full pessimistic locking | None | High | Moderate |
| Version history | Complete with branching | Full (linear) | - | - |
| Revision labels | A, B, C or custom schemes | Full (custom labels) | - | - |
| Rollback/restore | Full | Full | - | - |
| Compare revisions | Visual + metadata diff | Metadata diff only | Medium | Complex |
| Get Latest Revision | One-click retrieval | Partial (API only) | Medium | Simple |
Silo lacks pessimistic locking (check-out), which is critical for multi-user CAD environments where file merging is impractical. Visual diff comparison would require FreeCAD integration for CAD file visualization.
### C.2 Workflow Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Custom workflows | Full visual designer | None | Critical | Complex |
| State transitions | Configurable with permissions | Basic (status field only) | Critical | Complex |
| Parallel approvals | Multiple approvers required | None | High | Complex |
| Automatic transitions | Timer/condition-based | None | Medium | Moderate |
| Email notifications | On state change | None | High | Moderate |
| ECO process | Built-in change management | None | High | Complex |
| Child state conditions | Block parent if children invalid | None | Medium | Moderate |
Workflow management is the largest functional gap. SOLIDWORKS PDM offers sophisticated state machines with parallel approvals, automatic transitions, and deep integration with engineering change processes. Silo currently has only a simple status field (draft/review/released/obsolete) with no transition rules or approval processes.
### C.3 User Management & Security
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| User authentication | Windows AD, LDAP | Full (local, LDAP, OIDC) | - | - |
| Role-based permissions | Granular per folder/state | Partial (3-tier role model) | Medium | Moderate |
| Group management | Full | None | Medium | Moderate |
| Folder permissions | Read/write/delete per folder | None | Medium | Moderate |
| State permissions | Actions allowed per state | None | High | Moderate |
| Audit trail | Complete action logging | Full | - | - |
| Private files | Pre-check-in visibility control | None | Low | Simple |
Authentication is implemented with three backends (local, LDAP/FreeIPA, OIDC/Keycloak) and a 3-tier role model (admin > editor > viewer). Audit logging captures user actions. Remaining gaps: group management, folder-level permissions, and state-based permission rules.
### C.4 Search & Discovery
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Metadata search | Full with custom cards | Partial (API query params + fuzzy) | High | Moderate |
| Full-text content search | iFilters for Office, CAD | None | Medium | Complex |
| Quick search | Toolbar with history | Partial (fuzzy search API) | Medium | Simple |
| Saved searches | User-defined favorites | None | Medium | Simple |
| Advanced operators | AND, OR, NOT, wildcards | None | Medium | Simple |
| Multi-variable search | Search across multiple fields | None | Medium | Simple |
| Where-used search | Find all assemblies using part | Full | - | - |
Silo has API-level filtering, fuzzy search, and where-used queries. Remaining gaps: saved searches, advanced search operators, and a richer search UI. Content search (searching within CAD files) is not planned for the server.
### C.5 BOM Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Single-level BOM | Yes | Full | - | - |
| Multi-level BOM | Indented/exploded views | Full (recursive, configurable depth) | - | - |
| BOM comparison | Between revisions | None | Medium | Moderate |
| BOM export | Excel, XML, ERP formats | Full (CSV, ODS) | - | - |
| BOM import | Bulk BOM loading | Full (CSV with upsert) | - | - |
| Calculated BOMs | Quantities rolled up | None | Medium | Moderate |
| Reference designators | Full support | Full | - | - |
| Alternate parts | Substitute tracking | Full | - | - |
Multi-level BOM retrieval (recursive CTE with configurable depth) and BOM export (CSV, ODS) are implemented. BOM import supports CSV with upsert and cycle detection. Remaining gap: BOM comparison between revisions.
### C.6 CAD Integration
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Native CAD add-in | Deep SOLIDWORKS integration | FreeCAD workbench (silo-mod) | Medium | Complex |
| Property mapping | Bi-directional sync | Planned (silo-mod) | Medium | Moderate |
| Task pane | Embedded in CAD UI | Auth dock panel (silo-mod) | Medium | Complex |
| Lightweight components | Handle without full load | N/A | - | - |
| Drawing/model linking | Automatic association | Manual | Medium | Moderate |
| Multi-CAD support | Third-party formats | FreeCAD only | Low | - |
CAD integration is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The Silo server provides the REST API endpoints consumed by those clients.
### C.7 External Integrations
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| ERP integration | SAP, Dynamics, etc. | Partial (Odoo stubs) | Medium | Complex |
| API access | Full COM/REST API | Full REST API (78 endpoints) | - | - |
| Dispatch scripts | Automation without coding | None | Medium | Moderate |
| Task scheduler | Background processing | None | Medium | Moderate |
| Email system | SMTP integration | None | High | Simple |
| Web portal | Browser access | Full (React SPA + auth) | - | - |
Silo has a comprehensive REST API (78 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation.
### C.8 Reporting & Analytics
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Standard reports | Inventory, usage, activity | None | Medium | Moderate |
| Custom reports | User-defined queries | None | Medium | Moderate |
| Dashboard | Visual KPIs | None | Low | Moderate |
| Export formats | PDF, Excel, CSV | CSV and ODS | Medium | Simple |
Reporting capabilities are absent. Basic reports (item counts, revision activity, where-used) would provide immediate value.
### C.9 File Handling
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| File versioning | Automatic | Full (filesystem) | - | - |
| File preview | Thumbnails, 3D preview | None | Medium | Complex |
| File conversion | PDF, DXF generation | None | Medium | Complex |
| Replication | Multi-site sync | None | Low | Complex |
| File copy with refs | Copy tree with references | None | Medium | Moderate |
File storage works well. Thumbnail generation and file preview would significantly improve the web UI experience. Automatic conversion to PDF/DXF is valuable for sharing with non-CAD users.
---
## Appendix D: Feature Comparison Matrix
| Category | Feature | SW PDM Standard | SW PDM Pro | Silo Current | Silo Planned |
|----------|---------|-----------------|------------|--------------|--------------|
| **Version Control** | Check-in/out | Yes | Yes | No | Tier 1 |
| | Version history | Yes | Yes | Yes | - |
| | Rollback | Yes | Yes | Yes | - |
| | Revision labels/status | Yes | Yes | Yes | - |
| | Revision comparison | Yes | Yes | Yes (metadata) | - |
| **Workflow** | Custom workflows | Limited | Yes | No | Tier 4 |
| | Parallel approval | No | Yes | No | Tier 4 |
| | Notifications | No | Yes | No | Tier 1 |
| **Security** | User auth | Windows | Windows/LDAP | Yes (local, LDAP, OIDC) | - |
| | Permissions | Basic | Granular | Partial (role-based) | Tier 4 |
| | Audit trail | Basic | Full | Yes | - |
| **Search** | Metadata search | Yes | Yes | Partial (API + fuzzy) | Tier 0 |
| | Content search | No | Yes | No | Tier 2 |
| | Where-used | Yes | Yes | Yes | - |
| **BOM** | Single-level | Yes | Yes | Yes | - |
| | Multi-level | Yes | Yes | Yes (recursive) | - |
| | BOM export | Yes | Yes | Yes (CSV, ODS) | - |
| **Data** | CSV import/export | Yes | Yes | Yes | - |
| | ODS import/export | No | No | Yes | - |
| | Project management | Yes | Yes | Yes | - |
| **Integration** | API | Limited | Full | Full REST (78) | - |
| | ERP connectors | No | Yes | Partial (Odoo stubs) | Tier 6 |
| | Web access | No | Yes | Yes (React SPA + auth) | - |
| **Files** | Versioning | Yes | Yes | Yes | - |
| | Preview | Yes | Yes | No | Tier 2 |
| | Multi-site | No | Yes | No | Not Planned |

494
docs/INSTALL.md Normal file
View File

@@ -0,0 +1,494 @@
# Installing Silo
This guide covers two installation methods:
- **[Option A: Docker Compose](#option-a-docker-compose)** — self-contained stack with all services. Recommended for evaluation, small teams, and environments where Docker is the standard.
- **[Option B: Daemon Install](#option-b-daemon-install-systemd--external-services)** — systemd service with external PostgreSQL and optional LDAP/nginx. Files are stored on the local filesystem. Recommended for production deployments integrated with existing infrastructure.
Both methods produce the same result: a running Silo server with a web UI, REST API, and authentication.
---
## Table of Contents
- [Prerequisites](#prerequisites)
- [Option A: Docker Compose](#option-a-docker-compose)
- [A.1 Prerequisites](#a1-prerequisites)
- [A.2 Clone the Repository](#a2-clone-the-repository)
- [A.3 Run the Setup Script](#a3-run-the-setup-script)
- [A.4 Start the Stack](#a4-start-the-stack)
- [A.5 Verify the Installation](#a5-verify-the-installation)
- [A.6 LDAP Users and Groups](#a6-ldap-users-and-groups)
- [A.7 Optional: Enable Nginx Reverse Proxy](#a7-optional-enable-nginx-reverse-proxy)
- [A.8 Stopping, Starting, and Upgrading](#a8-stopping-starting-and-upgrading)
- [Option B: Daemon Install (systemd + External Services)](#option-b-daemon-install-systemd--external-services)
- [B.1 Architecture Overview](#b1-architecture-overview)
- [B.2 Prerequisites](#b2-prerequisites)
- [B.3 Set Up External Services](#b3-set-up-external-services)
- [B.4 Prepare the Host](#b4-prepare-the-host)
- [B.5 Configure Credentials](#b5-configure-credentials)
- [B.6 Deploy](#b6-deploy)
- [B.7 Set Up Nginx and TLS](#b7-set-up-nginx-and-tls)
- [B.8 Verify the Installation](#b8-verify-the-installation)
- [B.9 Upgrading](#b9-upgrading)
- [Post-Install Configuration](#post-install-configuration)
- [Further Reading](#further-reading)
---
## Prerequisites
Regardless of which method you choose:
- **Git** to clone the repository
- A machine with at least **2 GB RAM** and **10 GB free disk**
- Network access to pull container images or download Go/Node toolchains
---
## Option A: Docker Compose
A single Docker Compose file runs everything: PostgreSQL, OpenLDAP, and Silo. Files are stored on the local filesystem. An optional nginx container can be enabled for reverse proxying.
### A.1 Prerequisites
- [Docker Engine](https://docs.docker.com/engine/install/) 24+ with the [Compose plugin](https://docs.docker.com/compose/install/) (v2)
- `openssl` (used by the setup script to generate secrets)
Verify your installation:
```bash
docker --version # Docker Engine 24+
docker compose version # Docker Compose v2+
```
### A.2 Clone the Repository
```bash
git clone https://git.kindred-systems.com/kindred/silo.git
cd silo
```
### A.3 Run the Setup Script
The setup script generates credentials and configuration files:
```bash
./scripts/setup-docker.sh
```
It prompts for:
- Server domain (default: `localhost`)
- PostgreSQL password (auto-generated if you press Enter)
- OpenLDAP admin password and initial user (auto-generated)
- Silo local admin account (fallback when LDAP is unavailable)
For automated/CI environments, use non-interactive mode:
```bash
./scripts/setup-docker.sh --non-interactive
```
The script writes two files:
- `deployments/.env` — secrets for Docker Compose
- `deployments/config.docker.yaml` — Silo server configuration
### A.4 Start the Stack
```bash
docker compose -f deployments/docker-compose.allinone.yaml up -d
```
Wait for all services to become healthy:
```bash
docker compose -f deployments/docker-compose.allinone.yaml ps
```
You should see `silo-postgres`, `silo-openldap`, and `silo-api` all in a healthy state.
View logs:
```bash
# All services
docker compose -f deployments/docker-compose.allinone.yaml logs -f
# Silo only
docker compose -f deployments/docker-compose.allinone.yaml logs -f silo
```
### A.5 Verify the Installation
```bash
# Health check
curl http://localhost:8080/health
# Readiness check (includes database connectivity)
curl http://localhost:8080/ready
```
Open http://localhost:8080 in your browser. Log in with either:
- **LDAP account**: the username and password shown by the setup script (default: `siloadmin`)
- **Local admin**: the local admin credentials shown by the setup script (default: `admin`)
The credentials were printed at the end of the setup script output and are stored in `deployments/.env`.
### A.6 LDAP Users and Groups
The Docker stack includes an OpenLDAP server with three preconfigured groups that map to Silo roles:
| LDAP Group | Silo Role | Access Level |
|------------|-----------|-------------|
| `cn=silo-admins,ou=groups,dc=silo,dc=local` | admin | Full access |
| `cn=silo-users,ou=groups,dc=silo,dc=local` | editor | Create and modify items |
| `cn=silo-viewers,ou=groups,dc=silo,dc=local` | viewer | Read-only |
The initial LDAP user (default: `siloadmin`) is added to `silo-admins`.
**Add a new LDAP user:**
```bash
# From the host (using the exposed port)
ldapadd -x -H ldap://localhost:1389 \
-D "cn=admin,dc=silo,dc=local" \
-w "YOUR_LDAP_ADMIN_PASSWORD" << EOF
dn: cn=jdoe,ou=users,dc=silo,dc=local
objectClass: inetOrgPerson
cn: jdoe
sn: Doe
userPassword: changeme
mail: jdoe@example.com
EOF
```
**Add a user to a group:**
```bash
ldapmodify -x -H ldap://localhost:1389 \
-D "cn=admin,dc=silo,dc=local" \
-w "YOUR_LDAP_ADMIN_PASSWORD" << EOF
dn: cn=silo-users,ou=groups,dc=silo,dc=local
changetype: modify
add: member
member: cn=jdoe,ou=users,dc=silo,dc=local
EOF
```
**List all users:**
```bash
ldapsearch -x -H ldap://localhost:1389 \
-b "ou=users,dc=silo,dc=local" \
-D "cn=admin,dc=silo,dc=local" \
-w "YOUR_LDAP_ADMIN_PASSWORD" "(objectClass=inetOrgPerson)" cn mail memberOf
```
### A.7 Optional: Enable Nginx Reverse Proxy
To place nginx in front of Silo (for TLS termination or to serve on port 80):
```bash
docker compose -f deployments/docker-compose.allinone.yaml --profile nginx up -d
```
By default nginx listens on ports 80 and 443 and proxies to the Silo container. The configuration is at `deployments/nginx/nginx.conf`.
**To enable HTTPS**, edit `deployments/docker-compose.allinone.yaml` and uncomment the TLS certificate volume mounts in the `nginx` service, then uncomment the HTTPS server block in `deployments/nginx/nginx.conf`. See the comments in those files for details.
If you already have your own reverse proxy or load balancer, skip the nginx profile and point your proxy at port 8080.
### A.8 Stopping, Starting, and Upgrading
```bash
# Stop the stack (data is preserved in Docker volumes)
docker compose -f deployments/docker-compose.allinone.yaml down
# Start again
docker compose -f deployments/docker-compose.allinone.yaml up -d
# Stop and delete all data (WARNING: destroys database, files, and LDAP data)
docker compose -f deployments/docker-compose.allinone.yaml down -v
```
**To upgrade to a newer version:**
```bash
cd silo
git pull
docker compose -f deployments/docker-compose.allinone.yaml up -d --build
```
The Silo container is rebuilt from the updated source. Database migrations in `migrations/` are applied automatically on container startup via the PostgreSQL init mechanism.
---
## Option B: Daemon Install (systemd + External Services)
This method runs Silo as a systemd service on a dedicated host, connecting to externally managed PostgreSQL and optionally LDAP services. Files are stored on the local filesystem.
### B.1 Architecture Overview
```
┌──────────────────────┐
│ Silo Host │
│ ┌────────────────┐ │
HTTPS (443) ──►│ │ nginx │ │
│ └───────┬────────┘ │
│ │ :8080 │
│ ┌───────▼────────┐ │
│ │ silod │ │
│ │ (API server) │ │
│ │ Files: /opt/ │ │
│ │ silo/data │ │
│ └──────┬─────────┘ │
└─────────┼────────────┘
┌───────────▼──┐
│ PostgreSQL 16│
│ :5432 │
└──────────────┘
```
### B.2 Prerequisites
- Linux host (Debian/Ubuntu or RHEL/Fedora/AlmaLinux)
- Root or sudo access
- Network access to your PostgreSQL server
The setup script installs Go and other build dependencies automatically.
### B.3 Set Up External Services
#### PostgreSQL 16
Install PostgreSQL and create the Silo database:
- [PostgreSQL downloads](https://www.postgresql.org/download/)
```bash
# After installing PostgreSQL, create the database and user:
sudo -u postgres createuser silo
sudo -u postgres createdb -O silo silo
sudo -u postgres psql -c "ALTER USER silo WITH PASSWORD 'your-password';"
```
Ensure the Silo host can connect (check `pg_hba.conf` on the PostgreSQL server).
Verify:
```bash
psql -h YOUR_PG_HOST -U silo -d silo -c 'SELECT 1'
```
#### LDAP / FreeIPA (Optional)
For LDAP authentication, you need an LDAP server with user and group entries. Options:
- [FreeIPA](https://www.freeipa.org/page/Quick_Start_Guide) — full identity management (recommended for organizations already using it)
- [OpenLDAP](https://www.openldap.org/doc/admin26/) — lightweight LDAP server
Silo needs:
- A base DN (e.g., `dc=example,dc=com`)
- Users under a known OU (e.g., `cn=users,cn=accounts,dc=example,dc=com`)
- Groups that map to Silo roles (`admin`, `editor`, `viewer`)
- The `memberOf` overlay enabled (so user entries have `memberOf` attributes)
See [CONFIGURATION.md — LDAP](CONFIGURATION.md#ldap--freeipa) for the full LDAP configuration reference.
### B.4 Prepare the Host
Run the setup script on the target host:
```bash
# Copy and run the script
scp scripts/setup-host.sh root@YOUR_HOST:/tmp/
ssh root@YOUR_HOST 'bash /tmp/setup-host.sh'
```
Or directly on the host:
```bash
sudo bash scripts/setup-host.sh
```
The script:
1. Installs dependencies (git, Go 1.24)
2. Creates the `silo` system user
3. Creates directories (`/opt/silo`, `/etc/silo`)
4. Clones the repository
5. Creates the environment file template
To override the default database hostname:
```bash
SILO_DB_HOST=db.example.com sudo -E bash scripts/setup-host.sh
```
### B.5 Configure Credentials
Edit the environment file with your service credentials:
```bash
sudo nano /etc/silo/silod.env
```
```bash
# Database
SILO_DB_PASSWORD=your-database-password
# Authentication
SILO_SESSION_SECRET=generate-a-long-random-string
SILO_ADMIN_USERNAME=admin
SILO_ADMIN_PASSWORD=your-admin-password
```
Generate a session secret:
```bash
openssl rand -hex 32
```
Review the server configuration:
```bash
sudo nano /etc/silo/config.yaml
```
Update `database.host`, `storage.filesystem.root_dir`, `server.base_url`, and authentication settings for your environment. See [CONFIGURATION.md](CONFIGURATION.md) for all options.
### B.6 Deploy
Run the deploy script:
```bash
sudo /opt/silo/src/scripts/deploy.sh
```
The script:
1. Pulls latest code from git
2. Builds the `silod` binary and React frontend
3. Installs files to `/opt/silo` and `/etc/silo`
4. Runs database migrations
5. Installs and starts the systemd service
Deploy options:
```bash
# Skip git pull (use current checkout)
sudo /opt/silo/src/scripts/deploy.sh --no-pull
# Skip build (use existing binary)
sudo /opt/silo/src/scripts/deploy.sh --no-build
# Just restart the service
sudo /opt/silo/src/scripts/deploy.sh --restart-only
# Check service status
sudo /opt/silo/src/scripts/deploy.sh --status
```
To override the target host:
```bash
SILO_DEPLOY_TARGET=silo.example.com sudo -E scripts/deploy.sh
```
### B.7 Set Up Nginx and TLS
#### With FreeIPA (automated)
If your organization uses FreeIPA, the included script handles nginx setup, IPA enrollment, and certificate issuance:
```bash
sudo /opt/silo/src/scripts/setup-ipa-nginx.sh
```
Override the hostname if needed:
```bash
SILO_HOSTNAME=silo.example.com sudo -E /opt/silo/src/scripts/setup-ipa-nginx.sh
```
The script installs nginx, enrolls the host in FreeIPA, requests a TLS certificate from the IPA CA (auto-renewed by certmonger), and configures nginx as an HTTPS reverse proxy.
#### Manual nginx setup
Install nginx and create a config:
```bash
sudo apt install nginx # or: sudo dnf install nginx
```
Use the template at `deployments/nginx/nginx.conf` as a starting point. Copy it to `/etc/nginx/sites-available/silo`, update the `server_name` and certificate paths, then enable it:
```bash
sudo ln -sf /etc/nginx/sites-available/silo /etc/nginx/sites-enabled/silo
sudo nginx -t
sudo systemctl reload nginx
```
After enabling HTTPS, update `server.base_url` in `/etc/silo/config.yaml` to use `https://` and restart Silo:
```bash
sudo systemctl restart silod
```
### B.8 Verify the Installation
```bash
# Service status
sudo systemctl status silod
# Health check
curl http://localhost:8080/health
# Readiness check
curl http://localhost:8080/ready
# Follow logs
sudo journalctl -u silod -f
```
Open your configured base URL in a browser and log in.
### B.9 Upgrading
```bash
# Pull latest code and redeploy
sudo /opt/silo/src/scripts/deploy.sh
# Or deploy a specific version
cd /opt/silo/src
git fetch --all --tags
git checkout v1.2.3
sudo /opt/silo/src/scripts/deploy.sh --no-pull
```
New database migrations are applied automatically during deployment.
---
## Post-Install Configuration
After a successful installation:
- **Authentication**: Configure LDAP, OIDC, or local auth backends. See [CONFIGURATION.md — Authentication](CONFIGURATION.md#authentication).
- **Schemas**: Part numbering schemas are loaded from YAML files. See the `schemas/` directory and [CONFIGURATION.md — Schemas](CONFIGURATION.md#schemas).
- **Read-only mode**: Toggle write protection at runtime with `kill -USR1 $(pidof silod)` or by setting `server.read_only: true` in the config.
- **Ongoing maintenance**: See [DEPLOYMENT.md](DEPLOYMENT.md) for service management, log viewing, troubleshooting, and the security checklist.
---
## Further Reading
| Document | Description |
|----------|-------------|
| [CONFIGURATION.md](CONFIGURATION.md) | Complete `config.yaml` reference |
| [DEPLOYMENT.md](DEPLOYMENT.md) | Operations guide: maintenance, troubleshooting, security |
| [AUTH.md](AUTH.md) | Authentication system design |
| [AUTH_USER_GUIDE.md](AUTH_USER_GUIDE.md) | User guide for login, tokens, and roles |
| [SPECIFICATION.md](SPECIFICATION.md) | Full design specification and API reference |
| [STATUS.md](STATUS.md) | Implementation status |
| [GAP_ANALYSIS.md](GAP_ANALYSIS.md) | Gap analysis and revision control roadmap |
| [COMPONENT_AUDIT.md](COMPONENT_AUDIT.md) | Component audit tool design |

485
docs/KC_SERVER.md Normal file
View File

@@ -0,0 +1,485 @@
# .kc Server-Side Metadata Integration
**Status:** Draft
**Date:** February 2026
---
## 1. Purpose
When a `.kc` file is committed to Silo, the server extracts and indexes the `silo/` directory contents so that metadata is queryable, diffable, and streamable without downloading the full file. This document specifies the server-side processing pipeline, database storage, API endpoints, and SSE events that support the Create viewport widgets defined in [SILO_VIEWPORT.md](SILO_VIEWPORT.md).
The core principle: **the `.kc` file is the transport format; Silo is the index.** The `silo/` directory entries are extracted into database columns on commit and packed back into the ZIP on checkout. The server never modifies the FreeCAD standard zone (`Document.xml`, `.brp` files, `thumbnails/`).
---
## 2. Commit Pipeline
When a `.kc` file is uploaded via `POST /api/items/{partNumber}/file`, the server runs an extraction pipeline before returning success.
### 2.1 Pipeline Steps
```
Client uploads .kc file
|
v
+-----------------------------+
| 1. Store file to disk | (existing behavior -- unchanged)
| items/{pn}/rev{N}.kc |
+-----------------------------+
|
v
+-----------------------------+
| 2. Open ZIP, read silo/ |
| Parse each entry |
+-----------------------------+
|
v
+-----------------------------+
| 3. Validate manifest.json |
| - UUID matches item |
| - kc_version supported |
| - revision_hash present |
+-----------------------------+
|
v
+-----------------------------+
| 4. Index metadata |
| - Upsert item_metadata |
| - Upsert dependencies |
| - Append history entry |
| - Snapshot approvals |
| - Register macros |
| - Register job defs |
+-----------------------------+
|
v
+-----------------------------+
| 5. Broadcast SSE events |
| - revision.created |
| - metadata.updated |
| - bom.changed (if deps |
| differ from previous) |
+-----------------------------+
|
v
Return 201 Created
```
### 2.2 Validation Rules
| Check | Failure response |
|-------|-----------------|
| `silo/manifest.json` missing | `400 Bad Request` -- file is `.fcstd` not `.kc` |
| `manifest.uuid` doesn't match item's UUID | `409 Conflict` -- wrong item |
| `manifest.kc_version` > server's supported version | `422 Unprocessable` -- client newer than server |
| `manifest.revision_hash` matches current head | `200 OK` (no-op, file unchanged) |
| Any `silo/` JSON fails to parse | `422 Unprocessable` with path and parse error |
If validation fails, the blob is still stored (the user uploaded it), but no metadata indexing occurs. The item's revision is created with a `metadata_error` flag so the web UI can surface the problem.
### 2.3 Backward Compatibility
Plain `.fcstd` files (no `silo/` directory) continue to work exactly as today -- stored on disk, revision created, no metadata extraction. The pipeline short-circuits at step 2 when no `silo/` directory is found.
---
## 3. Database Schema
### 3.1 `item_metadata` Table
Stores the indexed contents of `silo/metadata.json` as structured JSONB, searchable and filterable via the existing item query endpoints.
```sql
CREATE TABLE item_metadata (
item_id UUID PRIMARY KEY REFERENCES items(id) ON DELETE CASCADE,
schema_name TEXT,
tags TEXT[] NOT NULL DEFAULT '{}',
lifecycle_state TEXT NOT NULL DEFAULT 'draft',
fields JSONB NOT NULL DEFAULT '{}',
kc_version TEXT,
manifest_uuid UUID,
silo_instance TEXT,
revision_hash TEXT,
updated_at TIMESTAMPTZ DEFAULT now(),
updated_by TEXT
);
CREATE INDEX idx_item_metadata_tags ON item_metadata USING GIN (tags);
CREATE INDEX idx_item_metadata_lifecycle ON item_metadata (lifecycle_state);
CREATE INDEX idx_item_metadata_fields ON item_metadata USING GIN (fields);
```
On commit, the server upserts this row from `silo/manifest.json` and `silo/metadata.json`. The `fields` column contains the schema-driven key-value pairs exactly as they appear in the JSON.
### 3.2 `item_dependencies` Table
Stores the indexed contents of `silo/dependencies.json`. Replaces the BOM for assembly relationships that originate from the CAD model.
```sql
CREATE TABLE item_dependencies (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
parent_item_id UUID REFERENCES items(id) ON DELETE CASCADE,
child_uuid UUID NOT NULL,
child_part_number TEXT,
child_revision INTEGER,
quantity DECIMAL,
label TEXT,
relationship TEXT NOT NULL DEFAULT 'component',
revision_number INTEGER NOT NULL,
created_at TIMESTAMPTZ DEFAULT now()
);
CREATE INDEX idx_item_deps_parent ON item_dependencies (parent_item_id);
CREATE INDEX idx_item_deps_child ON item_dependencies (child_uuid);
```
This table complements the existing `relationships` table. The `relationships` table is the server-authoritative BOM (editable via the web UI and API). The `item_dependencies` table is the CAD-authoritative record extracted from the file. BOM merge (per [BOM_MERGE.md](BOM_MERGE.md)) reconciles the two.
### 3.3 `item_approvals` Table
Stores the indexed contents of `silo/approvals.json`. Server-authoritative -- the `.kc` snapshot is a read cache.
```sql
CREATE TABLE item_approvals (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
item_id UUID REFERENCES items(id) ON DELETE CASCADE,
eco_number TEXT,
state TEXT NOT NULL DEFAULT 'draft',
updated_at TIMESTAMPTZ DEFAULT now(),
updated_by TEXT
);
CREATE TABLE approval_signatures (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
approval_id UUID REFERENCES item_approvals(id) ON DELETE CASCADE,
username TEXT NOT NULL,
role TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending',
signed_at TIMESTAMPTZ,
comment TEXT
);
```
These tables exist independent of `.kc` commits -- approvals are created and managed through the web UI and API. On `.kc` checkout, the current approval state is serialized into `silo/approvals.json` for offline display.
### 3.4 `item_macros` Table
Registers macros from `silo/macros/` for server-side discoverability and the future Macro Store module.
```sql
CREATE TABLE item_macros (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
item_id UUID REFERENCES items(id) ON DELETE CASCADE,
filename TEXT NOT NULL,
trigger TEXT NOT NULL DEFAULT 'manual',
content TEXT NOT NULL,
revision_number INTEGER NOT NULL,
created_at TIMESTAMPTZ DEFAULT now(),
UNIQUE(item_id, filename)
);
```
---
## 4. API Endpoints
These endpoints serve the viewport widgets in Create. All are under `/api/items/{partNumber}` and follow the existing auth model.
### 4.1 Metadata
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/metadata` | viewer | Get indexed metadata (schema fields, tags, lifecycle) |
| `PUT` | `/metadata` | editor | Update metadata fields from client |
| `PATCH` | `/metadata/lifecycle` | editor | Transition lifecycle state |
| `PATCH` | `/metadata/tags` | editor | Add/remove tags |
**`GET /api/items/{partNumber}/metadata`**
Returns the indexed metadata for viewport display. This is the fast path -- reads from `item_metadata` rather than downloading and parsing the `.kc` ZIP.
```json
{
"schema_name": "mechanical-part-v2",
"lifecycle_state": "draft",
"tags": ["structural", "aluminum"],
"fields": {
"material": "6061-T6",
"finish": "anodized",
"weight_kg": 0.34,
"category": "bracket"
},
"manifest": {
"uuid": "550e8400-e29b-41d4-a716-446655440000",
"silo_instance": "https://silo.example.com",
"revision_hash": "a1b2c3d4e5f6",
"kc_version": "1.0"
},
"updated_at": "2026-02-13T20:30:00Z",
"updated_by": "joseph"
}
```
**`PUT /api/items/{partNumber}/metadata`**
Accepts a partial update of schema fields. The server merges into the existing `fields` JSONB. This is the write-back path for the Metadata Editor widget.
```json
{
"fields": {
"material": "7075-T6",
"weight_kg": 0.31
}
}
```
The server validates field names against the schema descriptor. Unknown fields are rejected with `422`.
**`PATCH /api/items/{partNumber}/metadata/lifecycle`**
Transitions lifecycle state. The server validates the transition is permitted (e.g., `draft` -> `review` is allowed, `released` -> `draft` is not without admin override).
```json
{ "state": "review" }
```
### 4.2 Dependencies
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/dependencies` | viewer | Get CAD-extracted dependency list |
| `GET` | `/dependencies/resolve` | viewer | Resolve UUIDs to current part numbers and file status |
**`GET /api/items/{partNumber}/dependencies`**
Returns the raw dependency list from the last `.kc` commit.
**`GET /api/items/{partNumber}/dependencies/resolve`**
Returns the dependency list with each UUID resolved to its current part number, revision, and whether the file exists on disk. This is what the Dependency Table widget calls to populate the status column.
```json
{
"links": [
{
"uuid": "660e8400-...",
"part_number": "KC-BRK-0042",
"label": "Base Plate",
"revision": 2,
"quantity": 1,
"resolved": true,
"file_available": true
},
{
"uuid": "770e8400-...",
"part_number": "KC-HDW-0108",
"label": "M6 SHCS",
"revision": 1,
"quantity": 4,
"resolved": true,
"file_available": true
},
{
"uuid": "880e8400-...",
"part_number": null,
"label": "Cover Panel",
"revision": 1,
"quantity": 1,
"resolved": false,
"file_available": false
}
]
}
```
### 4.3 Approvals
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/approvals` | viewer | Get current approval state |
| `POST` | `/approvals` | editor | Create ECO / start approval workflow |
| `POST` | `/approvals/{id}/sign` | editor | Sign (approve/reject) |
These endpoints power the Approvals Viewer widget. The viewer is read-only in Create -- sign actions happen in the web UI, but the API exists for both.
### 4.4 Macros
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/macros` | viewer | List registered macros |
| `GET` | `/macros/{filename}` | viewer | Get macro source |
Read-only server-side. Macros are authored in Create and committed inside the `.kc`. The server indexes them for discoverability in the future Macro Store.
### 4.5 Existing Endpoints (unchanged)
The viewport widgets also consume these existing endpoints:
| Widget | Endpoint | Purpose |
|--------|----------|---------|
| History Viewer | `GET /api/items/{pn}/revisions` | Full revision list |
| History Viewer | `GET /api/items/{pn}/revisions/compare` | Property diff |
| Job Viewer | `GET /api/jobs?item={pn}&definition={name}&limit=1` | Last job run |
| Job Viewer | `POST /api/jobs` | Trigger job |
| Job Viewer | `GET /api/jobs/{id}/logs` | Job log |
| Manifest Viewer | `GET /api/items/{pn}` | Item details (UUID, etc.) |
No changes needed to these -- they already exist and return the data the widgets need.
---
## 5. Checkout Pipeline
When a client downloads a `.kc` via `GET /api/items/{partNumber}/file`, the server packs current server-side state into the `silo/` directory before serving the file. This ensures the client always gets the latest metadata, even if it was edited via the web UI since the last commit.
### 5.1 Pipeline Steps
```
Client requests file download
|
v
+-----------------------------+
| 1. Read .kc from disk |
+-----------------------------+
|
v
+-----------------------------+
| 2. Pack silo/ from DB |
| - manifest.json (item) |
| - metadata.json (index) |
| - history.json (revs) |
| - approvals.json (ECO) |
| - dependencies.json |
| - macros/ (index) |
| - jobs/ (job defs) |
+-----------------------------+
|
v
+-----------------------------+
| 3. Replace silo/ in ZIP |
| Remove old entries |
| Write packed entries |
+-----------------------------+
|
v
Stream .kc to client
```
### 5.2 Packing Rules
| `silo/` entry | Source | Notes |
|---------------|--------|-------|
| `manifest.json` | `item_metadata` + `items` table | UUID from item, revision_hash from latest revision |
| `metadata.json` | `item_metadata.fields` + tags + lifecycle | Serialized from indexed columns |
| `history.json` | `revisions` table | Last 20 revisions for this item |
| `approvals.json` | `item_approvals` + `approval_signatures` | Current ECO state, omitted if no active ECO |
| `dependencies.json` | `item_dependencies` | Current revision's dependency list |
| `macros/*.py` | `item_macros` | All registered macros |
| `jobs/*.yaml` | `job_definitions` filtered by item type | Job definitions matching this item's trigger filters |
### 5.3 Caching
Packing the `silo/` directory on every download has a cost. To mitigate:
- **ETag header**: The response includes an ETag computed from the revision number + metadata `updated_at`. If the client sends `If-None-Match`, the server can return `304 Not Modified`.
- **Lazy packing**: If the `.kc` blob's `silo/manifest.json` revision_hash matches the current head *and* `item_metadata.updated_at` is older than the blob's upload time, skip repacking entirely -- the blob is already current.
---
## 6. SSE Events
The viewport widgets subscribe to SSE for live updates. These events are broadcast when server-side metadata changes, whether via `.kc` commit, web UI edit, or API call.
| Event | Payload | Trigger |
|-------|---------|---------|
| `metadata.updated` | `{part_number, changed_fields[], lifecycle_state, updated_by}` | Metadata PUT/PATCH |
| `metadata.lifecycle` | `{part_number, from_state, to_state, updated_by}` | Lifecycle transition |
| `metadata.tags` | `{part_number, added[], removed[]}` | Tag add/remove |
| `approval.created` | `{part_number, eco_number, state}` | ECO created |
| `approval.signed` | `{part_number, eco_number, user, role, status}` | Approver action |
| `approval.completed` | `{part_number, eco_number, final_state}` | All approvers acted |
| `dependencies.changed` | `{part_number, added[], removed[], changed[]}` | Dependency diff on commit |
Existing events (`revision.created`, `job.*`, `bom.changed`) continue to work as documented in [SPECIFICATION.md](SPECIFICATION.md) and [WORKERS.md](WORKERS.md).
### 6.1 Widget Subscription Map
| Viewport widget | Subscribes to |
|-----------------|---------------|
| Manifest Viewer | -- (read-only, no live updates) |
| Metadata Editor | `metadata.updated`, `metadata.lifecycle`, `metadata.tags` |
| History Viewer | `revision.created` |
| Approvals Viewer | `approval.created`, `approval.signed`, `approval.completed` |
| Dependency Table | `dependencies.changed` |
| Job Viewer | `job.created`, `job.progress`, `job.completed`, `job.failed` |
| Macro Editor | -- (local-only until committed) |
---
## 7. Web UI Integration
The Silo web UI also benefits from indexed metadata. These are additions to existing pages, not new pages.
### 7.1 Items Page
The item detail panel gains a **Metadata** tab (alongside Main, Properties, Revisions, BOM, Where Used) showing the schema-driven form from `GET /api/items/{pn}/metadata`. Editable for editors.
### 7.2 Items List
New filterable columns: `lifecycle_state`, `tags`. The existing search endpoint gains metadata-aware filtering:
```
GET /api/items?lifecycle=released&tag=aluminum
GET /api/items/search?q=bracket&lifecycle=draft
```
### 7.3 Approvals Page
A new page accessible from the top navigation (visible when a future `approvals` module is enabled). Lists all active ECOs with their approval progress.
---
## 8. Migration
### 8.1 Database Migration
A single migration adds the `item_metadata`, `item_dependencies`, `item_approvals`, `approval_signatures`, and `item_macros` tables. Existing items have no metadata rows -- they're created on first `.kc` commit or via `PUT /api/items/{pn}/metadata`.
### 8.2 Backfill
For items that already have `.kc` files stored on disk (committed before this feature), an admin endpoint re-runs the extraction pipeline:
```
POST /api/admin/reindex-metadata
```
This iterates all items with `.kc` files, opens each ZIP, and indexes the `silo/` contents. Idempotent -- safe to run multiple times.
---
## 9. Implementation Order
| Phase | Server work | Supports client phase |
|-------|------------|----------------------|
| 1 | `item_metadata` table + `GET/PUT /metadata` + commit extraction | SILO_VIEWPORT Phase 1-2 (Manifest, Metadata) |
| 2 | Pack `silo/` on checkout + ETag caching | SILO_VIEWPORT Phase 1-3 |
| 3 | `item_dependencies` table + `/dependencies/resolve` | SILO_VIEWPORT Phase 5 (Dependency Table) |
| 4 | `item_macros` table + `/macros` endpoints | SILO_VIEWPORT Phase 6 (Macro Editor) |
| 5 | `item_approvals` tables + `/approvals` endpoints | SILO_VIEWPORT Phase 7 (Approvals Viewer) |
| 6 | SSE events for metadata/approvals/dependencies | SILO_VIEWPORT Phase 8 (Live integration) |
| 7 | Web UI metadata tab + list filters | Independent of client |
Phases 1-2 are prerequisite for the viewport to work with live data. Phases 3-6 can be built in parallel with client widget development. Phase 7 is web-UI-only and independent.
---
## 10. References
- [SILO_VIEWPORT.md](SILO_VIEWPORT.md) -- Client-side viewport widget specification
- [KC_SPECIFICATION.md](KC_SPECIFICATION.md) -- .kc file format specification
- [SPECIFICATION.md](SPECIFICATION.md) -- Silo server API reference
- [BOM_MERGE.md](BOM_MERGE.md) -- BOM merge rules (dependency reconciliation)
- [WORKERS.md](WORKERS.md) -- Job queue (job viewer data source)
- [MODULES.md](MODULES.md) -- Module system (approval module gating)
- [ROADMAP.md](ROADMAP.md) -- Platform roadmap tiers

745
docs/MODULES.md Normal file
View File

@@ -0,0 +1,745 @@
# Module System Specification
**Status:** Draft
**Last Updated:** 2026-02-14
---
## 1. Purpose
Silo's module system defines the boundary between required infrastructure and optional capabilities. Each module groups a set of API endpoints, UI views, and configuration parameters. Modules can be enabled or disabled at runtime by administrators via the web UI, and clients can query which modules are active to adapt their feature set.
The goal: after initial deployment (where `config.yaml` sets database, storage, and server bind), all further operational configuration happens through the admin settings UI. The YAML file becomes the bootstrap; the database becomes the runtime source of truth.
---
## 2. Module Registry
### 2.1 Required Modules
These cannot be disabled. They define what Silo *is*.
| Module ID | Name | Description |
|-----------|------|-------------|
| `core` | Core PDM | Items, revisions, files, BOM, search, import/export, part number generation |
| `schemas` | Schemas | Part numbering schema parsing, segment management, form descriptors |
| `storage` | Storage | Filesystem storage |
### 2.2 Optional Modules
| Module ID | Name | Default | Description |
|-----------|------|---------|-------------|
| `auth` | Authentication | `true` | Local, LDAP, OIDC authentication and RBAC |
| `projects` | Projects | `true` | Project management and item tagging |
| `audit` | Audit | `true` | Audit logging, completeness scoring |
| `odoo` | Odoo ERP | `false` | Odoo integration (config, sync-log, push/pull) |
| `freecad` | Create Integration | `true` | URI scheme, executable path, client settings |
| `jobs` | Job Queue | `false` | Async compute jobs, runner management |
| `dag` | Dependency DAG | `false` | Feature DAG sync, validation states, interference detection |
### 2.3 Module Dependencies
Some modules require others to function:
| Module | Requires |
|--------|----------|
| `dag` | `jobs` |
| `jobs` | `auth` (runner tokens) |
| `odoo` | `auth` |
When enabling a module, its dependencies are validated. The server rejects enabling `dag` without `jobs`. Disabling a module that others depend on shows a warning listing dependents.
---
## 3. Endpoint-to-Module Mapping
### 3.1 `core` (required)
```
# Health
GET /health
GET /ready
# Items
GET /api/items
GET /api/items/search
GET /api/items/by-uuid/{uuid}
GET /api/items/export.csv
GET /api/items/template.csv
GET /api/items/export.ods
GET /api/items/template.ods
POST /api/items
POST /api/items/import
POST /api/items/import.ods
GET /api/items/{partNumber}
PUT /api/items/{partNumber}
DELETE /api/items/{partNumber}
# Revisions
GET /api/items/{partNumber}/revisions
GET /api/items/{partNumber}/revisions/compare
GET /api/items/{partNumber}/revisions/{revision}
POST /api/items/{partNumber}/revisions
PATCH /api/items/{partNumber}/revisions/{revision}
POST /api/items/{partNumber}/revisions/{revision}/rollback
# Files
GET /api/items/{partNumber}/files
GET /api/items/{partNumber}/file
GET /api/items/{partNumber}/file/{revision}
POST /api/items/{partNumber}/file
POST /api/items/{partNumber}/files
DELETE /api/items/{partNumber}/files/{fileId}
PUT /api/items/{partNumber}/thumbnail
POST /api/uploads/presign
# BOM
GET /api/items/{partNumber}/bom
GET /api/items/{partNumber}/bom/expanded
GET /api/items/{partNumber}/bom/flat
GET /api/items/{partNumber}/bom/cost
GET /api/items/{partNumber}/bom/where-used
GET /api/items/{partNumber}/bom/export.csv
GET /api/items/{partNumber}/bom/export.ods
POST /api/items/{partNumber}/bom
POST /api/items/{partNumber}/bom/import
POST /api/items/{partNumber}/bom/merge
PUT /api/items/{partNumber}/bom/{childPartNumber}
DELETE /api/items/{partNumber}/bom/{childPartNumber}
# .kc Metadata
GET /api/items/{partNumber}/metadata
PUT /api/items/{partNumber}/metadata
PATCH /api/items/{partNumber}/metadata/lifecycle
PATCH /api/items/{partNumber}/metadata/tags
# .kc Dependencies
GET /api/items/{partNumber}/dependencies
GET /api/items/{partNumber}/dependencies/resolve
# .kc Macros
GET /api/items/{partNumber}/macros
GET /api/items/{partNumber}/macros/{filename}
# Part Number Generation
POST /api/generate-part-number
# Sheets
POST /api/sheets/diff
# Settings & Modules (admin)
GET /api/modules
GET /api/admin/settings
GET /api/admin/settings/{module}
PUT /api/admin/settings/{module}
POST /api/admin/settings/{module}/test
```
### 3.2 `schemas` (required)
```
GET /api/schemas
GET /api/schemas/{name}
GET /api/schemas/{name}/form
POST /api/schemas/{name}/segments/{segment}/values
PUT /api/schemas/{name}/segments/{segment}/values/{code}
DELETE /api/schemas/{name}/segments/{segment}/values/{code}
```
### 3.3 `storage` (required)
No dedicated endpoints — storage is consumed internally by file upload/download in `core`. Exposed through admin settings for connection status visibility.
### 3.4 `auth`
```
# Public (login flow)
GET /login
POST /login
POST /logout
GET /auth/oidc
GET /auth/callback
# Authenticated
GET /api/auth/me
GET /api/auth/tokens
POST /api/auth/tokens
DELETE /api/auth/tokens/{id}
# Web UI
GET /settings (account info, tokens)
POST /settings/tokens
POST /settings/tokens/{id}/revoke
```
When `auth` is disabled, all routes are open and a synthetic `dev` admin user is injected (current behavior).
### 3.5 `projects`
```
GET /api/projects
GET /api/projects/{code}
GET /api/projects/{code}/items
GET /api/projects/{code}/sheet.ods
POST /api/projects
PUT /api/projects/{code}
DELETE /api/projects/{code}
# Item-project tagging
GET /api/items/{partNumber}/projects
POST /api/items/{partNumber}/projects
DELETE /api/items/{partNumber}/projects/{code}
```
When disabled: project tag endpoints return `404`, project columns are hidden in UI list views, project filter is removed from item search.
### 3.6 `audit`
```
GET /api/audit/completeness
GET /api/audit/completeness/{partNumber}
```
When disabled: audit log table continues to receive writes (it's part of core middleware), but the completeness scoring endpoints and the Audit page in the web UI are hidden. Future: retention policies, export, and compliance reporting endpoints live here.
### 3.7 `odoo`
```
GET /api/integrations/odoo/config
GET /api/integrations/odoo/sync-log
PUT /api/integrations/odoo/config
POST /api/integrations/odoo/test-connection
POST /api/integrations/odoo/sync/push/{partNumber}
POST /api/integrations/odoo/sync/pull/{odooId}
```
### 3.8 `freecad`
No dedicated API endpoints currently. Configures URI scheme and executable path used by the web UI's "Open in Create" links and by CLI operations. Future: client configuration distribution endpoint.
### 3.9 `jobs`
```
# User-facing
GET /api/jobs
GET /api/jobs/{jobID}
GET /api/jobs/{jobID}/logs
POST /api/jobs
POST /api/jobs/{jobID}/cancel
# Job definitions
GET /api/job-definitions
GET /api/job-definitions/{name}
POST /api/job-definitions/reload
# Runner management (admin)
GET /api/runners
POST /api/runners
DELETE /api/runners/{runnerID}
# Runner-facing (runner token auth)
POST /api/runner/heartbeat
POST /api/runner/claim
PUT /api/runner/jobs/{jobID}/progress
POST /api/runner/jobs/{jobID}/complete
POST /api/runner/jobs/{jobID}/fail
POST /api/runner/jobs/{jobID}/log
PUT /api/runner/jobs/{jobID}/dag
```
### 3.10 `dag`
```
GET /api/items/{partNumber}/dag
GET /api/items/{partNumber}/dag/forward-cone/{nodeKey}
GET /api/items/{partNumber}/dag/dirty
PUT /api/items/{partNumber}/dag
POST /api/items/{partNumber}/dag/mark-dirty/{nodeKey}
```
---
## 4. Disabled Module Behavior
When a module is disabled:
1. **API routes** registered by that module return `404 Not Found` with body `{"error": "module '<id>' is not enabled"}`.
2. **Web UI** hides the module's navigation entry, page, and any inline UI elements (e.g., project tags on item cards).
3. **SSE events** from the module are not broadcast.
4. **Background goroutines** (e.g., job timeout sweeper, runner heartbeat checker) are not started.
5. **Database tables** are not dropped — they remain for re-enablement. No data loss on disable/enable cycle.
Implementation: each module's route group is wrapped in a middleware check:
```go
func RequireModule(id string) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !modules.IsEnabled(id) {
http.Error(w, `{"error":"module '`+id+`' is not enabled"}`, 404)
return
}
next.ServeHTTP(w, r)
})
}
}
```
---
## 5. Configuration Persistence
### 5.1 Precedence
```
Environment variables (highest — always wins, secrets live here)
Database overrides (admin UI writes here)
config.yaml (lowest — bootstrap defaults)
```
### 5.2 Database Table
```sql
-- Migration 014_settings.sql
CREATE TABLE settings_overrides (
key TEXT PRIMARY KEY, -- dotted path: "auth.ldap.enabled"
value JSONB NOT NULL, -- typed value
updated_by TEXT NOT NULL, -- username
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE module_state (
module_id TEXT PRIMARY KEY, -- "auth", "projects", etc.
enabled BOOLEAN NOT NULL,
updated_by TEXT NOT NULL,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
```
### 5.3 Load Sequence
On startup:
1. Parse `config.yaml` into Go config struct.
2. Query `settings_overrides` — merge each key into the struct using dotted path resolution.
3. Apply environment variable overrides (existing `SILO_*` vars).
4. Query `module_state` — override default enabled/disabled from YAML.
5. Validate module dependencies.
6. Register only enabled modules' route groups.
7. Start only enabled modules' background goroutines.
### 5.4 Runtime Updates
When an admin saves settings via `PUT /api/admin/settings/{module}`:
1. Validate the payload against the module's config schema.
2. Write changed keys to `settings_overrides`.
3. Update `module_state` if `enabled` changed.
4. Apply changes to the in-memory config (hot reload where safe).
5. Broadcast `settings.changed` SSE event with `{module, enabled, changed_keys}`.
6. For changes that require restart (e.g., `server.port`, `database.*`), return a `restart_required: true` flag in the response. The UI shows a banner.
### 5.5 What Requires Restart
| Config Area | Hot Reload | Restart Required |
|-------------|-----------|------------------|
| Module enable/disable | Yes | No |
| `auth.*` provider toggles | Yes | No |
| `auth.cors.allowed_origins` | Yes | No |
| `odoo.*` connection settings | Yes | No |
| `freecad.*` | Yes | No |
| `jobs.*` timeouts, directory | Yes | No |
| `server.host`, `server.port` | No | Yes |
| `database.*` | No | Yes |
| `storage.*` | No | Yes |
| `schemas.directory` | No | Yes |
---
## 6. Public Module Discovery Endpoint
```
GET /api/modules
```
**No authentication required.** Clients need this pre-login to know whether OIDC is available, whether projects exist, etc.
### 6.1 Response
```json
{
"modules": {
"core": {
"enabled": true,
"required": true,
"name": "Core PDM",
"version": "0.2"
},
"schemas": {
"enabled": true,
"required": true,
"name": "Schemas"
},
"storage": {
"enabled": true,
"required": true,
"name": "Storage"
},
"auth": {
"enabled": true,
"required": false,
"name": "Authentication",
"config": {
"local_enabled": true,
"ldap_enabled": true,
"oidc_enabled": true,
"oidc_issuer_url": "https://keycloak.example.com/realms/silo"
}
},
"projects": {
"enabled": true,
"required": false,
"name": "Projects"
},
"audit": {
"enabled": true,
"required": false,
"name": "Audit"
},
"odoo": {
"enabled": false,
"required": false,
"name": "Odoo ERP"
},
"freecad": {
"enabled": true,
"required": false,
"name": "Create Integration",
"config": {
"uri_scheme": "silo"
}
},
"jobs": {
"enabled": false,
"required": false,
"name": "Job Queue"
},
"dag": {
"enabled": false,
"required": false,
"name": "Dependency DAG",
"depends_on": ["jobs"]
}
},
"server": {
"version": "0.2",
"read_only": false
}
}
```
The `config` sub-object exposes only public, non-secret metadata needed by clients. Never includes passwords, tokens, or secret keys.
---
## 7. Admin Settings Endpoints
### 7.1 Get All Settings
```
GET /api/admin/settings
Authorization: Bearer <admin token>
```
Returns full config grouped by module with secrets redacted:
```json
{
"core": {
"server": {
"host": "0.0.0.0",
"port": 8080,
"base_url": "https://silo.example.com",
"read_only": false
}
},
"schemas": {
"directory": "/etc/silo/schemas",
"default": "kindred-rd"
},
"storage": {
"backend": "filesystem",
"filesystem": {
"root_dir": "/var/lib/silo/data"
},
"status": "connected"
},
"database": {
"host": "postgres",
"port": 5432,
"name": "silo",
"user": "silo",
"password": "****",
"sslmode": "disable",
"max_connections": 10,
"status": "connected"
},
"auth": {
"enabled": true,
"session_secret": "****",
"local": { "enabled": true },
"ldap": {
"enabled": true,
"url": "ldaps://ipa.example.com",
"base_dn": "dc=kindred,dc=internal",
"user_search_dn": "cn=users,cn=accounts,dc=kindred,dc=internal",
"bind_password": "****",
"role_mapping": { "...": "..." }
},
"oidc": {
"enabled": true,
"issuer_url": "https://keycloak.example.com/realms/silo",
"client_id": "silo",
"client_secret": "****",
"redirect_url": "https://silo.example.com/auth/callback"
},
"cors": { "allowed_origins": ["https://silo.example.com"] }
},
"projects": { "enabled": true },
"audit": { "enabled": true },
"odoo": { "enabled": false, "url": "", "database": "", "username": "" },
"freecad": { "uri_scheme": "silo", "executable": "" },
"jobs": {
"enabled": false,
"directory": "/etc/silo/jobdefs",
"runner_timeout": 90,
"job_timeout_check": 30,
"default_priority": 100
},
"dag": { "enabled": false }
}
```
### 7.2 Get Module Settings
```
GET /api/admin/settings/{module}
```
Returns just the module's config block.
### 7.3 Update Module Settings
```
PUT /api/admin/settings/{module}
Content-Type: application/json
{
"enabled": true,
"ldap": {
"enabled": true,
"url": "ldaps://ipa.example.com"
}
}
```
**Response:**
```json
{
"updated": ["auth.ldap.enabled", "auth.ldap.url"],
"restart_required": false
}
```
### 7.4 Test Connectivity
```
POST /api/admin/settings/{module}/test
```
Available for modules with external connections:
| Module | Test Action |
|--------|------------|
| `storage` | Verify filesystem storage directory is accessible |
| `auth` (ldap) | Attempt LDAP bind with configured credentials |
| `auth` (oidc) | Fetch OIDC discovery document from issuer URL |
| `odoo` | Attempt XML-RPC connection to Odoo |
**Response:**
```json
{
"success": true,
"message": "LDAP bind successful",
"latency_ms": 42
}
```
---
## 8. Config YAML Changes
The existing `config.yaml` gains a `modules` section. Existing top-level keys remain for backward compatibility — the module system reads from both locations.
```yaml
# Existing keys (unchanged, still work)
server:
host: "0.0.0.0"
port: 8080
database:
host: postgres
port: 5432
name: silo
user: silo
password: silodev
sslmode: disable
storage:
backend: filesystem
filesystem:
root_dir: /var/lib/silo/data
schemas:
directory: /etc/silo/schemas
auth:
enabled: true
session_secret: change-me
local:
enabled: true
# New: explicit module toggles (optional, defaults shown)
modules:
projects:
enabled: true
audit:
enabled: true
odoo:
enabled: false
freecad:
enabled: true
uri_scheme: silo
jobs:
enabled: false
directory: /etc/silo/jobdefs
runner_timeout: 90
job_timeout_check: 30
default_priority: 100
dag:
enabled: false
```
If a module is not listed under `modules:`, its default enabled state from Section 2.2 applies. The `auth.enabled` field continues to control the `auth` module (no duplication under `modules:`).
---
## 9. SSE Events
```
settings.changed {module, enabled, changed_keys[], updated_by}
```
Broadcast on any admin settings change. The web UI listens for this to:
- Show/hide navigation entries when modules are toggled.
- Display a "Settings updated by another admin" toast.
- Show a "Restart required" banner when flagged.
---
## 10. Web UI — Admin Settings Page
The Settings page (`/settings`) is restructured into sections:
### 10.1 Existing (unchanged)
- **Account** — username, display name, email, auth source, role badge.
- **API Tokens** — create, list, revoke.
### 10.2 New: Module Configuration (admin only)
Visible only to admin users. Each module gets a collapsible card:
```
┌─────────────────────────────────────────────────────┐
│ [toggle] Authentication [status] │
├─────────────────────────────────────────────────────┤
│ │
│ ── Local Auth ──────────────────────────────────── │
│ Enabled: [toggle] │
│ │
│ ── LDAP / FreeIPA ──────────────────────────────── │
│ Enabled: [toggle] │
│ URL: [ldaps://ipa.example.com ] │
│ Base DN: [dc=kindred,dc=internal ] [Test] │
│ │
│ ── OIDC / Keycloak ────────────────────────────── │
│ Enabled: [toggle] │
│ Issuer URL: [https://keycloak.example.com] [Test] │
│ Client ID: [silo ] │
│ │
│ ── CORS ────────────────────────────────────────── │
│ Allowed Origins: [tag input] │
│ │
│ [Save] │
└─────────────────────────────────────────────────────┘
```
Module cards for required modules (`core`, `schemas`, `storage`) show their status and config but have no enable/disable toggle.
Status indicators per module:
| Status | Badge | Meaning |
|--------|-------|---------|
| Active | `green` | Enabled and operational |
| Disabled | `overlay1` | Toggled off |
| Error | `red` | Enabled but connectivity or config issue |
| Setup Required | `yellow` | Enabled but missing required config (e.g., LDAP URL empty) |
### 10.3 Infrastructure Section (admin, read-only)
Shows connection status for required infrastructure:
- **Database** — host, port, name, connection pool usage, status badge.
- **Storage** — endpoint, bucket, SSL, status badge.
These are read-only in the UI (setup-only via YAML/env). The "Test" button is available to verify connectivity.
---
## 11. Implementation Order
1. **Migration 014**`settings_overrides` and `module_state` tables.
2. **Config loader refactor** — YAML → DB merge → env override pipeline.
3. **Module registry** — Go struct defining all modules with metadata, dependencies, defaults.
4. **`GET /api/modules`** — public endpoint, no auth.
5. **`RequireModule` middleware** — gate route groups by module state.
6. **Admin settings API**`GET/PUT /api/admin/settings/{module}`, test endpoints.
7. **Web UI settings page** — module cards with toggles, config forms, test buttons.
8. **SSE integration**`settings.changed` event broadcast.
---
## 12. Future Considerations
- **Module manifest format** — per ROADMAP.md, each module will eventually declare routes, views, hooks, and permissions via a manifest. This spec covers the runtime module registry; the manifest format is TBD.
- **Custom modules** — third-party modules that register against the endpoint registry. Requires the manifest contract and a plugin loading mechanism.
- **Per-module permissions** — beyond the current role hierarchy, modules may define fine-grained scopes (e.g., `jobs:admin`, `dag:write`).
- **Location & Inventory module** — when the Location/Inventory API is implemented (tables already exist), it becomes a new optional module.
- **Notifications module** — per ROADMAP.md Tier 1, notifications/subscriptions will be a dedicated module.
---
## 13. References
- [CONFIGURATION.md](CONFIGURATION.md) — Current config reference
- [ROADMAP.md](ROADMAP.md) — Module manifest, API endpoint registry
- [AUTH.md](AUTH.md) — Authentication architecture
- [WORKERS.md](WORKERS.md) — Job queue system
- [DAG.md](DAG.md) — Dependency DAG specification
- [SPECIFICATION.md](SPECIFICATION.md) — Full endpoint listing

442
docs/ROADMAP.md Normal file
View File

@@ -0,0 +1,442 @@
# Silo Platform Roadmap
**Version:** 2.0
**Date:** February 2026
Silo is the server component of the Kindred ecosystem. Its core function is storing and version-controlling engineering data (parts, assemblies, BOMs). This roadmap describes the expansion of Silo from a PDM server into a modular platform -- comparable to how Gitea/GitHub extend Git hosting with Actions, Wikis, Packages, and webhooks.
For a detailed comparison against SOLIDWORKS PDM, see [GAP_ANALYSIS.md](GAP_ANALYSIS.md).
---
## Guiding Principles
- **Modular architecture.** Every capability beyond core PDM is a module. Modules register against a central API endpoint registry and declare their menu entries, views, dependencies, and routes via a module manifest.
- **Odoo-aligned UX.** The web UI follows Odoo's navigation patterns: a top-level app launcher grid, breadcrumb navigation (`Module > List > Record > Sub-view`), and standard view types (list, form, kanban, calendar, pivot). This alignment provides a familiar experience for shops already using Odoo as their ERP, and a clean integration path for those who adopt it later.
- **Open by default.** Silo and all modules are open-source. Enterprise customers can fork, extend, and self-host. Developer tools for building and distributing custom Create forks are available to everyone, not just Kindred.
- **Odoo as reference ERP.** For shops on Odoo, a bridge module syncs Silo data to Odoo models (`mrp.bom`, `mrp.production`, `quality.check`, etc.). For shops on other ERPs, the open API serves as a documented integration surface. Silo's web UI is fully self-sufficient with no ERP dependency required.
---
## Foundational Contracts
### The .kc File Format
Silo introduces the `.kc` file format as an enhanced superset of FreeCAD's `.fcstd`. Both are ZIP bundles. A `.kc` file contains everything an `.fcstd` does, plus a `silo/` directory with platform metadata.
#### Standard FCStd contents (preserved as-is)
- `Document.xml`, `GuiDocument.xml`
- BREP geometry files (`.brp`)
- `thumbnails/`
#### Added .kc entries
| Path | Purpose |
|------|---------|
| `silo/manifest.json` | Silo instance origin, part UUID, revision hash, .kc schema version |
| `silo/metadata.json` | Custom schema field values, tags, lifecycle state |
| `silo/history.json` | Local revision log (lightweight; full history is server-side) |
| `silo/approvals.json` | ECO/approval state snapshot |
| `silo/dependencies.json` | Assembly link references by Silo UUID (not filepath) |
| `silo/macros/` | Embedded macro references or inline scripts bound to this part |
| `silo/inspection/` | GD&T annotations, tolerance data, CMM linkage metadata |
| `silo/thumbnails/` | Silo-generated renderings (separate from FreeCAD's built-in thumbnail) |
#### Interoperability
- **FCStd -> Silo:** On import, the `silo/` directory is generated with defaults. A UUID is assigned and the user is prompted for schema fields.
- **Silo -> FCStd:** On export, the `silo/` directory is stripped. The remaining contents are a valid `.fcstd`.
- **Round-trip safety:** FreeCAD ignores the `silo/` directory on save, so there is no risk of FreeCAD corrupting Silo metadata.
- **Schema versioning:** `silo/manifest.json` carries a format version for forward-compatible migrations.
### Module Manifest
Each module ships a manifest declaring its integration surface:
```
id, name, version, description
dependencies (other module IDs)
menu_entries (app launcher icon, label, route)
view_declarations (list, form, kanban, etc.)
api_routes (REST endpoints the module registers)
hooks (events the module listens to or emits)
permissions (required roles/scopes)
```
The exact format (JSON, TOML, or Python-based a la Odoo's `__manifest__.py`) is TBD. The contract is: a module is anything that provides a valid manifest and registers against the endpoint registry.
### Web UI Shell
The Silo web application provides the chrome that all modules render within.
- **App launcher:** Top-level grid of installed module icons. Driven by the API endpoint registry -- only enabled modules appear. Disabled modules show greyed with an "Enable" action for discoverability.
- **Breadcrumbs:** Every view follows `Module > List > Record > Sub-view`. Consistent across all modules.
- **View types:** List, form, kanban, calendar, pivot/reporting. Modules declare supported views in their manifest.
- **Schema-driven forms:** The user-customizable schema engine maps directly to form views, enabling end-users to define part metadata fields through the web UI without code changes.
---
## Dependency Tiers
Modules are organized into tiers based on what they depend on. Lower tiers must be stable before higher tiers are built.
### Tier 0 -- Foundation
Everything depends on these. They define what Silo *is*.
| Component | Description | Status |
|-----------|-------------|--------|
| **Core Silo** | Part/assembly storage, version control, auth, base REST API | Complete |
| **.kc Format Spec** | File format contract between Create and Silo | Complete |
| **API Endpoint Registry** | Module discovery, dynamic UI rendering, health checks | Not Started |
| **Web UI Shell** | App launcher, breadcrumbs, view framework, module rendering | Partial |
| **Python Scripting Engine** | Server-side hook execution, module extension point | Not Started |
| **Job Queue Infrastructure** | Redis/NATS shared async service for all compute modules | Not Started |
### Tier 1 -- Core Services
Broad downstream dependencies. These should be built early because retrofitting is painful.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Headless Create** | API-driven FreeCAD instance for file manipulation, geometry queries, format conversion, rendering | Core Silo, Job Queue | Not Started |
| **Notifications & Subscriptions** | Per-part watch lists, lifecycle event hooks, webhook delivery | Core Silo, Registry | Not Started |
| **Audit Trail / Compliance** | ITAR, ISO 9001, AS9100 traceability; module-level event journaling | Core Silo | Partial |
### Tier 2 -- File Intelligence & Collaboration
High-visibility features. Mostly low-hanging fruit once Tier 1 is solid.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Intelligent FCStd Diffing** | XML-based structural diff of .kc bundles | Headless Create | Not Started |
| **Thumbnail Generation** | Auto-rendered part/assembly previews | Headless Create | Not Started |
| **Macro Store** | Shared macro library across Create instances | Core Silo, Registry | Not Started |
| **Theme & Addon Manager** | Centralized distribution of UI themes and workbench addons | Core Silo, Registry | Not Started |
| **User-Customizable Schemas** | End-user defined part/form metadata via web UI | Core Silo, Scripting Engine | Not Started |
### Tier 3 -- Compute
Heavy async workloads. All route through the shared job queue.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Batch Jobs (CPU/GPU)** | FEA, CFD, rendering, bulk export | Job Queue, Headless Create | Not Started |
| **AI Broker** | LLM tasks (Ollama), GNN constraint optimization, appearance AI | Job Queue | Not Started |
| **Reporting & Analytics** | Part reuse, revision frequency, compute usage dashboards, cost roll-ups | Audit Trail, Core Silo | Not Started |
### Tier 4 -- Engineering Workflow
Process modules that formalize how engineering work moves through an organization.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Approval / ECO Workflow** | Engineering change orders, multi-stage review gates, digital signatures | Notifications, Audit Trail, Schemas | Not Started |
| **Shop Floor Drawing Distribution** | Controlled push-to-production drawings; web-based appliance displays on the floor | Headless Create, Approval Workflow | Not Started |
| **Import/Export Bridge** | STEP, IGES, 3MF connectors; SOLIDWORKS migration tooling; ERP adapters | Headless Create | Not Started |
| **Multi-tenant / Org Management** | Org boundaries, role-based permissioning, storage quotas | Core Auth, Audit Trail | Not Started |
### Tier 5 -- Manufacturing & Quality
Deep domain modules. Heavy spec work required independent of software dependencies.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **MES Module** | Manufacturing execution -- internal module or bridge to external MES | Approval Workflow, Schemas, Shop Floor Drawings | Not Started |
| **Quality / Tolerance Stackup** | Inspection data ingestion, CMM device linking, statistical tolerance analysis, material mapping | Schemas, Import Bridge | Not Started |
| **Inspection Plan Generator** | Auto-generate CMM programs or inspection checklists from GD&T drawings | Headless Create, Quality Module | Not Started |
| **BIM Inventory / Receiving** | Live facility model with real-time inventory location, explorable in a custom BIM-MES workbench in Create | Custom BIM-MES Workbench, Schemas, Notifications | Not Started |
### Tier 6 -- Platform & Ecosystem
Modules that serve the broader community and long-horizon use cases.
| Module | Description | Depends On | Status |
|--------|-------------|------------|--------|
| **Developer Tools** | Managed Gitea instance for in-house Create fork development; CI/CD to build and distribute fork updates to configured clients | Tier 0-1 stability | Not Started |
| **Digital Twin Sync** | Live sensor data mapped onto BIM/assembly models; operational monitoring | BIM Inventory, Reporting | Not Started |
| **ERP Adapters (Odoo, SAP, etc.)** | Bidirectional sync of parts, BOMs, ECOs, production orders to external ERP | Import/Export Bridge, MES, Schemas | Partial (Odoo stubs) |
---
## Near-Term Priorities
These are the concrete tasks that map to Tier 0 completion and the first steps into Tier 1. They replace the older Phase 1-6 calendar-based timelines.
### Tier 0 Completion
Complete MVP and stabilize core functionality.
| Task | Description | Status |
|------|-------------|--------|
| Unit test suite | Core API, database, partnum, file, CSV/ODS handler tests | Partial (~40%) |
| Date segment type | Implement `date` segment with strftime-style formatting | Not Started |
| Part number validation | Validate format against schema on creation | Not Started |
| Location CRUD API | Expose location hierarchy via REST | Not Started |
| Inventory API | Expose inventory operations via REST | Not Started |
**Success metrics:**
- All existing tests pass
- File upload/download works end-to-end
- FreeCAD users can checkout, modify, commit parts
### Multi-User Enablement
Enable team collaboration (feeds into Tier 1 and Tier 4).
| Task | Description | Status |
|------|-------------|--------|
| Check-out locking | Pessimistic locks with timeout | Not Started |
| User/group management | Create, assign, manage users and groups | Not Started |
| Folder permissions | Read/write/delete per folder hierarchy | Not Started |
**Success metrics:**
- 5+ concurrent users supported
- No data corruption under concurrent access
- Audit log captures all modifications
### Workflow Engine
Implement engineering change processes (Tier 4: Approval/ECO Workflow).
| Task | Description | Status |
|------|-------------|--------|
| Workflow designer | YAML-defined state machines | Not Started |
| State transitions | Configurable transition rules with permissions | Not Started |
| Approval workflows | Single and parallel approver gates | Not Started |
| Email notifications | SMTP integration for alerts on state changes | Not Started |
**Success metrics:**
- Engineering change process completable in Silo
- Email notifications delivered reliably
- Workflow state visible in web UI
### Search & Discovery
Improve findability and navigation (Tier 0 Web UI Shell).
| Task | Description | Status |
|------|-------------|--------|
| Advanced search UI | Web interface with filters and operators | Not Started |
| Saved searches | User-defined query favorites | Not Started |
**Success metrics:**
- Search returns results in <2 seconds
- Where-used queries complete in <5 seconds
---
## Gap Summary
For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_ANALYSIS.md#appendix-c-solidworks-pdm-comparison).
### Completed (Previously Critical/High)
1. ~~User authentication~~ -- local, LDAP, OIDC
2. ~~Role-based permissions~~ -- 3-tier role model (admin/editor/viewer)
3. ~~Audit trail~~ -- audit_log table with completeness scoring
4. ~~Where-used search~~ -- reverse parent lookup API
5. ~~Multi-level BOM API~~ -- recursive expansion with configurable depth
6. ~~BOM export~~ -- CSV and ODS formats
### Critical Gaps (Required for Team Use)
1. **Workflow engine** -- state machines with transitions and approvals
2. **Check-out locking** -- pessimistic locking for CAD files
### High Priority Gaps (Significant Value)
1. **Email notifications** -- alert users on state changes
2. **Web UI search** -- advanced search interface with saved searches
3. **Folder/state permissions** -- granular access control beyond role model
### Medium Priority Gaps (Nice to Have)
1. **Saved searches** -- frequently used queries
2. **File preview/thumbnails** -- visual browsing
3. **Reporting** -- activity and inventory reports
4. **Scheduled tasks** -- background automation
5. **BOM comparison** -- revision diff for assemblies
---
## Priority Notes
- **Headless Create** is the single highest-leverage Tier 1 item. It unblocks diffing, thumbnails, batch export, drawing distribution, and inspection plan generation.
- **Audit Trail** is unglamorous but critical to build early. Retrofitting compliance logging after modules ship is expensive and error-prone.
- **Tier 2** delivers visible, demo-able value quickly -- diffing, thumbnails, and the macro store are features users immediately understand.
- **Tiers 5-6** carry heavy domain complexity. They need detailed specification and industry consultation well before implementation begins.
- The **.kc format** and **module manifest** are the two foundational contracts. Getting these right determines how cleanly everything above them composes.
---
## Open Questions
1. **Module manifest format** -- JSON, TOML, or Python-based? Tradeoffs between simplicity and expressiveness.
2. **.kc thumbnail policy** -- Single canonical thumbnail vs. multi-view renders. Impacts file size and generation cost.
3. **Job queue technology** -- Redis Streams vs. NATS. Redis is already in the stack; NATS offers better pub/sub semantics for event-driven modules.
4. **Headless Create deployment** -- Sidecar container per Silo instance, or pool of workers behind the job queue?
5. **BIM-MES workbench scope** -- How much of FreeCAD BIM is reusable vs. needs to be purpose-built for inventory/facility modeling?
6. **Offline .kc workflow** -- How much of the `silo/` metadata is authoritative when disconnected? Reconciliation strategy on reconnect.
---
## Appendix A: Current Project Inventory
### Implemented Features (MVP Complete)
#### Core Database System
- PostgreSQL schema with 13 migrations
- UUID-based identifiers throughout
- Soft delete support via `archived_at` timestamps
- Atomic sequence generation for part numbers
#### Part Number Generation
- YAML schema parser with validation
- Segment types: `string`, `enum`, `serial`, `constant`
- Scope templates for serial counters (e.g., `{category}`, `{project}`)
- Format templates for custom output
#### Item Management
- Full CRUD operations for items
- Item types: part, assembly, drawing, document, tooling, purchased, electrical, software
- Custom properties via JSONB storage
- Project tagging with many-to-many relationships
#### Revision Control
- Append-only revision history
- Revision metadata: properties, file reference, checksum, comment
- Status tracking: draft, review, released, obsolete
- Labels/tags per revision
- Revision comparison (diff)
- Rollback functionality
#### File Management
- Filesystem-based file storage
- File upload/download via REST API
- SHA256 checksums for integrity
- Storage path: `items/{partNumber}/rev{N}.FCStd`
#### Bill of Materials (BOM)
- Relationship types: component, alternate, reference
- Multi-level BOM (recursive expansion with configurable depth)
- Where-used queries (reverse parent lookup)
- BOM CSV and ODS export/import with cycle detection
- Reference designators for electronics
- Quantity tracking with units
- Revision-specific child linking
#### Project Management
- Project CRUD operations
- Unique project codes (2-10 characters)
- Item-to-project tagging
- Project-filtered queries
#### Data Import/Export
- CSV export with configurable properties
- CSV import with dry-run validation
- ODS spreadsheet import/export (items, BOMs, project sheets)
- Template generation for import formatting
#### API & Web Interface
- REST API with 78 endpoints
- Authentication: local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak
- Role-based access control (admin > editor > viewer)
- API token management (SHA-256 hashed)
- Session management (PostgreSQL-backed, 24h lifetime)
- CSRF protection (nosurf on web forms)
- Middleware: logging, CORS, recovery, request ID
- Web UI -- React SPA (Vite + TypeScript, Catppuccin Mocha theme)
- Fuzzy search
- Health and readiness probes
#### Audit & Completeness
- Audit logging (database table with user/action/resource tracking)
- Item completeness scoring with weighted fields
- Category-specific property validation
- Tier classification (critical/low/partial/good/complete)
#### Configuration
- YAML configuration with environment variable overrides
- Multi-schema support
- Docker Compose deployment ready
### Partially Implemented
| Feature | Status | Notes |
|---------|--------|-------|
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull sync operations are stubs |
| Date segment type | Not started | Schema parser placeholder exists |
| Part number validation | Not started | API accepts but doesn't validate format |
| Location hierarchy CRUD | Schema only | Tables exist, no API endpoints |
| Inventory tracking | Schema only | Tables exist, no API endpoints |
| Unit tests | Partial | 11 Go test files across api, db, ods, partnum, schema packages |
---
## Appendix B: Phase 1 Detailed Tasks
### 1.1 File Storage -- COMPLETE
- [x] Filesystem storage backend
- [x] File upload via REST API
- [x] File download via REST API (latest and by revision)
- [x] SHA256 checksums on upload
### 1.2 Authentication & Authorization -- COMPLETE
- [x] Local authentication (bcrypt)
- [x] LDAP/FreeIPA authentication
- [x] OIDC/Keycloak authentication
- [x] Role-based access control (admin/editor/viewer)
- [x] API token management (SHA-256 hashed)
- [x] Session management (PostgreSQL-backed)
- [x] CSRF protection (nosurf)
- [x] Audit logging (database table)
### 1.3 Multi-level BOM & Export -- COMPLETE
- [x] Recursive BOM expansion with configurable depth
- [x] Where-used reverse lookup
- [x] BOM CSV export/import with cycle detection
- [x] BOM ODS export
- [x] ODS item export/import/template
### 1.4 Unit Test Suite
- [ ] Database connection and transaction tests
- [ ] Item CRUD operation tests
- [ ] Revision creation and retrieval tests
- [ ] Part number generation tests
- [ ] File upload/download tests
- [ ] CSV import/export tests
- [ ] API endpoint tests
### 1.5 Missing Segment Types
- [ ] Implement date segment type
- [ ] Add strftime-style format support
### 1.6 Location & Inventory APIs
- [ ] `GET /api/locations` - List locations
- [ ] `POST /api/locations` - Create location
- [ ] `GET /api/locations/{path}` - Get location
- [ ] `DELETE /api/locations/{path}` - Delete location
- [ ] `GET /api/inventory/{partNumber}` - Get inventory
- [ ] `POST /api/inventory/{partNumber}/adjust` - Adjust quantity
- [ ] `POST /api/inventory/{partNumber}/move` - Move between locations
---
## Appendix C: References
### SOLIDWORKS PDM Documentation
- [SOLIDWORKS PDM Product Page](https://www.solidworks.com/product/solidworks-pdm)
- [What's New in SOLIDWORKS PDM 2025](https://blogs.solidworks.com/solidworksblog/2024/10/whats-new-in-solidworks-pdm-2025.html)
- [Top 5 Enhancements in SOLIDWORKS PDM 2024](https://blogs.solidworks.com/solidworksblog/2023/10/top-5-enhancements-in-solidworks-pdm-2024.html)
- [SOLIDWORKS PDM Workflow Transitions](https://help.solidworks.com/2023/english/EnterprisePDM/Admin/c_workflow_transition.htm)
- [Ultimate Guide to SOLIDWORKS PDM Permissions](https://www.goengineer.com/blog/ultimate-guide-to-solidworks-pdm-permissions)
- [Searching in SOLIDWORKS PDM](https://help.solidworks.com/2021/english/EnterprisePDM/fileexplorer/c_searches.htm)
- [SOLIDWORKS PDM API Getting Started](https://3dswym.3dexperience.3ds.com/wiki/solidworks-news-info/getting-started-with-the-solidworks-pdm-api-solidpractices_gBCYaM75RgORBcpSO1m_Mw)
### Silo Documentation
- [Specification](SPECIFICATION.md)
- [Development Status](STATUS.md)
- [Deployment Guide](DEPLOYMENT.md)
- [Gap Analysis](GAP_ANALYSIS.md)

View File

@@ -37,7 +37,7 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb
┌─────────────────────────────────────────────────────────────┐
│ Silo Server (silod) │
│ - REST API (75 endpoints) │
│ - REST API (86 endpoints) │
│ - Authentication (local, LDAP, OIDC) │
│ - Schema parsing and validation │
│ - Part number generation engine │
@@ -49,9 +49,9 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────────┐
│ PostgreSQL │ │ MinIO
│ (psql.kindred.internal)│ │ - File storage │
│ - Item metadata │ │ - Versioned objects
│ PostgreSQL │ │ Local Filesystem
│ (psql.example.internal)│ │ - File storage │
│ - Item metadata │ │ - Revision files
│ - Relationships │ │ - Thumbnails │
│ - Revision history │ │ │
│ - Auth / Sessions │ │ │
@@ -63,8 +63,8 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb
| Component | Technology | Notes |
|-----------|------------|-------|
| Database | PostgreSQL 16 | Existing instance at psql.kindred.internal |
| File Storage | MinIO | S3-compatible, versioning enabled |
| Database | PostgreSQL 16 | Existing instance at psql.example.internal |
| File Storage | Local filesystem | Files stored under configurable root directory |
| CLI & API Server | Go (1.24) | chi/v5 router, pgx/v5 driver, zerolog |
| Authentication | Multi-backend | Local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak |
| Sessions | PostgreSQL pgxstore | alexedwards/scs, 24h lifetime |
@@ -83,7 +83,7 @@ An **item** is the fundamental entity. Items have:
- **Properties** (key-value pairs, schema-defined and custom)
- **Relationships** to other items
- **Revisions** (append-only history)
- **Files** (optional, stored in MinIO)
- **Files** (optional, stored on the local filesystem)
- **Location** (optional physical inventory location)
### 3.2 Database Schema (Conceptual)
@@ -115,7 +115,7 @@ CREATE TABLE revisions (
item_id UUID REFERENCES items(id) NOT NULL,
revision_number INTEGER NOT NULL,
properties JSONB NOT NULL, -- all properties at this revision
file_version TEXT, -- MinIO version ID if applicable
file_version TEXT, -- storage version ID if applicable
created_at TIMESTAMPTZ DEFAULT now(),
created_by TEXT, -- user identifier (future: LDAP DN)
comment TEXT,
@@ -345,7 +345,7 @@ CAD workbench and spreadsheet extension implementations are maintained in separa
### 5.1 File Storage Strategy
Files are stored as whole objects in MinIO with versioning enabled. Storage path convention: `items/{partNumber}/rev{N}.ext`. SHA-256 checksums are captured on upload for integrity verification.
Files are stored on the local filesystem under a configurable root directory. Storage path convention: `items/{partNumber}/rev{N}.ext`. SHA-256 checksums are captured on upload for integrity verification.
Future option: exploded storage (unpack ZIP-based CAD archives for better diffing).
@@ -439,7 +439,7 @@ Revisions are created explicitly by user action (not automatic):
### 7.3 Revision vs. File Version
- **Revision**: Silo metadata revision (tracked in PostgreSQL)
- **File Version**: MinIO object version (automatic on upload)
- **File Version**: File on disk corresponding to a revision
A single Silo revision may span multiple file uploads during editing. Only committed revisions create formal revision records.
@@ -598,12 +598,12 @@ See [AUTH.md](AUTH.md) for full architecture details and [AUTH_USER_GUIDE.md](AU
## 11. API Design
### 11.1 REST Endpoints (75 Implemented)
### 11.1 REST Endpoints (86 Implemented)
```
# Health (no auth)
GET /health # Basic health check
GET /ready # Readiness (DB + MinIO)
GET /ready # Readiness (DB)
# Auth (no auth required)
GET /login # Login page
@@ -615,19 +615,22 @@ GET /auth/callback # OIDC callback
# Public API (no auth required)
GET /api/auth/config # Auth backend configuration (for login UI)
# Server-Sent Events (require auth)
GET /api/events # SSE stream for real-time updates
# Auth API (require auth)
GET /api/auth/me # Current authenticated user
GET /api/auth/tokens # List user's API tokens
POST /api/auth/tokens # Create API token
DELETE /api/auth/tokens/{id} # Revoke API token
# Presigned Uploads (editor)
POST /api/uploads/presign # Get presigned MinIO upload URL [editor]
# Direct Uploads (editor)
POST /api/uploads/presign # Get upload URL [editor]
# Schemas (read: viewer, write: editor)
GET /api/schemas # List all schemas
GET /api/schemas/{name} # Get schema details
GET /api/schemas/{name}/properties # Get property schema for category
GET /api/schemas/{name}/form # Get form descriptor (field groups, widgets, category picker)
POST /api/schemas/{name}/segments/{segment}/values # Add enum value [editor]
PUT /api/schemas/{name}/segments/{segment}/values/{code} # Update enum value [editor]
DELETE /api/schemas/{name}/segments/{segment}/values/{code} # Delete enum value [editor]
@@ -644,6 +647,7 @@ DELETE /api/projects/{code} # Delete project [ed
# Items (read: viewer, write: editor)
GET /api/items # List/filter items
GET /api/items/search # Fuzzy search
GET /api/items/by-uuid/{uuid} # Get item by UUID
GET /api/items/export.csv # Export items to CSV
GET /api/items/template.csv # CSV import template
GET /api/items/export.ods # Export items to ODS
@@ -689,9 +693,24 @@ GET /api/items/{partNumber}/bom/export.csv # Export BOM as CSV
GET /api/items/{partNumber}/bom/export.ods # Export BOM as ODS
POST /api/items/{partNumber}/bom # Add BOM entry [editor]
POST /api/items/{partNumber}/bom/import # Import BOM from CSV [editor]
POST /api/items/{partNumber}/bom/merge # Merge BOM from ODS with conflict resolution [editor]
PUT /api/items/{partNumber}/bom/{childPartNumber} # Update BOM entry [editor]
DELETE /api/items/{partNumber}/bom/{childPartNumber} # Remove BOM entry [editor]
# .kc Metadata (read: viewer, write: editor)
GET /api/items/{partNumber}/metadata # Get indexed .kc metadata
PUT /api/items/{partNumber}/metadata # Update metadata fields [editor]
PATCH /api/items/{partNumber}/metadata/lifecycle # Transition lifecycle state [editor]
PATCH /api/items/{partNumber}/metadata/tags # Add/remove tags [editor]
# .kc Dependencies (viewer)
GET /api/items/{partNumber}/dependencies # List raw dependencies
GET /api/items/{partNumber}/dependencies/resolve # Resolve UUIDs to part numbers + file availability
# .kc Macros (viewer)
GET /api/items/{partNumber}/macros # List registered macros
GET /api/items/{partNumber}/macros/{filename} # Get macro source content
# Audit (viewer)
GET /api/audit/completeness # Item completeness scores
GET /api/audit/completeness/{partNumber} # Item detail breakdown
@@ -730,16 +749,149 @@ POST /api/inventory/{partNumber}/move
---
## 11.3 .kc File Integration
Silo supports the `.kc` file format — a ZIP archive that is a superset of FreeCAD's `.fcstd`. A `.kc` file contains everything an `.fcstd` does, plus a `silo/` directory with platform metadata.
#### Standard entries (preserved as-is)
`Document.xml`, `GuiDocument.xml`, BREP geometry files (`.brp`), `thumbnails/`
#### Silo entries (`silo/` directory)
| Path | Purpose |
|------|---------|
| `silo/manifest.json` | Instance origin, part UUID, revision hash, `.kc` schema version |
| `silo/metadata.json` | Custom schema field values, tags, lifecycle state |
| `silo/history.json` | Local revision log (server-generated on checkout) |
| `silo/dependencies.json` | Assembly link references by Silo UUID |
| `silo/macros/*.py` | Embedded macro scripts bound to this part |
#### Commit-time extraction
When a `.kc` file is uploaded via `POST /api/items/{partNumber}/file`, the server:
1. Opens the ZIP and scans for `silo/` entries
2. Parses `silo/manifest.json` and validates the UUID matches the item
3. Upserts `silo/metadata.json` fields into the `item_metadata` table
4. Replaces `silo/dependencies.json` entries in the `item_dependencies` table
5. Replaces `silo/macros/*.py` entries in the `item_macros` table
6. Broadcasts SSE events: `metadata.updated`, `dependencies.changed`, `macros.changed`
Extraction is best-effort — failures are logged as warnings but do not block the upload.
#### Checkout-time packing
When a `.kc` file is downloaded via `GET /api/items/{partNumber}/file/{revision}`, the server repacks the `silo/` directory with current database state:
- `silo/manifest.json` — current item UUID and metadata freshness
- `silo/metadata.json` — latest schema fields, tags, lifecycle state
- `silo/history.json` — last 20 revisions from the database
- `silo/dependencies.json` — current dependency list from `item_dependencies`
Non-silo ZIP entries are passed through unchanged. If the file is a plain `.fcstd` (no `silo/` directory), it is served as-is.
ETag caching: the server computes an ETag from `revision_number:metadata.updated_at` and returns `304 Not Modified` when the client's `If-None-Match` header matches.
#### Lifecycle state machine
The `lifecycle_state` field in `item_metadata` follows this state machine:
```
draft → review → released → obsolete
↑ ↓
└────────┘
```
Valid transitions are enforced by `PATCH /metadata/lifecycle`. Invalid transitions return `422 Unprocessable Entity`.
#### Metadata response shape
```json
{
"schema_name": "kindred-rd",
"lifecycle_state": "draft",
"tags": ["prototype", "v2"],
"fields": {"material": "AL6061", "finish": "anodized"},
"manifest": {
"uuid": "550e8400-e29b-41d4-a716-446655440000",
"silo_instance": "silo.example.com",
"revision_hash": "abc123",
"kc_version": "1.0"
},
"updated_at": "2026-02-18T12:00:00Z",
"updated_by": "forbes"
}
```
#### Dependency response shape
```json
[
{
"uuid": "550e8400-...",
"part_number": "F01-0042",
"revision": 3,
"quantity": 4.0,
"label": "M5 Bolt",
"relationship": "component"
}
]
```
#### Resolved dependency response shape
```json
[
{
"uuid": "550e8400-...",
"part_number": "F01-0042",
"label": "M5 Bolt",
"revision": 3,
"quantity": 4.0,
"resolved": true,
"file_available": true
}
]
```
#### Macro list response shape
```json
[
{"filename": "validate_dims.py", "trigger": "manual", "revision_number": 5}
]
```
#### Macro detail response shape
```json
{
"filename": "validate_dims.py",
"trigger": "manual",
"content": "import FreeCAD\n...",
"revision_number": 5
}
```
#### Database tables (migration 018)
- `item_metadata` — schema fields, lifecycle state, tags, manifest info
- `item_dependencies` — parent/child UUID references with quantity and relationship type
- `item_macros` — filename, trigger type, source content, indexed per item
---
## 12. MVP Scope
### 12.1 Implemented
- [x] PostgreSQL database schema (11 migrations)
- [x] PostgreSQL database schema (13 migrations)
- [x] YAML schema parser for part numbering
- [x] Part number generation engine
- [x] CLI tool (`cmd/silo`)
- [x] API server (`cmd/silod`) with 75 endpoints
- [x] MinIO integration for file storage with versioning
- [x] API server (`cmd/silod`) with 86 endpoints
- [x] Filesystem-based file storage
- [x] BOM relationships (component, alternate, reference)
- [x] Multi-level BOM (recursive expansion with configurable depth)
- [x] Where-used queries (reverse parent lookup)
@@ -760,6 +912,12 @@ POST /api/inventory/{partNumber}/move
- [x] Audit logging and completeness scoring
- [x] CSRF protection (nosurf)
- [x] Fuzzy search
- [x] .kc file extraction pipeline (metadata, dependencies, macros indexed on commit)
- [x] .kc file packing on checkout (manifest, metadata, history, dependencies)
- [x] .kc metadata API (get, update fields, lifecycle transitions, tags)
- [x] .kc dependency API (list, resolve with file availability)
- [x] .kc macro API (list, get source content)
- [x] ETag caching for .kc file downloads
- [x] Property schema versioning framework
- [x] Docker Compose deployment (dev and prod)
- [x] systemd service and deployment scripts

View File

@@ -10,12 +10,12 @@
| Component | Status | Notes |
|-----------|--------|-------|
| PostgreSQL schema | Complete | 11 migrations applied |
| PostgreSQL schema | Complete | 18 migrations applied |
| YAML schema parser | Complete | Supports enum, serial, constant, string segments |
| Part number generator | Complete | Scoped sequences, category-based format |
| API server (`silod`) | Complete | 75 REST endpoints via chi/v5 |
| API server (`silod`) | Complete | 86 REST endpoints via chi/v5 |
| CLI tool (`silo`) | Complete | Item registration and management |
| MinIO file storage | Complete | Upload, download, versioning, checksums |
| Filesystem file storage | Complete | Upload, download, checksums |
| Revision control | Complete | Append-only history, rollback, comparison, status/labels |
| Project management | Complete | CRUD, many-to-many item tagging |
| CSV import/export | Complete | Dry-run validation, template generation |
@@ -29,7 +29,12 @@
| CSRF protection | Complete | nosurf on web forms |
| Fuzzy search | Complete | sahilm/fuzzy library |
| Web UI | Complete | React SPA (Vite + TypeScript), 6 pages, Catppuccin Mocha theme |
| File attachments | Complete | Presigned uploads, item file association, thumbnails |
| File attachments | Complete | Direct uploads, item file association, thumbnails |
| .kc extraction pipeline | Complete | Metadata, dependencies, macros indexed on commit |
| .kc checkout packing | Complete | Manifest, metadata, history, dependencies repacked on download |
| .kc metadata API | Complete | GET/PUT metadata, lifecycle transitions, tag management |
| .kc dependency API | Complete | List raw deps, resolve UUIDs to part numbers + file availability |
| .kc macro API | Complete | List macros, get source content by filename |
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull are stubs |
| Docker Compose | Complete | Dev and production configurations |
| Deployment scripts | Complete | setup-host, deploy, init-db, setup-ipa-nginx |
@@ -55,8 +60,8 @@ FreeCAD workbench and LibreOffice Calc extension are maintained in separate repo
| Service | Host | Status |
|---------|------|--------|
| PostgreSQL | psql.kindred.internal:5432 | Running |
| MinIO | localhost:9000 (API) / :9001 (console) | Configured |
| PostgreSQL | psql.example.internal:5432 | Running |
| File Storage | /opt/silo/data (filesystem) | Configured |
| Silo API | localhost:8080 | Builds successfully |
---
@@ -92,5 +97,12 @@ The schema defines 170 category codes across 10 groups:
| 007_revision_status.sql | Revision status and labels |
| 008_odoo_integration.sql | Odoo ERP integration tables (integrations, sync_log) |
| 009_auth.sql | Authentication system (users, api_tokens, sessions, audit_log, user tracking columns) |
| 010_item_extended_fields.sql | Extended item fields (sourcing_type, sourcing_link, standard_cost, long_description) |
| 010_item_extended_fields.sql | Extended item fields (sourcing_type, long_description) |
| 011_item_files.sql | Item file attachments (item_files table, thumbnail_key column) |
| 012_bom_source.sql | BOM entry source tracking |
| 013_move_cost_sourcing_to_props.sql | Move sourcing_link and standard_cost from item columns to revision properties |
| 014_settings.sql | Settings overrides and module state tables |
| 015_jobs.sql | Job queue, runner, and job log tables |
| 016_dag.sql | Dependency DAG nodes and edges |
| 017_locations.sql | Location hierarchy and inventory tracking |
| 018_kc_metadata.sql | .kc metadata tables (item_metadata, item_dependencies, item_macros, item_approvals, approval_signatures) |

515
docs/STYLE.md Normal file
View File

@@ -0,0 +1,515 @@
# Silo Style Guide
> Living reference for the Silo web UI. All modules must follow these conventions to maintain visual consistency across the platform.
---
## Color System
Silo uses the [Catppuccin Mocha](https://github.com/catppuccin/catppuccin) palette exclusively. All colors are referenced via CSS custom properties defined at `:root`.
### Palette
```
--ctp-rosewater: #f5e0dc
--ctp-flamingo: #f2cdcd
--ctp-pink: #f5c2e7
--ctp-mauve: #cba6f7
--ctp-red: #f38ba8
--ctp-maroon: #eba0ac
--ctp-peach: #fab387
--ctp-yellow: #f9e2af
--ctp-green: #a6e3a1
--ctp-teal: #94e2d5
--ctp-sky: #89dceb
--ctp-sapphire: #74c7ec
--ctp-blue: #89b4fa
--ctp-lavender: #b4befe
--ctp-text: #cdd6f4
--ctp-subtext1: #bac2de
--ctp-subtext0: #a6adc8
--ctp-overlay2: #9399b2
--ctp-overlay1: #7f849c
--ctp-overlay0: #6c7086
--ctp-surface2: #585b70
--ctp-surface1: #45475a
--ctp-surface0: #313244
--ctp-base: #1e1e2e
--ctp-mantle: #181825
--ctp-crust: #11111b
```
### Semantic Roles
| Role | Token | Usage |
|------|-------|-------|
| Page background | `--ctp-base` | Main content area |
| Panel background | `--ctp-mantle` | Sidebars, detail panes, headers |
| Inset/input background | `--ctp-crust` | Form inputs, code blocks, drop zones |
| Primary accent | `--ctp-mauve` | Primary buttons, active states, links, selection highlights |
| Secondary accent | `--ctp-blue` | Informational highlights, secondary actions |
| Success | `--ctp-green` | Confirmations, positive status |
| Warning | `--ctp-yellow` | Caution states, pending actions |
| Danger | `--ctp-red` | Destructive actions, errors, required indicators |
| Informational | `--ctp-teal` | Auto-generated metadata, system-assigned values |
| Body text | `--ctp-text` | Primary content |
| Secondary text | `--ctp-subtext1` | Descriptions, timestamps |
| Muted text | `--ctp-overlay1` | Placeholders, disabled states |
| Borders | `--ctp-surface0` | Dividers, panel edges |
| Hover borders | `--ctp-surface1` | Interactive element borders, row separators |
| Focus ring | `rgba(203, 166, 247, 0.25)` | `box-shadow` on focused inputs (mauve at 25%) |
### Accent Usage for Data Types
| Data type | Color | Token |
|-----------|-------|-------|
| Assembly | `--ctp-mauve` | Badge, icon tint |
| Part | `--ctp-green` | Badge, icon tint |
| Document | `--ctp-blue` | Badge, icon tint |
| Purchased | `--ctp-peach` | Badge, icon tint |
| Phantom | `--ctp-overlay1` | Badge, icon tint |
These mappings are used anywhere item types appear: list badges, detail pane headers, BOM entries, tree views.
---
## Typography
### Scale
| Role | Size | Weight | Token/Color | Transform |
|------|------|--------|-------------|-----------|
| Page title | 1.1rem | 600 | `--ctp-text` | None |
| Section header | 11px | 600 | `--ctp-overlay0` | Uppercase, `letter-spacing: 0.06em` |
| Form label | 11px | 600 | `--ctp-overlay1` | Uppercase, `letter-spacing: 0.05em` |
| Body text | 13px | 400 | `--ctp-text` | None |
| Table cell | 12px | 400 | `--ctp-text` | None |
| Caption / metadata | 11px | 400 | `--ctp-subtext0` | None |
| Badge text | 10px | 600 | Varies | Uppercase |
| Breadcrumb segment | 13px | 500 | `--ctp-subtext1` | None |
| Breadcrumb active | 13px | 600 | `--ctp-text` | None |
### Font Stack
```css
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
```
No external font dependencies. System fonts ensure fast rendering and native feel across platforms.
### Rules
- Never use font sizes below 10px.
- Use `font-weight: 600` for emphasis instead of bold (700). Reserve 700 for page titles only when extra weight is needed.
- `text-transform: uppercase` is reserved for section headers, form labels, and badges. Never uppercase body text or descriptions.
---
## Spacing
Base unit: **4px**. All spacing values are multiples of 4.
| Token | Value | Usage |
|-------|-------|-------|
| `xs` | 4px (0.25rem) | Tight gaps: icon-to-label, tag internal padding |
| `sm` | 8px (0.5rem) | Compact spacing: between related fields, badge padding |
| `md` | 12px (0.75rem) | Standard: form group gaps, sidebar section padding |
| `lg` | 16px (1rem) | Section separation, card padding |
| `xl` | 24px (1.5rem) | Page-level padding, major section breaks |
| `2xl` | 32px (2rem) | Page horizontal padding |
### Application
- **Page padding:** `1.5rem 2rem` (24px vertical, 32px horizontal)
- **Sidebar section padding:** `1rem 1.25rem`
- **Form grid gap:** `1.25rem 1.5rem` (row gap × column gap)
- **Table row height:** 36px minimum (padding included)
- **Table cell padding:** `0.4rem 0.75rem`
---
## Layout
### Page Structure
Every module page follows the same shell:
```
┌─────────────────────────────────────────────────┐
│ Top Nav (52px) │
├──────────┬──────────────────────────────────────┤
│ App Menu │ Page Header (58px) │
│ (icons) ├──────────────────────┬───────────────┤
│ │ Content Area │ Detail Pane │
│ │ │ (360px) │
│ │ │ │
│ │ │ │
└──────────┴──────────────────────┴───────────────┘
```
- **Top nav:** `52px` height, `--ctp-mantle` background, `1px solid --ctp-surface0` bottom border.
- **App menu sidebar:** Icon strip on the left. Module icons, tooltips on hover. Active module highlighted with `--ctp-mauve` indicator.
- **Page header:** `58px` height, `--ctp-mantle` background. Contains page title (with module icon), action buttons right-aligned.
- **Content area:** `--ctp-base` background. Scrollable. Contains list views, kanban boards, or other primary content.
- **Detail pane:** `360px` fixed width, `--ctp-mantle` background, `1px solid --ctp-surface0` left border. Appears on record selection.
### Grid Patterns
**Two-column form:**
```css
display: grid;
grid-template-columns: 1fr 1fr;
gap: 1.25rem 1.5rem;
max-width: 800px;
```
**List + detail:**
```css
display: grid;
grid-template-columns: 1fr 360px;
min-height: calc(100vh - 52px - 58px);
```
### Breakpoints
Not currently required. Silo targets desktop browsers on engineering workstations. If mobile support is added later, breakpoints will be defined at `768px` and `1024px`.
---
## Components
### Buttons
Four tiers. All buttons share a base style:
```css
display: inline-flex;
align-items: center;
gap: 0.35rem;
padding: 0.4rem 0.85rem;
border-radius: 6px;
font-size: 12px;
font-weight: 500;
cursor: pointer;
transition: all 0.15s;
```
| Tier | Name | Background | Border | Text | Hover |
|------|------|-----------|--------|------|-------|
| Primary | `.btn-primary` | `--ctp-mauve` | `--ctp-mauve` | `--ctp-crust` | `--ctp-lavender` bg + border |
| Secondary | `.btn` (default) | `--ctp-surface0` | `--ctp-surface1` | `--ctp-text` | `--ctp-surface1` bg, `--ctp-overlay0` border |
| Ghost | `.btn-ghost` | transparent | transparent | `--ctp-subtext0` | `--ctp-surface0` bg, `--ctp-text` text |
| Danger | `.btn-danger` | transparent | `--ctp-surface1` | `--ctp-red` | `rgba(243, 139, 168, 0.1)` bg, `--ctp-red` border |
Primary is used once per visible context (the main action). All other actions use secondary or ghost. Danger is only for destructive actions and always requires confirmation.
### Badges
Used for type indicators, status labels, and tags.
```css
display: inline-flex;
align-items: center;
padding: 0.15rem 0.5rem;
border-radius: 4px;
font-size: 10px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.03em;
```
Badges use a translucent background derived from their accent color:
```css
/* Example: assembly badge */
background: rgba(203, 166, 247, 0.15); /* --ctp-mauve at 15% */
color: var(--ctp-mauve);
```
Standard badge colors follow the [accent usage table](#accent-usage-for-data-types). Status badges:
| Status | Color |
|--------|-------|
| Active / Released | `--ctp-green` |
| Draft / In Progress | `--ctp-blue` |
| Review / Pending | `--ctp-yellow` |
| Obsolete / Rejected | `--ctp-red` |
| Locked | `--ctp-overlay1` |
### Form Inputs
All inputs share a base style:
```css
background: var(--ctp-crust);
border: 1px solid var(--ctp-surface1);
border-radius: 6px;
padding: 0.45rem 0.65rem;
font-size: 12px;
color: var(--ctp-text);
transition: border-color 0.15s;
```
| State | Border | Shadow |
|-------|--------|--------|
| Default | `--ctp-surface1` | None |
| Hover | `--ctp-overlay0` | None |
| Focus | `--ctp-mauve` | `0 0 0 0.2rem rgba(203, 166, 247, 0.25)` |
| Error | `--ctp-red` | `0 0 0 0.2rem rgba(243, 139, 168, 0.15)` |
| Disabled | `--ctp-surface0` | None, `opacity: 0.5` |
Placeholder text: `--ctp-overlay0`. Labels sit above inputs (never inline or floating).
### Tag Input
Used for multi-value fields (projects, tags):
```css
display: flex;
flex-wrap: wrap;
gap: 0.3rem;
padding: 0.35rem 0.5rem;
background: var(--ctp-crust);
border: 1px solid var(--ctp-surface1);
border-radius: 6px;
min-height: 36px;
```
Individual tags use the badge pattern: `rgba(accent, 0.15)` background with accent text. Remove button (×) at `opacity: 0.6`, `1.0` on hover.
### Tables
```css
width: 100%;
border-collapse: collapse;
font-size: 12px;
```
| Element | Style |
|---------|-------|
| Header row | `background: --ctp-mantle`, `font-size: 11px`, uppercase, `--ctp-overlay1` text |
| Body row | `border-bottom: 1px solid --ctp-surface0` |
| Row hover | `background: --ctp-surface0` |
| Row selected | `background: rgba(203, 166, 247, 0.08)` |
| Cell padding | `0.4rem 0.75rem` |
| Text columns | Left-aligned |
| Number columns | Right-aligned |
| Date columns | Right-aligned |
| Action columns | Center-aligned |
Row actions use icon buttons (not text links). Icons at 14px, `--ctp-overlay1` default, `--ctp-text` on hover.
### Tabs
Used in detail panes and module sub-views:
```css
display: flex;
gap: 0;
border-bottom: 2px solid var(--ctp-surface0);
```
| State | Style |
|-------|-------|
| Default | `padding: 0.5rem 1rem`, `--ctp-subtext0` text, no border |
| Hover | `--ctp-text` text |
| Active | `--ctp-text` text, `font-weight: 600`, `border-bottom: 2px solid --ctp-mauve` (overlaps container border) |
### Section Dividers
Used to visually group form fields:
```css
display: flex;
align-items: center;
gap: 0.75rem;
grid-column: 1 / -1; /* span full form grid */
margin-top: 0.75rem;
```
Contains a label (`11px`, uppercase, `--ctp-overlay0`) and a horizontal line (`flex: 1`, `1px solid --ctp-surface0`).
### Sidebar Sections
Stacked vertically within detail panes:
```css
padding: 1rem 1.25rem;
border-bottom: 1px solid var(--ctp-surface0);
```
Last section has no bottom border. Section titles follow the section header typography (11px, uppercase, `--ctp-overlay0`).
### Tooltips
Appear on hover after a 300ms delay. Position: above the target element by default, flip below if insufficient space.
```css
background: var(--ctp-surface0);
border: 1px solid var(--ctp-surface1);
border-radius: 4px;
padding: 0.3rem 0.6rem;
font-size: 11px;
color: var(--ctp-text);
box-shadow: 0 4px 12px rgba(17, 17, 27, 0.4);
```
### Breadcrumbs
Module navigation breadcrumbs:
```
Module Name > List View > Record Name > Sub-view
```
Separator: `>` character in `--ctp-overlay0`. Segments are clickable links in `--ctp-subtext1`. Active (final) segment is `--ctp-text` at `font-weight: 600`.
### Dropdowns / Selects
Follow the input base style. The dropdown menu:
```css
background: var(--ctp-surface0);
border: 1px solid var(--ctp-surface1);
border-radius: 6px;
box-shadow: 0 8px 24px rgba(17, 17, 27, 0.5);
padding: 0.25rem;
max-height: 240px;
overflow-y: auto;
```
Menu items:
```css
padding: 0.4rem 0.65rem;
border-radius: 4px;
font-size: 12px;
color: var(--ctp-text);
cursor: pointer;
```
Hover: `background: --ctp-surface1`. Selected: `background: rgba(203, 166, 247, 0.12)`, `color: --ctp-mauve`, `font-weight: 600`.
---
## Icons
Use [Lucide](https://lucide.dev) icons. Size: 14px for inline/table contexts, 16px for buttons and navigation, 20px for page headers and empty states.
Stroke width: 1.5px (Lucide default). Color inherits from parent text color unless explicitly set.
Do not mix icon libraries. If Lucide does not have a suitable icon, request one be added or create a custom SVG following Lucide's 24×24 grid and stroke conventions.
---
## Transitions & Animation
All interactive state changes use `transition: all 0.15s ease`. This applies to hover, focus, active, and open/close states.
No entrance animations on page load. Content renders immediately. Skeleton loaders are acceptable for async data using a pulsing `--ctp-surface0``--ctp-surface1` gradient.
Dropdown menus and tooltips appear instantly (no slide/fade). Collapse/expand panels (if used) transition `max-height` at `0.2s ease`.
---
## Styling Implementation
Silo's React frontend uses **inline `React.CSSProperties` objects** with `var(--ctp-*)` token references. This is the project convention and must not be changed.
### Rules
- No CSS modules, no Tailwind, no external CSS-in-JS libraries.
- Styles are defined as `const` objects at the top of each component file.
- Shared style patterns (button base, input base) can be extracted to a `styles/` directory as exported `CSSProperties` objects.
- Use `as const` or `as React.CSSProperties` for type safety.
- Pseudo-classes (`:hover`, `:focus`) require state-driven inline styles or a thin CSS file for the base pseudo-class rules.
### Example
```typescript
const styles = {
container: {
display: 'grid',
gridTemplateColumns: '1fr 360px',
height: '100%',
overflow: 'hidden',
} as React.CSSProperties,
sidebar: {
background: 'var(--ctp-mantle)',
borderLeft: '1px solid var(--ctp-surface0)',
display: 'flex',
flexDirection: 'column' as const,
overflowY: 'auto' as const,
} as React.CSSProperties,
};
```
### Pseudo-class CSS
A single `silo-base.css` file provides pseudo-class rules that cannot be expressed inline:
```css
/* Hover, focus, and active states for core interactive elements */
.silo-input:hover { border-color: var(--ctp-overlay0); }
.silo-input:focus { border-color: var(--ctp-mauve); box-shadow: 0 0 0 0.2rem rgba(203, 166, 247, 0.25); }
.silo-btn:hover { /* per-tier overrides */ }
.silo-row:hover { background: var(--ctp-surface0); }
```
Components apply the corresponding class names alongside their inline styles. This is the only place class-based styling is used.
---
## Do / Don't
| Do | Don't |
|----|-------|
| Use `var(--ctp-*)` for every color | Hardcode hex values |
| Use the 4px spacing scale | Use arbitrary padding/margins |
| Use Lucide icons at standard sizes | Mix icon libraries |
| Use inline `CSSProperties` | Use CSS modules or Tailwind |
| One primary button per visible context | Multiple competing primary buttons |
| Use translucent accent backgrounds for badges | Use solid bright backgrounds for badges |
| Use icon buttons for row-level table actions | Use text links in table rows |
| Define styles as `const` at file top | Inline style objects in JSX |
| Show tooltips on icon-only buttons | Leave icon buttons unlabeled |
| Use section dividers to group form fields | Use cards or borders around field groups |
| Follow the breadcrumb pattern for navigation | Use nested tab bars |
---
## Appendix: CSS Custom Properties Block
Paste this at the root of the application stylesheet:
```css
:root {
--ctp-rosewater: #f5e0dc;
--ctp-flamingo: #f2cdcd;
--ctp-pink: #f5c2e7;
--ctp-mauve: #cba6f7;
--ctp-red: #f38ba8;
--ctp-maroon: #eba0ac;
--ctp-peach: #fab387;
--ctp-yellow: #f9e2af;
--ctp-green: #a6e3a1;
--ctp-teal: #94e2d5;
--ctp-sky: #89dceb;
--ctp-sapphire: #74c7ec;
--ctp-blue: #89b4fa;
--ctp-lavender: #b4befe;
--ctp-text: #cdd6f4;
--ctp-subtext1: #bac2de;
--ctp-subtext0: #a6adc8;
--ctp-overlay2: #9399b2;
--ctp-overlay1: #7f849c;
--ctp-overlay0: #6c7086;
--ctp-surface2: #585b70;
--ctp-surface1: #45475a;
--ctp-surface0: #313244;
--ctp-base: #1e1e2e;
--ctp-mantle: #181825;
--ctp-crust: #11111b;
}
```

364
docs/WORKERS.md Normal file
View File

@@ -0,0 +1,364 @@
# Worker System Specification
**Status:** Draft
**Last Updated:** 2026-02-13
---
## 1. Purpose
The worker system provides async compute job execution for Silo. Jobs are defined as YAML files, managed by the Silo server, and executed by external runner processes. The system is general-purpose -- while DAG validation is the first use case, it supports any compute workload: geometry export, thumbnail rendering, FEA/CFD batch jobs, report generation, and data migration.
---
## 2. Architecture
```
YAML Job Definitions (files on disk, version-controllable)
|
v
Silo Server (parser, scheduler, state machine, REST API, SSE events)
|
v
Runners (silorunner binary, polls via REST, executes Headless Create)
```
**Three layers:**
1. **Job definitions** -- YAML files in a configurable directory (default `/etc/silo/jobdefs`). Each file defines a job type: what triggers it, what it operates on, what computation to perform, and what runner capabilities are required. These are the source of truth and can be version-controlled alongside other Silo config.
2. **Silo server** -- Parses YAML definitions on startup and upserts them into the `job_definitions` table. Creates job instances when triggers fire (revision created, BOM changed, manual). Manages job lifecycle, enforces timeouts, and broadcasts status via SSE.
3. **Runners** -- Separate `silorunner` processes that authenticate with Silo via API tokens, poll for available jobs, claim them atomically, execute the compute, and report results. A runner host must have Headless Create and silo-mod installed for geometry jobs.
---
## 3. Job Definition Format
Job definitions are YAML files with the following structure:
```yaml
job:
name: assembly-validate
version: 1
description: "Validate assembly by rebuilding its dependency subgraph"
trigger:
type: revision_created # revision_created, bom_changed, manual, schedule
filter:
item_type: assembly # only trigger for assemblies
scope:
type: assembly # item, assembly, project
compute:
type: validate # validate, rebuild, diff, export, custom
command: create-validate # runner-side command identifier
args: # passed to runner as JSON
rebuild_mode: incremental
check_interference: true
runner:
tags: [create] # required runner capabilities
timeout: 900 # seconds before job is marked failed (default 600)
max_retries: 2 # retry count on failure (default 1)
priority: 50 # lower = higher priority (default 100)
```
### 3.1 Trigger Types
| Type | Description |
|------|-------------|
| `revision_created` | Fires when a new revision is created on an item matching the filter |
| `bom_changed` | Fires when a BOM merge completes |
| `manual` | Only triggered via `POST /api/jobs` |
| `schedule` | Future: cron-like scheduling (not yet implemented) |
### 3.2 Trigger Filters
The `filter` map supports key-value matching against item properties:
| Key | Description |
|-----|-------------|
| `item_type` | Match item type: `part`, `assembly`, `drawing`, etc. |
| `schema` | Match schema name |
All filter keys must match for the trigger to fire. An empty filter matches all items.
### 3.3 Scope Types
| Type | Description |
|------|-------------|
| `item` | Job operates on a single item |
| `assembly` | Job operates on an assembly and its BOM tree |
| `project` | Job operates on all items in a project |
### 3.4 Compute Commands
The `command` field identifies what the runner should execute. Built-in commands:
| Command | Description |
|---------|-------------|
| `create-validate` | Open file in Headless Create, rebuild features, report validation results |
| `create-export` | Open file, export to specified format (STEP, IGES, 3MF) |
| `create-dag-extract` | Open file, extract feature DAG, output as JSON |
| `create-thumbnail` | Open file, render thumbnail image |
Custom commands can be added by extending silo-mod's `silo.runner` module.
---
## 4. Job Lifecycle
```
pending → claimed → running → completed
→ failed
→ cancelled
```
| State | Description |
|-------|-------------|
| `pending` | Job created, waiting for a runner to claim it |
| `claimed` | Runner has claimed the job. `expires_at` is set. |
| `running` | Runner has started execution (reported via progress update) |
| `completed` | Runner reported success. `result` JSONB contains output. |
| `failed` | Runner reported failure, timeout expired, or max retries exceeded |
| `cancelled` | Admin cancelled the job before completion |
### 4.1 Claim Semantics
Runners claim jobs via `POST /api/runner/claim`. The server uses PostgreSQL's `SELECT FOR UPDATE SKIP LOCKED` to ensure exactly-once delivery:
```sql
WITH claimable AS (
SELECT id FROM jobs
WHERE status = 'pending'
AND runner_tags <@ $2::text[]
ORDER BY priority ASC, created_at ASC
LIMIT 1
FOR UPDATE SKIP LOCKED
)
UPDATE jobs SET
status = 'claimed',
runner_id = $1,
claimed_at = now(),
expires_at = now() + (timeout_seconds || ' seconds')::interval
FROM claimable
WHERE jobs.id = claimable.id
RETURNING jobs.*;
```
The `runner_tags <@ $2::text[]` condition ensures the runner has all tags required by the job. A runner with tags `["create", "linux", "gpu"]` can claim a job requiring `["create"]`, but not one requiring `["create", "windows"]`.
### 4.2 Timeout Enforcement
A background sweeper runs every 30 seconds (configurable via `jobs.job_timeout_check`) and marks expired jobs as failed:
```sql
UPDATE jobs SET status = 'failed', error_message = 'job timed out'
WHERE status IN ('claimed', 'running')
AND expires_at < now();
```
### 4.3 Retry
When a job fails and `retry_count < max_retries`, a new job is created with the same definition and scope, with `retry_count` incremented.
---
## 5. Runners
### 5.1 Registration
Runners are registered via `POST /api/runners` (admin only). The server generates a token (shown once) and stores the SHA-256 hash in the `runners` table. This follows the same pattern as API tokens in `internal/auth/token.go`.
### 5.2 Authentication
Runners authenticate via `Authorization: Bearer silo_runner_<token>`. A dedicated `RequireRunnerAuth` middleware validates the token against the `runners` table and injects a `RunnerIdentity` into the request context.
### 5.3 Heartbeat
Runners send `POST /api/runner/heartbeat` every 30 seconds. The server updates `last_heartbeat` and sets `status = 'online'`. A background sweeper marks runners as `offline` if their heartbeat is older than `runner_timeout` seconds (default 90).
### 5.4 Tags
Each runner declares capability tags (e.g., `["create", "linux", "gpu"]`). Jobs require specific tags via the `runner.tags` field in their YAML definition. A runner can only claim jobs whose required tags are a subset of the runner's tags.
### 5.5 Runner Config
The `silorunner` binary reads its config from a YAML file:
```yaml
server_url: "https://silo.example.com"
token: "silo_runner_abc123..."
name: "worker-01"
tags: ["create", "linux"]
poll_interval: 5 # seconds between claim attempts
create_path: "/usr/bin/create" # path to Headless Create binary (with silo-mod installed)
```
Or via environment variables: `SILO_RUNNER_SERVER_URL`, `SILO_RUNNER_TOKEN`, etc.
### 5.6 Deployment
Runner prerequisites:
- `silorunner` binary (built from `cmd/silorunner/`)
- Headless Create (Kindred's fork of FreeCAD) with silo-mod workbench installed
- Network access to Silo server API
Runners can be deployed as:
- Bare metal processes alongside Create installations
- Docker containers with Create pre-installed
- Scaled horizontally by registering multiple runners with different names
---
## 6. Job Log
Each job has an append-only log stored in the `job_log` table. Runners append entries via `POST /api/runner/jobs/{jobID}/log`:
```json
{
"level": "info",
"message": "Rebuilding Pad003...",
"metadata": {"node_key": "Pad003", "progress_pct": 45}
}
```
Log levels: `debug`, `info`, `warn`, `error`.
---
## 7. SSE Events
All job lifecycle transitions are broadcast via Silo's SSE broker. Clients subscribe to `/api/events` and receive:
| Event Type | Payload | When |
|------------|---------|------|
| `job.created` | `{id, definition_name, item_id, status, priority}` | Job created |
| `job.claimed` | `{id, runner_id, runner_name}` | Runner claims job |
| `job.progress` | `{id, progress, progress_message}` | Runner reports progress (0-100) |
| `job.completed` | `{id, result_summary, duration_seconds}` | Job completed successfully |
| `job.failed` | `{id, error_message}` | Job failed |
| `job.cancelled` | `{id, cancelled_by}` | Admin cancelled job |
| `runner.online` | `{id, name, tags}` | Runner heartbeat (first after offline) |
| `runner.offline` | `{id, name}` | Runner heartbeat timeout |
---
## 8. REST API
### 8.1 Job Endpoints (user-facing, require auth)
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/api/jobs` | viewer | List jobs (filterable by status, item, definition) |
| `GET` | `/api/jobs/{jobID}` | viewer | Get job details |
| `GET` | `/api/jobs/{jobID}/logs` | viewer | Get job log entries |
| `POST` | `/api/jobs` | editor | Manually trigger a job |
| `POST` | `/api/jobs/{jobID}/cancel` | editor | Cancel a pending/running job |
### 8.2 Job Definition Endpoints
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/api/job-definitions` | viewer | List loaded definitions |
| `GET` | `/api/job-definitions/{name}` | viewer | Get specific definition |
| `POST` | `/api/job-definitions/reload` | admin | Re-read YAML from disk |
### 8.3 Runner Management Endpoints (admin)
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `GET` | `/api/runners` | admin | List registered runners |
| `POST` | `/api/runners` | admin | Register runner (returns token) |
| `DELETE` | `/api/runners/{runnerID}` | admin | Delete runner |
### 8.4 Runner-Facing Endpoints (runner token auth)
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `POST` | `/api/runner/heartbeat` | runner | Send heartbeat |
| `POST` | `/api/runner/claim` | runner | Claim next available job |
| `PUT` | `/api/runner/jobs/{jobID}/progress` | runner | Report progress |
| `POST` | `/api/runner/jobs/{jobID}/complete` | runner | Report completion with result |
| `POST` | `/api/runner/jobs/{jobID}/fail` | runner | Report failure |
| `POST` | `/api/runner/jobs/{jobID}/log` | runner | Append log entry |
| `PUT` | `/api/runner/jobs/{jobID}/dag` | runner | Sync DAG results after compute |
---
## 9. Configuration
Add to `config.yaml`:
```yaml
jobs:
directory: /etc/silo/jobdefs # path to YAML job definitions
runner_timeout: 90 # seconds before marking runner offline
job_timeout_check: 30 # seconds between timeout sweeps
default_priority: 100 # default job priority
```
---
## 10. Example Job Definitions
### Assembly Validation
```yaml
job:
name: assembly-validate
version: 1
description: "Validate assembly by rebuilding its dependency subgraph"
trigger:
type: revision_created
filter:
item_type: assembly
scope:
type: assembly
compute:
type: validate
command: create-validate
args:
rebuild_mode: incremental
check_interference: true
runner:
tags: [create]
timeout: 900
max_retries: 2
priority: 50
```
### STEP Export
```yaml
job:
name: part-export-step
version: 1
description: "Export a part to STEP format"
trigger:
type: manual
scope:
type: item
compute:
type: export
command: create-export
args:
format: step
output_key_template: "exports/{part_number}_rev{revision}.step"
runner:
tags: [create]
timeout: 300
max_retries: 1
priority: 100
```
---
## 11. References
- [DAG.md](DAG.md) -- Dependency DAG specification
- [MULTI_USER_EDITS.md](MULTI_USER_EDITS.md) -- Multi-user editing specification
- [ROADMAP.md](ROADMAP.md) -- Tier 0 Job Queue Infrastructure, Tier 1 Headless Create

View File

@@ -1,6 +1,6 @@
# Silo Frontend Specification
Current as of 2026-02-08. Documents the React + Vite + TypeScript frontend (migration from Go templates is complete).
Current as of 2026-02-11. Documents the React + Vite + TypeScript frontend (migration from Go templates is complete).
## Overview
@@ -68,6 +68,7 @@ web/
│ └── AuthContext.tsx AuthProvider with login/logout/refresh methods
├── hooks/
│ ├── useAuth.ts Context consumer hook
│ ├── useFormDescriptor.ts Fetches form descriptor from /api/schemas/{name}/form (replaces useCategories)
│ ├── useItems.ts Items fetching with search, filters, pagination, debounce
│ └── useLocalStorage.ts Typed localStorage persistence hook
├── styles/
@@ -271,63 +272,81 @@ Vite dev server runs on port 5173 with proxy config in `vite.config.ts` forwardi
## New Frontend Tasks
# CreateItemPane Redesign Specification
# CreateItemPane — Schema-Driven Dynamic Form
**Date**: 2026-02-06
**Scope**: Replace existing `CreateItemPane.tsx` with a two-column layout, multi-stage category picker, file attachment via MinIO, and full use of screen real estate.
**Date**: 2026-02-10
**Scope**: `CreateItemPane.tsx` renders a dynamic form driven entirely by the form descriptor API (`GET /api/schemas/{name}/form`). All field groups, field types, widgets, and category-specific fields are defined in YAML and resolved server-side.
**Parent**: Items page (`ItemsPage.tsx`) — renders in the detail pane area per existing in-pane CRUD pattern.
---
## Layout
The pane uses a CSS Grid two-column layout instead of the current single-column form:
Single-column scrollable form with a green header bar. Field groups are rendered dynamically from the form descriptor. Category-specific field groups appear after global groups when a category is selected.
```
┌────────────────────────────────────────────────────────────────────┐
│ Header: "New Item" [green bar] Cancel │ Create │
├──────────────────────────────────────────────────────┤ │
│ Auto-
── Identity ────────────────────────────────────── │ assigned
[Part Number *] [Type * v] │ metadata
[Description ] │
Category * [Domain │ Group │ Subtype ] │──────────────
Mechanical│ Structural│ Bracket │ │
Electrical│ Bearings │ Plate │ │ Attachments
... │ ... │ ... │ │ ┌─ ─ ─ ─ ┐
── Sourcing ────────────────────────────────────── │ │ Drop
│ [Sourcing Type v] [Standard Cost $ ] │ zone │
[Unit of Measure v] [Sourcing Link ] └─ ─ ─ ─ ┘
│ file.FCStd
── Details ─────────────────────────────────────── │ drawing.pdf
[Long Description ]
[Projects: [tag][tag] type to search... ] │──────────────│
│ Thumbnail
│ [preview]
──────────────────────────────────────────────────────┴──────────────┘
┌──────────────────────────────────────────────────────────────────────┐
│ Header: "New Item" [green bar] Cancel │ Create │
├──────────────────────────────────────────────────────────────────────┤
Category * [Domain buttons: F C R S E M T A P X]
[Subcategory search + filtered list]
── Identity ──────────────────────────────────────────────────────
[Type * (auto-derived from category)] [Description ]
── Sourcing ──────────────────────────────────────────────────────
[Sourcing Type v] [Manufacturer] [MPN] [Supplier] [SPN]
│ [Sourcing Link]
── Cost & Lead Time ──────────────────────────────────────────────
[Standard Cost $] [Lead Time Days] [Min Order Qty]
── Status ────────────────────────────────────────────────────────
[Lifecycle Status v] [RoHS Compliant ☐] [Country of Origin]
│ ── Details ───────────────────────────────────────────────────────
│ [Long Description ] │
│ [Projects: [tag][tag] type to search... ] │
│ [Notes ] │
│ │
│ ── Fastener Specifications (category-specific) ─────────────────── │
│ [Material] [Finish] [Thread Size] [Head Type] [Drive Type] ... │
│ │
└──────────────────────────────────────────────────────────────────────┘
```
Grid definition: `grid-template-columns: 1fr 320px`. The left column scrolls independently if content overflows. The right sidebar is a flex column with sections separated by `--ctp-surface1` borders.
## Data Source — Form Descriptor API
All form structure is fetched from `GET /api/schemas/kindred-rd/form`, which returns:
- `category_picker`: Multi-stage picker config (domain → subcategory)
- `item_fields`: Definitions for item-level fields (description, item_type, sourcing_type, etc.)
- `field_groups`: Ordered groups with resolved field metadata (Identity, Sourcing, Cost, Status, Details)
- `category_field_groups`: Per-category-prefix groups (e.g., Fastener Specifications for `F` prefix)
- `field_overrides`: Widget hints (currency, url, select, checkbox)
The YAML schema (`schemas/kindred-rd.yaml`) is the single source of truth. Adding a new field or category in YAML propagates to all clients with no code changes.
## File Location
`web/src/components/items/CreateItemPane.tsx` (replaces existing file)
`web/src/components/items/CreateItemPane.tsx`
New supporting files:
Supporting files:
| File | Purpose |
|------|---------|
| `web/src/components/items/CategoryPicker.tsx` | Multi-stage category selector |
| `web/src/components/items/FileDropZone.tsx` | Drag-and-drop file upload with MinIO presigned URLs |
| `web/src/components/items/CategoryPicker.tsx` | Multi-stage domain/subcategory selector |
| `web/src/components/items/FileDropZone.tsx` | Drag-and-drop file upload |
| `web/src/components/items/TagInput.tsx` | Multi-select tag input for projects |
| `web/src/hooks/useCategories.ts` | Fetches category tree from schema data |
| `web/src/hooks/useFormDescriptor.ts` | Fetches and caches form descriptor from `/api/schemas/{name}/form` |
| `web/src/hooks/useFileUpload.ts` | Manages presigned URL upload flow |
## Component Breakdown
### CreateItemPane
Top-level orchestrator. Manages form state, submission, and layout.
Top-level orchestrator. Renders dynamic form from the form descriptor.
**Props** (unchanged interface):
@@ -341,72 +360,68 @@ interface CreateItemPaneProps {
**State**:
```typescript
const [form, setForm] = useState<CreateItemForm>({
part_number: '',
item_type: 'part',
description: '',
category_path: [], // e.g. ['Mechanical', 'Structural', 'Bracket']
sourcing_type: 'manufactured',
standard_cost: '',
unit_of_measure: 'ea',
sourcing_link: '',
long_description: '',
project_ids: [],
});
const [attachments, setAttachments] = useState<PendingAttachment[]>([]);
const [thumbnail, setThumbnail] = useState<PendingAttachment | null>(null);
const { descriptor, categories, loading } = useFormDescriptor();
const [category, setCategory] = useState(''); // selected category code, e.g. "F01"
const [fields, setFields] = useState<Record<string, string>>({}); // all field values keyed by name
const [error, setError] = useState<string | null>(null);
const [submitting, setSubmitting] = useState(false);
```
A single `fields` record holds all form values (both item-level and property fields). The `ITEM_LEVEL_FIELDS` set (`description`, `item_type`, `sourcing_type`, `long_description`) determines which fields go into the top-level request vs. the `properties` map on submission.
**Auto-derivation**: When a category is selected, `item_type` is automatically set based on the `derived_from_category` mapping in the form descriptor (e.g., category prefix `A``assembly`, `T``tooling`, default → `part`).
**Dynamic rendering**: A `renderField()` function maps each field's `widget` type to the appropriate input:
| Widget | Rendered As |
|--------|-------------|
| `text` | `<input type="text">` |
| `number` | `<input type="number">` |
| `textarea` | `<textarea>` |
| `select` | `<select>` with `<option>` elements from `field.options` |
| `checkbox` | `<input type="checkbox">` |
| `currency` | `<input type="number">` with currency prefix (e.g., "$") |
| `url` | `<input type="url">` |
| `tag_input` | `TagInput` component with search endpoint |
**Submission flow**:
1. Validate required fields (part_number, item_type, category_path length === 3).
2. `POST /api/items` with form data → returns created `Item` with UUID.
3. For each attachment in `attachments[]`, call the file association endpoint: `POST /api/items/{id}/files` with the MinIO object key returned from upload.
4. If thumbnail exists, `PUT /api/items/{id}/thumbnail` with the object key.
5. Call `onCreated(item)`.
1. Validate required fields (category must be selected).
2. Split `fields` into item-level fields and properties using `ITEM_LEVEL_FIELDS`.
3. `POST /api/items` with `{ part_number: '', item_type, description, sourcing_type, long_description, category, properties: {...} }`.
4. Call `onCreated(item)`.
If step 2 fails, show error banner. If file association fails, show warning but still navigate (item was created, files can be re-attached).
**Header bar**: Green (`--ctp-green` background, `--ctp-crust` text) per existing create-pane convention. "New Item" title on left, Cancel (ghost button) and Create Item (primary button, `--ctp-green` bg) on right.
**Header bar**: Green (`--ctp-green` background, `--ctp-crust` text). "New Item" title on left, Cancel and Create Item buttons on right.
### CategoryPicker
Three-column scrollable list for hierarchical category selection.
Multi-stage category selector driven by the form descriptor's `category_picker.stages` config.
**Props**:
```typescript
interface CategoryPickerProps {
value: string[]; // current selection path, e.g. ['Mechanical', 'Structural']
onChange: (path: string[]) => void;
categories: CategoryNode[]; // top-level nodes
}
interface CategoryNode {
name: string;
children?: CategoryNode[];
value: string; // selected category code, e.g. "F01"
onChange: (code: string) => void;
categories: Record<string, string>; // flat code → description map
stages?: CategoryPickerStage[]; // from form descriptor
}
```
**Rendering**: Three side-by-side `<div>` columns inside a container with `border: 1px solid var(--ctp-surface1)` and `border-radius: 0.4rem`. Each column has:
**Rendering**: Two-stage selection:
- A sticky header row (10px uppercase, `--ctp-overlay0` text, `--ctp-mantle` background) labeling the tier. Labels come from the schema definition if available, otherwise "Level 1", "Level 2", "Level 3".
- A scrollable list of options. Each option is a `<div>` row, 28px height, `0.85rem` font. Hover: `--ctp-surface0` background. Selected: translucent mauve background (`rgba(203, 166, 247, 0.12)`), `--ctp-mauve` text, weight 600.
- If a node has children, show a `` chevron on the right side of the row.
1. **Domain row**: Horizontal row of buttons, one per domain from `stages[0].values` (F=Fasteners, C=Fluid Fittings, etc.). Selected domain has mauve highlight.
2. **Subcategory list**: Filtered list of categories matching the selected domain prefix. Includes a search input for filtering. Each row shows code and description.
Column 1 always shows all top-level nodes. Column 2 shows children of the selected Column 1 node. Column 3 shows children of the selected Column 2 node. If nothing is selected in a column, the next column shows an empty state with muted text: "Select a [tier name]".
If no `stages` prop is provided, falls back to a flat searchable list of all categories.
Below the picker, render a breadcrumb trail: `Mechanical Structural Bracket` in `--ctp-mauve` with `` separators in `--ctp-overlay0`. Only show segments that are selected.
Below the picker, the selected category is shown as a breadcrumb: `Fasteners F01 — Hex Cap Screw` in `--ctp-mauve`.
**Data source**: Categories are derived from schemas. The `useCategories` hook calls `GET /api/schemas` and transforms the response into a `CategoryNode[]` tree. The exact mapping depends on how schemas define category hierarchies — if schemas don't currently support hierarchical categories, this requires a backend addition (see Backend Changes section).
**Max height**: 180px per column with `overflow-y: auto`.
**Data source**: Categories come from `useFormDescriptor()` which derives them from the `category_picker` stages and `values_by_domain` in the form descriptor response.
### FileDropZone
Handles drag-and-drop and click-to-browse file uploads with MinIO presigned URL flow.
Handles drag-and-drop and click-to-browse file uploads.
**Props**:
@@ -420,7 +435,7 @@ interface FileDropZoneProps {
interface PendingAttachment {
file: File;
objectKey: string; // MinIO key after upload
objectKey: string; // storage key after upload
uploadProgress: number; // 0-100
uploadStatus: 'pending' | 'uploading' | 'complete' | 'error';
error?: string;
@@ -447,7 +462,7 @@ Clicking the zone opens a hidden `<input type="file" multiple>`.
1. On file selection/drop, immediately request a presigned upload URL: `POST /api/uploads/presign` with `{ filename, content_type, size }`.
2. Backend returns `{ object_key, upload_url, expires_at }`.
3. `PUT` the file directly to the presigned MinIO URL using `XMLHttpRequest` (for progress tracking).
3. `PUT` the file directly to the presigned URL using `XMLHttpRequest` (for progress tracking).
4. On completion, update `PendingAttachment.uploadStatus` to `'complete'` and store the `object_key`.
5. The `object_key` is later sent to the item creation endpoint to associate the file.
@@ -478,17 +493,17 @@ The dropdown is an absolutely-positioned `<div>` below the input container, `--c
**For projects**: `searchFn` calls `GET /api/projects?q={query}` and maps to `{ id: project.id, label: project.code + ' — ' + project.name }`.
### useCategories Hook
### useFormDescriptor Hook
```typescript
function useCategories(): {
categories: CategoryNode[];
function useFormDescriptor(schemaName = "kindred-rd"): {
descriptor: FormDescriptor | null;
categories: Record<string, string>; // flat code → description map derived from descriptor
loading: boolean;
error: string | null;
}
```
Fetches `GET /api/schemas` on mount and transforms into a category tree. Caches in a module-level variable so repeated renders don't refetch. If the API doesn't currently support hierarchical categories, this returns a flat list as a single-tier picker until the backend is extended.
Fetches `GET /api/schemas/{name}/form` on mount. Caches the result in a module-level variable so repeated renders/mounts don't refetch. Derives a flat `categories` map from the `category_picker` stages and `values_by_domain` in the response. Replaces the old `useCategories` hook (deleted).
### useFileUpload Hook
@@ -542,40 +557,42 @@ const styles = {
## Form Sections
The form is visually divided by section headers. Each header is a flex row containing a label (11px uppercase, `--ctp-overlay0`) and a `flex: 1` horizontal line (`1px solid --ctp-surface0`). Sections span `grid-column: 1 / -1`.
Form sections are rendered dynamically from the `field_groups` array in the form descriptor. Each section header is a flex row containing a label (11px uppercase, `--ctp-overlay0`) and a `flex: 1` horizontal line (`1px solid --ctp-surface0`).
| Section | Fields |
|---------|--------|
| Identity | Part Number*, Type*, Description, Category* |
| Sourcing | Sourcing Type, Standard Cost, Unit of Measure, Sourcing Link |
| Details | Long Description, Projects |
**Global field groups** (from `ui.field_groups` in YAML):
## Sidebar Sections
| Group Key | Label | Fields |
|-----------|-------|--------|
| identity | Identity | item_type, description |
| sourcing | Sourcing | sourcing_type, manufacturer, manufacturer_pn, supplier, supplier_pn, sourcing_link |
| cost | Cost & Lead Time | standard_cost, lead_time_days, minimum_order_qty |
| status | Status | lifecycle_status, rohs_compliant, country_of_origin |
| details | Details | long_description, projects, notes |
The right sidebar is divided into three sections with `borderBottom: 1px solid var(--ctp-surface0)`:
**Category-specific field groups** (from `ui.category_field_groups` in YAML, shown when a category is selected):
**Auto-assigned metadata**: Read-only key-value rows showing:
- UUID: "On create" in `--ctp-teal` italic
- Revision: "A" (hardcoded initial)
- Created By: current user's display name from `useAuth()`
| Prefix | Group | Example Fields |
|--------|-------|----------------|
| F | Fastener Specifications | material, finish, thread_size, head_type, drive_type, ... |
| C | Fitting Specifications | material, connection_type, size_1, pressure_rating, ... |
| R | Motion Specifications | bearing_type, bore_diameter, load_rating, ... |
| ... | ... | (one group per category prefix, defined in YAML) |
**Attachments**: `FileDropZone` component. Takes `flex: 1` to fill available space.
**Thumbnail**: A 4:3 aspect ratio placeholder box (`--ctp-crust` bg, `--ctp-surface0` border) with centered text "Generated from CAD file or upload manually". Clicking opens file picker filtered to images. If a thumbnail is uploaded, show it as an `<img>` with `object-fit: cover`.
Note: `sourcing_link` and `standard_cost` are revision properties (stored in the `properties` JSONB), not item-level DB columns. They were migrated from item-level fields in PR #1 (migration 013).
## Backend Changes
Items 1-3 and 5 below are implemented (migration `011_item_files.sql`, `internal/api/file_handlers.go`). Item 4 (hierarchical categories) remains open.
Items 1-5 below are implemented. Item 4 (hierarchical categories) is resolved by the form descriptor's multi-stage category picker.
### 1. Presigned Upload URL -- IMPLEMENTED
```
POST /api/uploads/presign
Request: { "filename": "bracket.FCStd", "content_type": "application/octet-stream", "size": 2400000 }
Response: { "object_key": "uploads/tmp/{uuid}/{filename}", "upload_url": "https://minio.../...", "expires_at": "2026-02-06T..." }
Response: { "object_key": "uploads/tmp/{uuid}/{filename}", "upload_url": "https://...", "expires_at": "2026-02-06T..." }
```
The Go handler generates a presigned PUT URL via the MinIO SDK. Objects are uploaded to a temporary prefix. On item creation, they're moved/linked to the item's permanent prefix.
The Go handler generates a presigned PUT URL for direct upload. Objects are uploaded to a temporary prefix. On item creation, they're moved/linked to the item's permanent prefix.
### 2. File Association -- IMPLEMENTED
@@ -595,35 +612,16 @@ Request: { "object_key": "uploads/tmp/{uuid}/thumb.png" }
Response: 204
```
Stores the thumbnail at `items/{item_id}/thumbnail.png` in MinIO. Updates `item.thumbnail_key` column.
Stores the thumbnail at `items/{item_id}/thumbnail.png` in storage. Updates `item.thumbnail_key` column.
### 4. Hierarchical Categories -- NOT IMPLEMENTED
### 4. Hierarchical Categories -- IMPLEMENTED (via Form Descriptor)
If schemas don't currently support a hierarchical category tree, one of these approaches:
Resolved by the schema-driven form descriptor (`GET /api/schemas/{name}/form`). The YAML schema's `ui.category_picker` section defines multi-stage selection:
**Option A — Schema-driven**: Add a `category_tree` JSON column to the `schemas` table that defines the hierarchy. The `GET /api/schemas` response already returns schemas; the frontend transforms this into the picker tree.
- **Stage 1 (domain)**: Groups categories by first character of category code (F=Fasteners, C=Fluid Fittings, etc.). Values defined in `ui.category_picker.stages[0].values`.
- **Stage 2 (subcategory)**: Auto-derived by the Go backend's `ValuesByDomain()` method, which groups the category enum values by their first character.
**Option B — Dedicated table**:
```sql
CREATE TABLE categories (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
parent_id UUID REFERENCES categories(id),
sort_order INT NOT NULL DEFAULT 0,
UNIQUE(parent_id, name)
);
```
With endpoints:
```
GET /api/categories → flat list with parent_id, frontend builds tree
POST /api/categories → { name, parent_id? }
PUT /api/categories/{id} → { name, sort_order }
DELETE /api/categories/{id} → cascade check
```
**Recommendation**: Option B is more flexible and keeps categories as a first-class entity. The three-tier picker doesn't need to be limited to exactly three levels — it can render as many columns as the deepest category path, but three is the practical default (Domain → Group → Subtype).
No separate `categories` table is needed — the existing schema enum values are the single source of truth. Adding a new category code to the YAML propagates to the picker automatically.
### 5. Database Schema Addition -- IMPLEMENTED
@@ -641,46 +639,89 @@ CREATE TABLE item_files (
CREATE INDEX idx_item_files_item ON item_files(item_id);
ALTER TABLE items ADD COLUMN thumbnail_key TEXT;
ALTER TABLE items ADD COLUMN category_id UUID REFERENCES categories(id);
ALTER TABLE items ADD COLUMN sourcing_type TEXT NOT NULL DEFAULT 'manufactured';
ALTER TABLE items ADD COLUMN sourcing_link TEXT;
ALTER TABLE items ADD COLUMN standard_cost NUMERIC(12,2);
ALTER TABLE items ADD COLUMN unit_of_measure TEXT NOT NULL DEFAULT 'ea';
ALTER TABLE items ADD COLUMN long_description TEXT;
```
## Implementation Order
1. **TagInput component**reusable, no backend changes needed, uses existing projects API.
2. **CategoryPicker component**start with flat/mock data, wire to real API after backend adds categories.
3. **FileDropZone + useFileUpload**requires presigned URL backend endpoint first.
4. **CreateItemPane rewrite**compose the above into the two-column layout.
5. **Backend: categories table + endpoints** — unblocks real category data.
6. **Backend: presigned uploads + item_files** — unblocks file attachments.
7. **Backend: items table migration** — adds new columns (sourcing_type, standard_cost, etc.).
1. **[DONE] Deduplicate sourcing_link/standard_cost** — Migrated from item-level DB columns to revision properties (migration 013). Removed from Go structs, API types, frontend types.
2. **[DONE] Form descriptor API** — Added `ui` section to YAML, Go structs + validation, `GET /api/schemas/{name}/form` endpoint.
3. **[DONE] useFormDescriptor hook** — Replaces `useCategories`, fetches and caches form descriptor.
4. **[DONE] CategoryPicker rewrite** — Multi-stage domain/subcategory picker driven by form descriptor.
5. **[DONE] CreateItemPane rewrite** — Dynamic form rendering from field groups, widget-based field rendering.
6. **TagInput component** — reusable, no backend changes needed, uses existing projects API.
7. **FileDropZone + useFileUpload** — requires presigned URL backend endpoint (already implemented).
Steps 1-2 can start immediately. Steps 5-7 can run in parallel once specified. Step 4 ties it all together.
## Types Added
## Types to Add
Add to `web/src/api/types.ts`:
The following types were added to `web/src/api/types.ts` for the form descriptor system:
```typescript
// Categories
interface Category {
id: string;
// Form descriptor types (from GET /api/schemas/{name}/form)
interface FormFieldDescriptor {
name: string;
parent_id: string | null;
sort_order: number;
type: string;
widget: string;
label: string;
required?: boolean;
default?: string;
unit?: string;
description?: string;
options?: string[];
currency?: string;
derived_from_category?: Record<string, string>;
search_endpoint?: string;
}
interface CategoryNode {
name: string;
id: string;
children?: CategoryNode[];
interface FormFieldGroup {
key: string;
label: string;
order: number;
fields: FormFieldDescriptor[];
}
// File uploads
interface CategoryPickerStage {
name: string;
label: string;
values?: Record<string, string>;
values_by_domain?: Record<string, Record<string, string>>;
}
interface CategoryPickerDescriptor {
style: string;
stages: CategoryPickerStage[];
}
interface ItemFieldDef {
type: string;
widget: string;
label: string;
required?: boolean;
default?: string;
options?: string[];
derived_from_category?: Record<string, string>;
search_endpoint?: string;
}
interface FieldOverride {
widget?: string;
currency?: string;
options?: string[];
}
interface FormDescriptor {
schema_name: string;
format: string;
category_picker: CategoryPickerDescriptor;
item_fields: Record<string, ItemFieldDef>;
field_groups: FormFieldGroup[];
category_field_groups: Record<string, FormFieldGroup[]>;
field_overrides: Record<string, FieldOverride>;
}
// File uploads (unchanged)
interface PresignRequest {
filename: string;
content_type: string;
@@ -703,20 +744,6 @@ interface ItemFile {
created_at: string;
}
// Extended create request
interface CreateItemRequest {
part_number: string;
item_type: 'part' | 'assembly' | 'document';
description?: string;
category_id?: string;
sourcing_type?: 'manufactured' | 'purchased' | 'phantom';
standard_cost?: number;
unit_of_measure?: string;
sourcing_link?: string;
long_description?: string;
project_ids?: string[];
}
// Pending upload (frontend only, not an API type)
interface PendingAttachment {
file: File;
@@ -726,3 +753,5 @@ interface PendingAttachment {
error?: string;
}
```
Note: `sourcing_link` and `standard_cost` have been removed from the `Item`, `CreateItemRequest`, and `UpdateItemRequest` interfaces — they are now stored as revision properties and rendered dynamically from the form descriptor.

13
go.mod
View File

@@ -11,7 +11,6 @@ require (
github.com/go-ldap/ldap/v3 v3.4.12
github.com/google/uuid v1.6.0
github.com/jackc/pgx/v5 v5.5.4
github.com/minio/minio-go/v7 v7.0.66
github.com/rs/zerolog v1.32.0
github.com/sahilm/fuzzy v0.1.1
golang.org/x/crypto v0.47.0
@@ -21,28 +20,16 @@ require (
require (
github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
github.com/go-jose/go-jose/v4 v4.1.3 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/puddle/v2 v2.2.1 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.17.4 // indirect
github.com/klauspost/cpuid/v2 v2.2.6 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/minio/md5-simd v1.1.2 // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/rs/xid v1.5.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
golang.org/x/net v0.48.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.40.0 // indirect
golang.org/x/text v0.33.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
)

27
go.sum
View File

@@ -13,8 +13,6 @@ github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ3
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 h1:BP4M0CvQ4S3TGls2FvczZtj5Re/2ZzkV9VwqPHH/3Bo=
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
github.com/go-chi/chi/v5 v5.0.12 h1:9euLV5sTrTNTRUU9POmDUvfxyj6LAABLUcEWO+JJb4s=
@@ -26,7 +24,6 @@ github.com/go-jose/go-jose/v4 v4.1.3/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9
github.com/go-ldap/ldap/v3 v3.4.12 h1:1b81mv7MagXZ7+1r7cLTWmyuTqVqdwbtJSjC0DAp9s4=
github.com/go-ldap/ldap/v3 v3.4.12/go.mod h1:+SPAGcTtOfmGsCb3h1RFiq4xpp4N636G75OEace8lNo=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
@@ -51,13 +48,6 @@ github.com/jcmturner/gokrb5/v8 v8.4.4 h1:x1Sv4HaTpepFkXbt2IkL29DXRf8sOfZXo8eRKh6
github.com/jcmturner/gokrb5/v8 v8.4.4/go.mod h1:1btQEpgT6k+unzCwX1KdWMEwPPkkgBtP+F6aCACiMrs=
github.com/jcmturner/rpc/v2 v2.0.3 h1:7FXXj8Ti1IaVFpSAziCZWNzbNuZmnvw/i6CqLNdWfZY=
github.com/jcmturner/rpc/v2 v2.0.3/go.mod h1:VUJYCIDm3PVOEHw8sgt091/20OJjskO/YJki3ELg/Hc=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/compress v1.17.4 h1:Ej5ixsIri7BrIjBkRZLTo6ghwrEtHFk7ijlczPW4fZ4=
github.com/klauspost/compress v1.17.4/go.mod h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.2.6 h1:ndNyv040zDGIDh8thGkXYjnFtiN02M1PVVF+JE/48xc=
github.com/klauspost/cpuid/v2 v2.2.6/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
@@ -73,31 +63,17 @@ github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovk
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
github.com/minio/minio-go/v7 v7.0.66 h1:bnTOXOHjOqv/gcMuiVbN9o2ngRItvqE774dG9nq0Dzw=
github.com/minio/minio-go/v7 v7.0.66/go.mod h1:DHAgmyQEGdW3Cif0UooKOyrT3Vxs82zNdV6tkKhRtbs=
github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM=
github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/rs/xid v1.5.0 h1:mKX4bl4iPYJtEIxp6CYiUuLQ/8DYMoz0PUdtGgMFRVc=
github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/rs/zerolog v1.32.0 h1:keLypqrlIjaFsbmJOBdB/qvyF8KEtCWHwobLp5l/mQ0=
github.com/rs/zerolog v1.32.0/go.mod h1:/7mN4D5sKwJLZQ2b/znpjC3/GQWY/xaDXUM0kKWRHss=
github.com/sahilm/fuzzy v0.1.1 h1:ceu5RHF8DGgoi+/dR5PsECjCDH1BE3Fnmpo7aVXOdRA=
github.com/sahilm/fuzzy v0.1.1/go.mod h1:VFvziUEIMCrT6A6tw2RFIXPXXmzXbOsSHF0DOI8ZK9Y=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
@@ -133,7 +109,6 @@ golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -166,8 +141,6 @@ gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -0,0 +1,391 @@
package api
import (
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/workflow"
)
// ApprovalResponse is the JSON representation for approval endpoints.
type ApprovalResponse struct {
ID string `json:"id"`
WorkflowName string `json:"workflow"`
ECONumber *string `json:"eco_number"`
State string `json:"state"`
UpdatedAt string `json:"updated_at"`
UpdatedBy *string `json:"updated_by"`
Signatures []SignatureResponse `json:"signatures"`
}
// SignatureResponse is the JSON representation for a signature.
type SignatureResponse struct {
Username string `json:"username"`
Role string `json:"role"`
Status string `json:"status"`
SignedAt *string `json:"signed_at"`
Comment *string `json:"comment"`
}
// CreateApprovalRequest is the JSON body for POST /approvals.
type CreateApprovalRequest struct {
Workflow string `json:"workflow"`
ECONumber string `json:"eco_number"`
Signers []SignerRequest `json:"signers"`
}
// SignerRequest defines a signer in the create request.
type SignerRequest struct {
Username string `json:"username"`
Role string `json:"role"`
}
// SignApprovalRequest is the JSON body for POST /approvals/{id}/sign.
type SignApprovalRequest struct {
Status string `json:"status"`
Comment *string `json:"comment"`
}
func approvalToResponse(a *db.ItemApproval) ApprovalResponse {
sigs := make([]SignatureResponse, len(a.Signatures))
for i, s := range a.Signatures {
var signedAt *string
if s.SignedAt != nil {
t := s.SignedAt.UTC().Format("2006-01-02T15:04:05Z")
signedAt = &t
}
sigs[i] = SignatureResponse{
Username: s.Username,
Role: s.Role,
Status: s.Status,
SignedAt: signedAt,
Comment: s.Comment,
}
}
return ApprovalResponse{
ID: a.ID,
WorkflowName: a.WorkflowName,
ECONumber: a.ECONumber,
State: a.State,
UpdatedAt: a.UpdatedAt.UTC().Format("2006-01-02T15:04:05Z"),
UpdatedBy: a.UpdatedBy,
Signatures: sigs,
}
}
// HandleGetApprovals returns all approvals with signatures for an item.
// GET /api/items/{partNumber}/approvals
func (s *Server) HandleGetApprovals(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
approvals, err := s.approvals.ListByItemWithSignatures(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list approvals")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list approvals")
return
}
resp := make([]ApprovalResponse, len(approvals))
for i, a := range approvals {
resp[i] = approvalToResponse(a)
}
writeJSON(w, http.StatusOK, resp)
}
// HandleCreateApproval creates an ECO with a workflow and signers.
// POST /api/items/{partNumber}/approvals
func (s *Server) HandleCreateApproval(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
var req CreateApprovalRequest
if err := readJSON(r, &req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if len(req.Signers) == 0 {
writeError(w, http.StatusBadRequest, "invalid_body", "At least one signer is required")
return
}
// Validate workflow exists
wf, ok := s.workflows[req.Workflow]
if !ok {
writeError(w, http.StatusBadRequest, "invalid_workflow", "Workflow '"+req.Workflow+"' not found")
return
}
// Validate each signer's role matches a gate in the workflow
for _, signer := range req.Signers {
if !wf.HasRole(signer.Role) {
writeError(w, http.StatusBadRequest, "invalid_role",
"Role '"+signer.Role+"' is not defined in workflow '"+req.Workflow+"'")
return
}
}
// Validate all required gates have at least one signer
signerRoles := make(map[string]bool)
for _, signer := range req.Signers {
signerRoles[signer.Role] = true
}
for _, gate := range wf.RequiredGates() {
if !signerRoles[gate.Role] {
writeError(w, http.StatusBadRequest, "missing_required_signer",
"Required role '"+gate.Role+"' ("+gate.Label+") has no assigned signer")
return
}
}
username := ""
if user := auth.UserFromContext(ctx); user != nil {
username = user.Username
}
var ecoNumber *string
if req.ECONumber != "" {
ecoNumber = &req.ECONumber
}
approval := &db.ItemApproval{
ItemID: item.ID,
WorkflowName: req.Workflow,
ECONumber: ecoNumber,
State: "pending",
UpdatedBy: &username,
}
if err := s.approvals.Create(ctx, approval); err != nil {
s.logger.Error().Err(err).Msg("failed to create approval")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to create approval")
return
}
// Add signature rows for each signer
for _, signer := range req.Signers {
sig := &db.ApprovalSignature{
ApprovalID: approval.ID,
Username: signer.Username,
Role: signer.Role,
Status: "pending",
}
if err := s.approvals.AddSignature(ctx, sig); err != nil {
s.logger.Error().Err(err).Msg("failed to add signature")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to add signer")
return
}
}
// Re-fetch with signatures for response
approval, err = s.approvals.GetWithSignatures(ctx, approval.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get approval")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get approval")
return
}
resp := approvalToResponse(approval)
writeJSON(w, http.StatusCreated, resp)
s.broker.Publish("approval.created", mustMarshal(map[string]any{
"part_number": partNumber,
"approval_id": approval.ID,
"workflow": approval.WorkflowName,
"eco_number": approval.ECONumber,
}))
}
// HandleSignApproval records an approve or reject signature.
// POST /api/items/{partNumber}/approvals/{id}/sign
func (s *Server) HandleSignApproval(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
approvalID := chi.URLParam(r, "id")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
approval, err := s.approvals.GetWithSignatures(ctx, approvalID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get approval")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get approval")
return
}
if approval == nil || approval.ItemID != item.ID {
writeError(w, http.StatusNotFound, "not_found", "Approval not found")
return
}
if approval.State != "pending" {
writeError(w, http.StatusUnprocessableEntity, "invalid_state",
"Approval is in state '"+approval.State+"', signatures can only be added when 'pending'")
return
}
var req SignApprovalRequest
if err := readJSON(r, &req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if req.Status != "approved" && req.Status != "rejected" {
writeError(w, http.StatusBadRequest, "invalid_status", "Status must be 'approved' or 'rejected'")
return
}
// Get the caller's username
username := ""
if user := auth.UserFromContext(ctx); user != nil {
username = user.Username
}
// Check that the caller has a pending signature on this approval
sig, err := s.approvals.GetSignatureForUser(ctx, approvalID, username)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get signature")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to check signature")
return
}
if sig == nil {
writeError(w, http.StatusForbidden, "not_a_signer", "You are not a signer on this approval")
return
}
if sig.Status != "pending" {
writeError(w, http.StatusConflict, "already_signed", "You have already signed this approval")
return
}
// Update the signature
if err := s.approvals.UpdateSignature(ctx, sig.ID, req.Status, req.Comment); err != nil {
s.logger.Error().Err(err).Msg("failed to update signature")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to update signature")
return
}
s.broker.Publish("approval.signed", mustMarshal(map[string]any{
"part_number": partNumber,
"approval_id": approvalID,
"username": username,
"status": req.Status,
}))
// Evaluate auto-advance based on workflow rules
wf := s.workflows[approval.WorkflowName]
if wf != nil {
// Re-fetch signatures after update
approval, err = s.approvals.GetWithSignatures(ctx, approvalID)
if err == nil && approval != nil {
newState := evaluateApprovalState(wf, approval)
if newState != "" && newState != approval.State {
if err := s.approvals.UpdateState(ctx, approvalID, newState, username); err != nil {
s.logger.Warn().Err(err).Msg("failed to auto-advance approval state")
} else {
approval.State = newState
s.broker.Publish("approval.completed", mustMarshal(map[string]any{
"part_number": partNumber,
"approval_id": approvalID,
"state": newState,
}))
}
}
}
}
// Return updated approval
if approval == nil {
approval, _ = s.approvals.GetWithSignatures(ctx, approvalID)
}
if approval != nil {
writeJSON(w, http.StatusOK, approvalToResponse(approval))
} else {
w.WriteHeader(http.StatusOK)
}
}
// HandleListWorkflows returns all loaded workflow definitions.
// GET /api/workflows
func (s *Server) HandleListWorkflows(w http.ResponseWriter, r *http.Request) {
resp := make([]map[string]any, 0, len(s.workflows))
for _, wf := range s.workflows {
resp = append(resp, map[string]any{
"name": wf.Name,
"version": wf.Version,
"description": wf.Description,
"gates": wf.Gates,
})
}
writeJSON(w, http.StatusOK, resp)
}
// evaluateApprovalState checks workflow rules against current signatures
// and returns the new state, or "" if no transition is needed.
func evaluateApprovalState(wf *workflow.Workflow, approval *db.ItemApproval) string {
// Check for any rejection
if wf.Rules.AnyReject != "" {
for _, sig := range approval.Signatures {
if sig.Status == "rejected" {
return wf.Rules.AnyReject
}
}
}
// Check if all required roles have approved
if wf.Rules.AllRequiredApprove != "" {
requiredRoles := make(map[string]bool)
for _, gate := range wf.RequiredGates() {
requiredRoles[gate.Role] = true
}
// For each required role, check that all signers with that role have approved
for _, sig := range approval.Signatures {
if requiredRoles[sig.Role] && sig.Status != "approved" {
return "" // at least one required signer hasn't approved yet
}
}
// All required signers approved
return wf.Rules.AllRequiredApprove
}
return ""
}
// readJSON decodes a JSON request body.
func readJSON(r *http.Request, v any) error {
return json.NewDecoder(r.Body).Decode(v)
}

View File

@@ -101,6 +101,8 @@ var manufacturedWeights = map[string]float64{
// Weight 1: engineering detail (category-specific default)
"sourcing_type": 1,
"lifecycle_status": 1,
// Weight 1: engineering detail
"has_files": 1,
// Weight 0.5: less relevant for in-house
"manufacturer": 0.5,
"supplier": 0.5,
@@ -112,8 +114,6 @@ var manufacturedWeights = map[string]float64{
var itemLevelFields = map[string]bool{
"description": true,
"sourcing_type": true,
"sourcing_link": true,
"standard_cost": true,
"long_description": true,
}
@@ -207,6 +207,7 @@ func scoreItem(
categoryProps map[string]schema.PropertyDefinition,
hasBOM bool,
bomChildCount int,
hasFiles bool,
categoryName string,
projects []string,
includeFields bool,
@@ -255,18 +256,6 @@ func scoreItem(
processField("description", "item", "string", item.Description)
processField("sourcing_type", "item", "string", item.SourcingType)
var sourcingLinkVal any
if item.SourcingLink != nil {
sourcingLinkVal = *item.SourcingLink
}
processField("sourcing_link", "item", "string", sourcingLinkVal)
var stdCostVal any
if item.StandardCost != nil {
stdCostVal = *item.StandardCost
}
processField("standard_cost", "item", "number", stdCostVal)
var longDescVal any
if item.LongDescription != nil {
longDescVal = *item.LongDescription
@@ -276,6 +265,7 @@ func scoreItem(
// Score has_bom for manufactured/assembly items.
if sourcingType == "manufactured" || isAssembly {
processField("has_bom", "computed", "boolean", hasBOM)
processField("has_files", "computed", "boolean", hasFiles)
}
// Score property fields from schema.
@@ -283,10 +273,6 @@ func scoreItem(
if skipFields[key] || itemLevelFields[key] {
continue
}
// sourcing_link and standard_cost are already handled at item level.
if key == "sourcing_link" || key == "standard_cost" {
continue
}
value := item.Properties[key]
processField(key, "property", def.Type, value)
}
@@ -412,6 +398,13 @@ func (s *Server) HandleAuditCompleteness(w http.ResponseWriter, r *http.Request)
return
}
fileStats, err := s.items.BatchGetFileStats(ctx, itemIDs)
if err != nil {
s.logger.Error().Err(err).Msg("failed to batch get file stats")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load file stats")
return
}
// Look up the schema for category resolution.
sch := s.schemas["kindred-rd"]
var catSegment *schema.Segment
@@ -440,9 +433,10 @@ func (s *Server) HandleAuditCompleteness(w http.ResponseWriter, r *http.Request)
bomCount := bomCounts[item.ID]
hasBOM := bomCount > 0
hasFiles := fileStats[item.ID].Count > 0
projects := projectCodes[item.ID]
result := scoreItem(item, categoryProps, hasBOM, bomCount, categoryName, projects, false)
result := scoreItem(item, categoryProps, hasBOM, bomCount, hasFiles, categoryName, projects, false)
allResults = append(allResults, *result)
}
@@ -544,6 +538,15 @@ func (s *Server) HandleAuditItemDetail(w http.ResponseWriter, r *http.Request) {
}
projects := projectCodes[item.ID]
// Get file stats.
fileStats, err := s.items.BatchGetFileStats(ctx, []string{item.ID})
if err != nil {
s.logger.Error().Err(err).Str("pn", partNumber).Msg("failed to get file stats for audit")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to load file stats")
return
}
hasFiles := fileStats[item.ID].Count > 0
// Category resolution.
cat := extractCategory(item.PartNumber)
categoryName := cat
@@ -561,7 +564,7 @@ func (s *Server) HandleAuditItemDetail(w http.ResponseWriter, r *http.Request) {
categoryProps = sch.PropertySchemas.GetPropertiesForCategory(cat)
}
result := scoreItem(iwp, categoryProps, hasBOM, bomCount, categoryName, projects, true)
result := scoreItem(iwp, categoryProps, hasBOM, bomCount, hasFiles, categoryName, projects, true)
writeJSON(w, http.StatusOK, result)
}

View File

@@ -0,0 +1,106 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
)
func newAuditRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Get("/api/audit/completeness", s.HandleAuditCompleteness)
r.Get("/api/audit/completeness/{partNumber}", s.HandleAuditItemDetail)
return r
}
func TestHandleAuditCompletenessEmpty(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newAuditRouter(s)
req := httptest.NewRequest("GET", "/api/audit/completeness", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
}
func TestHandleAuditCompleteness(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newAuditRouter(s)
createItemDirect(t, s, "AUD-001", "audit item 1", nil)
createItemDirect(t, s, "AUD-002", "audit item 2", nil)
req := httptest.NewRequest("GET", "/api/audit/completeness", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding response: %v", err)
}
// Should have items array
items, ok := resp["items"]
if !ok {
t.Fatal("response missing 'items' key")
}
itemList, ok := items.([]any)
if !ok {
t.Fatal("'items' is not an array")
}
if len(itemList) < 2 {
t.Errorf("expected at least 2 audit items, got %d", len(itemList))
}
}
func TestHandleAuditItemDetail(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newAuditRouter(s)
cost := 50.0
createItemDirect(t, s, "AUDDET-001", "audit detail item", &cost)
req := httptest.NewRequest("GET", "/api/audit/completeness/AUDDET-001", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding response: %v", err)
}
if resp["part_number"] != "AUDDET-001" {
t.Errorf("part_number: got %v, want %q", resp["part_number"], "AUDDET-001")
}
if _, ok := resp["score"]; !ok {
t.Error("response missing 'score' field")
}
if _, ok := resp["tier"]; !ok {
t.Error("response missing 'tier' field")
}
}
func TestHandleAuditItemDetailNotFound(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newAuditRouter(s)
req := httptest.NewRequest("GET", "/api/audit/completeness/NOPE-999", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("status: got %d, want %d", w.Code, http.StatusNotFound)
}
}

View File

@@ -0,0 +1,212 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/testutil"
"github.com/rs/zerolog"
)
// newAuthTestServer creates a Server with a real auth service (for token tests).
func newAuthTestServer(t *testing.T) *Server {
t.Helper()
pool := testutil.MustConnectTestPool(t)
database := db.NewFromPool(pool)
users := db.NewUserRepository(database)
tokens := db.NewTokenRepository(database)
authSvc := auth.NewService(zerolog.Nop(), users, tokens)
broker := NewBroker(zerolog.Nop())
state := NewServerState(zerolog.Nop(), nil, broker)
return NewServer(
zerolog.Nop(),
database,
map[string]*schema.Schema{},
"", // schemasDir
nil, // storage
authSvc, // authService
nil, // sessionManager
nil, // oidcBackend
nil, // authConfig
broker,
state,
nil, // jobDefs
"", // jobDefsDir
modules.NewRegistry(), // modules
nil, // cfg
nil, // workflows
)
}
// ensureTestUser creates a user in the DB and returns their ID.
func ensureTestUser(t *testing.T, s *Server, username string) string {
t.Helper()
u := &db.User{
Username: username,
DisplayName: "Test " + username,
Email: username + "@test.local",
AuthSource: "local",
Role: "admin",
}
users := db.NewUserRepository(s.db)
if err := users.Upsert(context.Background(), u); err != nil {
t.Fatalf("upserting user: %v", err)
}
return u.ID
}
func newAuthRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Get("/api/auth/me", s.HandleGetCurrentUser)
r.Post("/api/auth/tokens", s.HandleCreateToken)
r.Get("/api/auth/tokens", s.HandleListTokens)
r.Delete("/api/auth/tokens/{id}", s.HandleRevokeToken)
r.Get("/api/auth/config", s.HandleAuthConfig)
return r
}
func TestHandleGetCurrentUser(t *testing.T) {
s := newTestServer(t)
router := newAuthRouter(s)
req := authRequest(httptest.NewRequest("GET", "/api/auth/me", nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding response: %v", err)
}
if resp["username"] != "testadmin" {
t.Errorf("username: got %v, want %q", resp["username"], "testadmin")
}
if resp["role"] != "admin" {
t.Errorf("role: got %v, want %q", resp["role"], "admin")
}
}
func TestHandleGetCurrentUserUnauth(t *testing.T) {
s := newTestServer(t)
router := newAuthRouter(s)
// No auth context
req := httptest.NewRequest("GET", "/api/auth/me", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusUnauthorized {
t.Errorf("status: got %d, want %d", w.Code, http.StatusUnauthorized)
}
}
func TestHandleAuthConfig(t *testing.T) {
s := newTestServer(t)
router := newAuthRouter(s)
req := httptest.NewRequest("GET", "/api/auth/config", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding response: %v", err)
}
// With nil oidc and nil authConfig, both should be false
if resp["oidc_enabled"] != false {
t.Errorf("oidc_enabled: got %v, want false", resp["oidc_enabled"])
}
}
func TestHandleCreateAndListTokens(t *testing.T) {
s := newAuthTestServer(t)
router := newAuthRouter(s)
// Create a user in the DB so token generation can associate
userID := ensureTestUser(t, s, "tokenuser")
// Inject user with the DB-assigned ID
u := &auth.User{
ID: userID,
Username: "tokenuser",
DisplayName: "Test tokenuser",
Role: auth.RoleAdmin,
AuthSource: "local",
}
// Create token
body := `{"name":"test-token"}`
req := httptest.NewRequest("POST", "/api/auth/tokens", strings.NewReader(body))
req = req.WithContext(auth.ContextWithUser(req.Context(), u))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create token status: got %d, want %d; body: %s", w.Code, http.StatusCreated, w.Body.String())
}
var createResp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &createResp); err != nil {
t.Fatalf("decoding create response: %v", err)
}
if createResp["token"] == nil || createResp["token"] == "" {
t.Error("expected token in response")
}
tokenID, _ := createResp["id"].(string)
// List tokens
req = httptest.NewRequest("GET", "/api/auth/tokens", nil)
req = req.WithContext(auth.ContextWithUser(req.Context(), u))
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("list tokens status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var tokens []map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &tokens); err != nil {
t.Fatalf("decoding list response: %v", err)
}
if len(tokens) != 1 {
t.Errorf("expected 1 token, got %d", len(tokens))
}
// Revoke token
req = httptest.NewRequest("DELETE", "/api/auth/tokens/"+tokenID, nil)
req = req.WithContext(auth.ContextWithUser(req.Context(), u))
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNoContent {
t.Errorf("revoke token status: got %d, want %d; body: %s", w.Code, http.StatusNoContent, w.Body.String())
}
// List again — should be empty
req = httptest.NewRequest("GET", "/api/auth/tokens", nil)
req = req.WithContext(auth.ContextWithUser(req.Context(), u))
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
json.Unmarshal(w.Body.Bytes(), &tokens)
if len(tokens) != 0 {
t.Errorf("expected 0 tokens after revoke, got %d", len(tokens))
}
}

View File

@@ -1,6 +1,7 @@
package api
import (
"context"
"encoding/csv"
"encoding/json"
"fmt"
@@ -284,6 +285,8 @@ func (s *Server) HandleAddBOMEntry(w http.ResponseWriter, r *http.Request) {
}
writeJSON(w, http.StatusCreated, entry)
go s.triggerJobs(context.Background(), "bom_changed", parent.ID, parent)
}
// HandleUpdateBOMEntry updates an existing BOM relationship.
@@ -352,6 +355,8 @@ func (s *Server) HandleUpdateBOMEntry(w http.ResponseWriter, r *http.Request) {
return
}
go s.triggerJobs(context.Background(), "bom_changed", parent.ID, parent)
// Reload and return updated entry
entries, err := s.relationships.GetBOM(ctx, parent.ID)
if err == nil {
@@ -418,6 +423,8 @@ func (s *Server) HandleDeleteBOMEntry(w http.ResponseWriter, r *http.Request) {
Msg("BOM entry removed")
w.WriteHeader(http.StatusNoContent)
go s.triggerJobs(context.Background(), "bom_changed", parent.ID, parent)
}
// Helper functions
@@ -573,8 +580,20 @@ func (s *Server) HandleGetBOMCost(w http.ResponseWriter, r *http.Request) {
for i, e := range entries {
unitCost := 0.0
leaf, err := s.items.GetByID(ctx, e.ItemID)
if err == nil && leaf != nil && leaf.StandardCost != nil {
unitCost = *leaf.StandardCost
if err == nil && leaf != nil {
// Get standard_cost from revision properties
if revs, rerr := s.items.GetRevisions(ctx, leaf.ID); rerr == nil {
for _, rev := range revs {
if rev.RevisionNumber == leaf.CurrentRevision && rev.Properties != nil {
if sc, ok := rev.Properties["standard_cost"]; ok {
if cost, cok := sc.(float64); cok {
unitCost = cost
}
}
break
}
}
}
}
extCost := e.TotalQuantity * unitCost
totalCost += extCost
@@ -594,6 +613,56 @@ func (s *Server) HandleGetBOMCost(w http.ResponseWriter, r *http.Request) {
})
}
// BOM merge request/response types
// MergeBOMRequest represents a request to merge assembly BOM entries.
type MergeBOMRequest struct {
Source string `json:"source"`
Entries []MergeBOMEntry `json:"entries"`
}
// MergeBOMEntry represents a single entry in a merge request.
type MergeBOMEntry struct {
ChildPartNumber string `json:"child_part_number"`
Quantity *float64 `json:"quantity"`
}
// MergeBOMResponse represents the result of a BOM merge.
type MergeBOMResponse struct {
Status string `json:"status"`
Diff MergeBOMDiff `json:"diff"`
Warnings []MergeWarning `json:"warnings"`
ResolveURL string `json:"resolve_url"`
}
// MergeBOMDiff categorizes changes from a merge operation.
type MergeBOMDiff struct {
Added []MergeDiffEntry `json:"added"`
Removed []MergeDiffEntry `json:"removed"`
QuantityChanged []MergeQtyChange `json:"quantity_changed"`
Unchanged []MergeDiffEntry `json:"unchanged"`
}
// MergeDiffEntry represents an added, removed, or unchanged BOM entry.
type MergeDiffEntry struct {
PartNumber string `json:"part_number"`
Quantity *float64 `json:"quantity"`
}
// MergeQtyChange represents a BOM entry whose quantity changed.
type MergeQtyChange struct {
PartNumber string `json:"part_number"`
OldQuantity *float64 `json:"old_quantity"`
NewQuantity *float64 `json:"new_quantity"`
}
// MergeWarning represents a warning generated during merge.
type MergeWarning struct {
Type string `json:"type"`
PartNumber string `json:"part_number"`
Message string `json:"message"`
}
// BOM CSV headers matching the user-specified format.
var bomCSVHeaders = []string{
"Item", "Level", "Source", "PN", "Seller Description",
@@ -976,3 +1045,200 @@ func (s *Server) HandleImportBOMCSV(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, result)
}
// HandleMergeBOM merges assembly-derived BOM entries into the server's BOM.
// Added entries are created, quantity changes are applied, and entries present
// in the server but missing from the request are flagged as warnings (not deleted).
func (s *Server) HandleMergeBOM(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
parent, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get parent item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get parent item")
return
}
if parent == nil {
writeError(w, http.StatusNotFound, "not_found", "Parent item not found")
return
}
var req MergeBOMRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_json", err.Error())
return
}
if len(req.Entries) == 0 {
writeError(w, http.StatusBadRequest, "invalid_request", "entries must not be empty")
return
}
// Fetch existing BOM (includes Source field)
existing, err := s.relationships.GetBOM(ctx, parent.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get existing BOM")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get existing BOM")
return
}
// Build lookup map by child part number
existingMap := make(map[string]*db.BOMEntry, len(existing))
for _, e := range existing {
existingMap[e.ChildPartNumber] = e
}
var username *string
if user := auth.UserFromContext(ctx); user != nil {
username = &user.Username
}
diff := MergeBOMDiff{
Added: make([]MergeDiffEntry, 0),
Removed: make([]MergeDiffEntry, 0),
QuantityChanged: make([]MergeQtyChange, 0),
Unchanged: make([]MergeDiffEntry, 0),
}
var warnings []MergeWarning
// Process incoming entries
for _, entry := range req.Entries {
if entry.ChildPartNumber == "" {
continue
}
child, err := s.items.GetByPartNumber(ctx, entry.ChildPartNumber)
if err != nil {
s.logger.Error().Err(err).Str("child", entry.ChildPartNumber).Msg("failed to look up child")
warnings = append(warnings, MergeWarning{
Type: "error",
PartNumber: entry.ChildPartNumber,
Message: fmt.Sprintf("Error looking up item: %s", err.Error()),
})
continue
}
if child == nil {
warnings = append(warnings, MergeWarning{
Type: "not_found",
PartNumber: entry.ChildPartNumber,
Message: fmt.Sprintf("Item '%s' not found in database", entry.ChildPartNumber),
})
continue
}
if ex, ok := existingMap[entry.ChildPartNumber]; ok {
// Entry already exists — check quantity
oldQty := ex.Quantity
newQty := entry.Quantity
if quantitiesEqual(oldQty, newQty) {
diff.Unchanged = append(diff.Unchanged, MergeDiffEntry{
PartNumber: entry.ChildPartNumber,
Quantity: newQty,
})
} else {
// Update quantity
if err := s.relationships.Update(ctx, ex.RelationshipID, nil, newQty, nil, nil, nil, nil, username); err != nil {
s.logger.Error().Err(err).Str("child", entry.ChildPartNumber).Msg("failed to update quantity")
warnings = append(warnings, MergeWarning{
Type: "error",
PartNumber: entry.ChildPartNumber,
Message: fmt.Sprintf("Failed to update quantity: %s", err.Error()),
})
} else {
diff.QuantityChanged = append(diff.QuantityChanged, MergeQtyChange{
PartNumber: entry.ChildPartNumber,
OldQuantity: oldQty,
NewQuantity: newQty,
})
}
}
delete(existingMap, entry.ChildPartNumber)
} else {
// New entry — create
rel := &db.Relationship{
ParentItemID: parent.ID,
ChildItemID: child.ID,
RelType: "component",
Quantity: entry.Quantity,
Source: "assembly",
CreatedBy: username,
}
if err := s.relationships.Create(ctx, rel); err != nil {
if strings.Contains(err.Error(), "cycle") {
warnings = append(warnings, MergeWarning{
Type: "cycle",
PartNumber: entry.ChildPartNumber,
Message: fmt.Sprintf("Adding '%s' would create a cycle", entry.ChildPartNumber),
})
} else {
s.logger.Error().Err(err).Str("child", entry.ChildPartNumber).Msg("failed to create relationship")
warnings = append(warnings, MergeWarning{
Type: "error",
PartNumber: entry.ChildPartNumber,
Message: fmt.Sprintf("Failed to create: %s", err.Error()),
})
}
continue
}
diff.Added = append(diff.Added, MergeDiffEntry{
PartNumber: entry.ChildPartNumber,
Quantity: entry.Quantity,
})
}
}
// Remaining entries in existingMap are not in the merge request
for pn, e := range existingMap {
if e.Source == "assembly" {
diff.Removed = append(diff.Removed, MergeDiffEntry{
PartNumber: pn,
Quantity: e.Quantity,
})
warnings = append(warnings, MergeWarning{
Type: "unreferenced",
PartNumber: pn,
Message: "Present in server BOM but not in assembly",
})
}
}
resp := MergeBOMResponse{
Status: "merged",
Diff: diff,
Warnings: warnings,
ResolveURL: fmt.Sprintf("/items/%s/bom", partNumber),
}
s.logger.Info().
Str("parent", partNumber).
Int("added", len(diff.Added)).
Int("updated", len(diff.QuantityChanged)).
Int("unchanged", len(diff.Unchanged)).
Int("unreferenced", len(diff.Removed)).
Int("warnings", len(warnings)).
Msg("BOM merge completed")
s.broker.Publish("bom.merged", mustMarshal(map[string]any{
"part_number": partNumber,
"added": len(diff.Added),
"quantity_changed": len(diff.QuantityChanged),
"unchanged": len(diff.Unchanged),
"unreferenced": len(diff.Removed),
}))
// Trigger auto-jobs (e.g. assembly validation)
go s.triggerJobs(context.Background(), "bom_changed", parent.ID, parent)
writeJSON(w, http.StatusOK, resp)
}
// quantitiesEqual compares two nullable float64 quantities.
func quantitiesEqual(a, b *float64) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}

View File

@@ -11,6 +11,7 @@ import (
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/testutil"
"github.com/rs/zerolog"
@@ -35,6 +36,11 @@ func newTestServer(t *testing.T) *Server {
nil, // authConfig (nil = dev mode)
broker,
state,
nil, // jobDefs
"", // jobDefsDir
modules.NewRegistry(), // modules
nil, // cfg
nil, // workflows
)
}
@@ -55,12 +61,15 @@ func newTestRouter(s *Server) http.Handler {
func createItemDirect(t *testing.T, s *Server, pn, desc string, cost *float64) {
t.Helper()
item := &db.Item{
PartNumber: pn,
ItemType: "part",
Description: desc,
StandardCost: cost,
PartNumber: pn,
ItemType: "part",
Description: desc,
}
if err := s.items.Create(context.Background(), item, nil); err != nil {
var props map[string]any
if cost != nil {
props = map[string]any{"standard_cost": *cost}
}
if err := s.items.Create(context.Background(), item, props); err != nil {
t.Fatalf("creating item %s: %v", pn, err)
}
}

View File

@@ -18,8 +18,34 @@ type Event struct {
// sseClient represents a single connected SSE consumer.
type sseClient struct {
ch chan Event
closed chan struct{}
ch chan Event
closed chan struct{}
userID string
workstationID string
mu sync.RWMutex
itemFilters map[string]struct{}
}
// WatchItem adds an item ID to this client's filter set.
func (c *sseClient) WatchItem(itemID string) {
c.mu.Lock()
c.itemFilters[itemID] = struct{}{}
c.mu.Unlock()
}
// UnwatchItem removes an item ID from this client's filter set.
func (c *sseClient) UnwatchItem(itemID string) {
c.mu.Lock()
delete(c.itemFilters, itemID)
c.mu.Unlock()
}
// IsWatchingItem returns whether this client is watching a specific item.
func (c *sseClient) IsWatchingItem(itemID string) bool {
c.mu.RLock()
_, ok := c.itemFilters[itemID]
c.mu.RUnlock()
return ok
}
const (
@@ -52,10 +78,13 @@ func NewBroker(logger zerolog.Logger) *Broker {
}
// Subscribe adds a new client and returns it. The caller must call Unsubscribe when done.
func (b *Broker) Subscribe() *sseClient {
func (b *Broker) Subscribe(userID, workstationID string) *sseClient {
c := &sseClient{
ch: make(chan Event, clientChanSize),
closed: make(chan struct{}),
ch: make(chan Event, clientChanSize),
closed: make(chan struct{}),
userID: userID,
workstationID: workstationID,
itemFilters: make(map[string]struct{}),
}
b.mu.Lock()
b.clients[c] = struct{}{}
@@ -106,6 +135,49 @@ func (b *Broker) Publish(eventType string, data string) {
b.mu.RUnlock()
}
// publishTargeted sends an event only to clients matching the predicate.
// Targeted events get an ID but are not stored in the history ring buffer.
func (b *Broker) publishTargeted(eventType, data string, match func(*sseClient) bool) {
ev := Event{
ID: b.eventID.Add(1),
Type: eventType,
Data: data,
}
b.mu.RLock()
for c := range b.clients {
if match(c) {
select {
case c.ch <- ev:
default:
b.logger.Warn().Uint64("event_id", ev.ID).Str("type", eventType).Msg("dropped targeted event for slow client")
}
}
}
b.mu.RUnlock()
}
// PublishToItem sends an event only to clients watching a specific item.
func (b *Broker) PublishToItem(itemID, eventType, data string) {
b.publishTargeted(eventType, data, func(c *sseClient) bool {
return c.IsWatchingItem(itemID)
})
}
// PublishToWorkstation sends an event only to the specified workstation.
func (b *Broker) PublishToWorkstation(workstationID, eventType, data string) {
b.publishTargeted(eventType, data, func(c *sseClient) bool {
return c.workstationID == workstationID
})
}
// PublishToUser sends an event to all connections for a specific user.
func (b *Broker) PublishToUser(userID, eventType, data string) {
b.publishTargeted(eventType, data, func(c *sseClient) bool {
return c.userID == userID
})
}
// ClientCount returns the number of connected SSE clients.
func (b *Broker) ClientCount() int {
b.mu.RLock()

View File

@@ -10,7 +10,7 @@ import (
func TestBrokerSubscribeUnsubscribe(t *testing.T) {
b := NewBroker(zerolog.Nop())
c := b.Subscribe()
c := b.Subscribe("", "")
if b.ClientCount() != 1 {
t.Fatalf("expected 1 client, got %d", b.ClientCount())
}
@@ -23,7 +23,7 @@ func TestBrokerSubscribeUnsubscribe(t *testing.T) {
func TestBrokerPublish(t *testing.T) {
b := NewBroker(zerolog.Nop())
c := b.Subscribe()
c := b.Subscribe("", "")
defer b.Unsubscribe(c)
b.Publish("item.created", `{"part_number":"F01-0001"}`)
@@ -46,7 +46,7 @@ func TestBrokerPublish(t *testing.T) {
func TestBrokerPublishDropsSlow(t *testing.T) {
b := NewBroker(zerolog.Nop())
c := b.Subscribe()
c := b.Subscribe("", "")
defer b.Unsubscribe(c)
// Fill the client's channel
@@ -89,9 +89,9 @@ func TestBrokerEventsSince(t *testing.T) {
func TestBrokerClientCount(t *testing.T) {
b := NewBroker(zerolog.Nop())
c1 := b.Subscribe()
c2 := b.Subscribe()
c3 := b.Subscribe()
c1 := b.Subscribe("", "")
c2 := b.Subscribe("", "")
c3 := b.Subscribe("", "")
if b.ClientCount() != 3 {
t.Fatalf("expected 3 clients, got %d", b.ClientCount())
@@ -111,7 +111,7 @@ func TestBrokerClientCount(t *testing.T) {
func TestBrokerShutdown(t *testing.T) {
b := NewBroker(zerolog.Nop())
c := b.Subscribe()
c := b.Subscribe("", "")
b.Shutdown()
@@ -145,3 +145,128 @@ func TestBrokerMonotonicIDs(t *testing.T) {
}
}
}
func TestWatchUnwatchItem(t *testing.T) {
b := NewBroker(zerolog.Nop())
c := b.Subscribe("user1", "ws1")
defer b.Unsubscribe(c)
if c.IsWatchingItem("item-abc") {
t.Fatal("should not be watching item-abc before WatchItem")
}
c.WatchItem("item-abc")
if !c.IsWatchingItem("item-abc") {
t.Fatal("should be watching item-abc after WatchItem")
}
c.UnwatchItem("item-abc")
if c.IsWatchingItem("item-abc") {
t.Fatal("should not be watching item-abc after UnwatchItem")
}
}
func TestPublishToItem(t *testing.T) {
b := NewBroker(zerolog.Nop())
watcher := b.Subscribe("user1", "ws1")
defer b.Unsubscribe(watcher)
bystander := b.Subscribe("user2", "ws2")
defer b.Unsubscribe(bystander)
watcher.WatchItem("item-abc")
b.PublishToItem("item-abc", "edit.started", `{"item_id":"item-abc"}`)
// Watcher should receive the event.
select {
case ev := <-watcher.ch:
if ev.Type != "edit.started" {
t.Fatalf("expected edit.started, got %s", ev.Type)
}
case <-time.After(time.Second):
t.Fatal("watcher did not receive targeted event")
}
// Bystander should not.
select {
case ev := <-bystander.ch:
t.Fatalf("bystander should not receive targeted event, got %s", ev.Type)
case <-time.After(50 * time.Millisecond):
// expected
}
}
func TestPublishToWorkstation(t *testing.T) {
b := NewBroker(zerolog.Nop())
target := b.Subscribe("user1", "ws-target")
defer b.Unsubscribe(target)
other := b.Subscribe("user1", "ws-other")
defer b.Unsubscribe(other)
b.PublishToWorkstation("ws-target", "sync.update", `{"data":"x"}`)
select {
case ev := <-target.ch:
if ev.Type != "sync.update" {
t.Fatalf("expected sync.update, got %s", ev.Type)
}
case <-time.After(time.Second):
t.Fatal("target workstation did not receive event")
}
select {
case ev := <-other.ch:
t.Fatalf("other workstation should not receive event, got %s", ev.Type)
case <-time.After(50 * time.Millisecond):
// expected
}
}
func TestPublishToUser(t *testing.T) {
b := NewBroker(zerolog.Nop())
c1 := b.Subscribe("user1", "ws1")
defer b.Unsubscribe(c1)
c2 := b.Subscribe("user1", "ws2")
defer b.Unsubscribe(c2)
c3 := b.Subscribe("user2", "ws3")
defer b.Unsubscribe(c3)
b.PublishToUser("user1", "user.notify", `{"msg":"hello"}`)
// Both user1 connections should receive.
for _, c := range []*sseClient{c1, c2} {
select {
case ev := <-c.ch:
if ev.Type != "user.notify" {
t.Fatalf("expected user.notify, got %s", ev.Type)
}
case <-time.After(time.Second):
t.Fatal("user1 client did not receive event")
}
}
// user2 should not.
select {
case ev := <-c3.ch:
t.Fatalf("user2 should not receive event, got %s", ev.Type)
case <-time.After(50 * time.Millisecond):
// expected
}
}
func TestTargetedEventsNotInHistory(t *testing.T) {
b := NewBroker(zerolog.Nop())
c := b.Subscribe("user1", "ws1")
defer b.Unsubscribe(c)
c.WatchItem("item-abc")
b.Publish("broadcast", `{}`)
b.PublishToItem("item-abc", "targeted", `{}`)
events := b.EventsSince(0)
if len(events) != 1 {
t.Fatalf("expected 1 event in history (broadcast only), got %d", len(events))
}
if events[0].Type != "broadcast" {
t.Fatalf("expected broadcast event in history, got %s", events[0].Type)
}
}

View File

@@ -51,9 +51,7 @@ var csvColumns = []string{
"category",
"projects", // comma-separated project codes
"sourcing_type",
"sourcing_link",
"long_description",
"standard_cost",
}
// HandleExportCSV exports items to CSV format.
@@ -158,14 +156,8 @@ func (s *Server) HandleExportCSV(w http.ResponseWriter, r *http.Request) {
row[6] = category
row[7] = projectCodes
row[8] = item.SourcingType
if item.SourcingLink != nil {
row[9] = *item.SourcingLink
}
if item.LongDescription != nil {
row[10] = *item.LongDescription
}
if item.StandardCost != nil {
row[11] = strconv.FormatFloat(*item.StandardCost, 'f', -1, 64)
row[9] = *item.LongDescription
}
// Property columns
@@ -366,9 +358,17 @@ func (s *Server) HandleImportCSV(w http.ResponseWriter, r *http.Request) {
// Parse extended fields
sourcingType := getCSVValue(record, colIndex, "sourcing_type")
sourcingLink := getCSVValue(record, colIndex, "sourcing_link")
longDesc := getCSVValue(record, colIndex, "long_description")
stdCostStr := getCSVValue(record, colIndex, "standard_cost")
// sourcing_link and standard_cost are now properties — add to properties map
if sl := getCSVValue(record, colIndex, "sourcing_link"); sl != "" {
properties["sourcing_link"] = sl
}
if sc := getCSVValue(record, colIndex, "standard_cost"); sc != "" {
if cost, err := strconv.ParseFloat(sc, 64); err == nil {
properties["standard_cost"] = cost
}
}
// Create item
item := &db.Item{
@@ -382,17 +382,9 @@ func (s *Server) HandleImportCSV(w http.ResponseWriter, r *http.Request) {
if sourcingType != "" {
item.SourcingType = sourcingType
}
if sourcingLink != "" {
item.SourcingLink = &sourcingLink
}
if longDesc != "" {
item.LongDescription = &longDesc
}
if stdCostStr != "" {
if cost, err := strconv.ParseFloat(stdCostStr, 64); err == nil {
item.StandardCost = &cost
}
}
if err := s.items.Create(ctx, item, properties); err != nil {
result.Errors = append(result.Errors, CSVImportErr{
@@ -585,9 +577,7 @@ func isStandardColumn(col string) bool {
"objects": true, // FreeCAD objects data - skip on import
"archived_at": true,
"sourcing_type": true,
"sourcing_link": true,
"long_description": true,
"standard_cost": true,
}
return standardCols[col]
}

View File

@@ -0,0 +1,260 @@
package api
import (
"bytes"
"encoding/json"
"io"
"mime/multipart"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/testutil"
"github.com/rs/zerolog"
"github.com/kindredsystems/silo/internal/db"
)
// findSchemasDir walks upward to find the project root and returns
// the path to the schemas/ directory.
func findSchemasDir(t *testing.T) string {
t.Helper()
dir, err := os.Getwd()
if err != nil {
t.Fatalf("getting working directory: %v", err)
}
for {
if _, err := os.Stat(filepath.Join(dir, "go.mod")); err == nil {
return filepath.Join(dir, "schemas")
}
parent := filepath.Dir(dir)
if parent == dir {
t.Fatal("could not find project root")
}
dir = parent
}
}
// newTestServerWithSchemas creates a Server backed by a real test DB with schemas loaded.
func newTestServerWithSchemas(t *testing.T) *Server {
t.Helper()
pool := testutil.MustConnectTestPool(t)
database := db.NewFromPool(pool)
broker := NewBroker(zerolog.Nop())
state := NewServerState(zerolog.Nop(), nil, broker)
schemasDir := findSchemasDir(t)
schemas, err := schema.LoadAll(schemasDir)
if err != nil {
t.Fatalf("loading schemas: %v", err)
}
return NewServer(
zerolog.Nop(),
database,
schemas,
schemasDir,
nil, // storage
nil, // authService
nil, // sessionManager
nil, // oidcBackend
nil, // authConfig
broker,
state,
nil, // jobDefs
"", // jobDefsDir
modules.NewRegistry(), // modules
nil, // cfg
nil, // workflows
)
}
func newCSVRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Get("/api/items/export.csv", s.HandleExportCSV)
r.Get("/api/items/template.csv", s.HandleCSVTemplate)
r.Post("/api/items/import", s.HandleImportCSV)
r.Get("/api/items/{partNumber}/bom/export.csv", s.HandleExportBOMCSV)
return r
}
func TestHandleExportCSVEmpty(t *testing.T) {
s := newTestServer(t)
router := newCSVRouter(s)
req := httptest.NewRequest("GET", "/api/items/export.csv", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
ct := w.Header().Get("Content-Type")
if !strings.Contains(ct, "text/csv") {
t.Errorf("content-type: got %q, want text/csv", ct)
}
// Should have header row only
lines := strings.Split(strings.TrimSpace(w.Body.String()), "\n")
if len(lines) != 1 {
t.Errorf("expected 1 line (header only), got %d", len(lines))
}
}
func TestHandleExportCSVWithItems(t *testing.T) {
s := newTestServer(t)
router := newCSVRouter(s)
createItemDirect(t, s, "CSV-001", "first csv item", nil)
createItemDirect(t, s, "CSV-002", "second csv item", nil)
req := httptest.NewRequest("GET", "/api/items/export.csv", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
lines := strings.Split(strings.TrimSpace(w.Body.String()), "\n")
// header + 2 data rows
if len(lines) != 3 {
t.Errorf("expected 3 lines (header + 2 rows), got %d", len(lines))
}
}
func TestHandleCSVTemplate(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newCSVRouter(s)
req := httptest.NewRequest("GET", "/api/items/template.csv?schema=kindred-rd", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
ct := w.Header().Get("Content-Type")
if !strings.Contains(ct, "text/csv") {
t.Errorf("content-type: got %q, want text/csv", ct)
}
// Should contain at least "category" and "description" columns
header := strings.Split(strings.TrimSpace(w.Body.String()), "\n")[0]
if !strings.Contains(header, "category") {
t.Error("template header missing 'category' column")
}
if !strings.Contains(header, "description") {
t.Error("template header missing 'description' column")
}
}
// csvMultipartBody creates a multipart form body with a CSV file and optional form fields.
func csvMultipartBody(t *testing.T, csvContent string, fields map[string]string) (*bytes.Buffer, string) {
t.Helper()
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile("file", "import.csv")
if err != nil {
t.Fatalf("creating form file: %v", err)
}
io.WriteString(part, csvContent)
for k, v := range fields {
writer.WriteField(k, v)
}
writer.Close()
return body, writer.FormDataContentType()
}
func TestHandleImportCSVDryRun(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newCSVRouter(s)
csv := "category,description\nF01,Dry run widget\nF01,Dry run gadget\n"
body, contentType := csvMultipartBody(t, csv, map[string]string{"dry_run": "true"})
req := authRequest(httptest.NewRequest("POST", "/api/items/import", body))
req.Header.Set("Content-Type", contentType)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var result CSVImportResult
if err := json.Unmarshal(w.Body.Bytes(), &result); err != nil {
t.Fatalf("decoding response: %v", err)
}
if result.TotalRows != 2 {
t.Errorf("total_rows: got %d, want 2", result.TotalRows)
}
// Dry run should not create items
if len(result.CreatedItems) != 0 {
t.Errorf("dry run should not create items, got %d", len(result.CreatedItems))
}
}
func TestHandleImportCSVCommit(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newCSVRouter(s)
csv := "category,description\nF01,Committed widget\n"
body, contentType := csvMultipartBody(t, csv, nil)
req := authRequest(httptest.NewRequest("POST", "/api/items/import", body))
req.Header.Set("Content-Type", contentType)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var result CSVImportResult
if err := json.Unmarshal(w.Body.Bytes(), &result); err != nil {
t.Fatalf("decoding response: %v", err)
}
if result.SuccessCount != 1 {
t.Errorf("success_count: got %d, want 1", result.SuccessCount)
}
if len(result.CreatedItems) != 1 {
t.Errorf("created_items: got %d, want 1", len(result.CreatedItems))
}
}
func TestHandleExportBOMCSV(t *testing.T) {
s := newTestServer(t)
router := newCSVRouter(s)
createItemDirect(t, s, "BOMCSV-P", "parent", nil)
createItemDirect(t, s, "BOMCSV-C", "child", nil)
addBOMDirect(t, s, "BOMCSV-P", "BOMCSV-C", 3)
req := httptest.NewRequest("GET", "/api/items/BOMCSV-P/bom/export.csv", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
ct := w.Header().Get("Content-Type")
if !strings.Contains(ct, "text/csv") {
t.Errorf("content-type: got %q, want text/csv", ct)
}
lines := strings.Split(strings.TrimSpace(w.Body.String()), "\n")
// header + 1 BOM entry
if len(lines) != 2 {
t.Errorf("expected 2 lines (header + 1 row), got %d", len(lines))
}
}

View File

@@ -0,0 +1,271 @@
package api
import (
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/db"
)
// dagSyncRequest is the payload for PUT /api/items/{partNumber}/dag.
type dagSyncRequest struct {
RevisionNumber int `json:"revision_number"`
Nodes []dagSyncNode `json:"nodes"`
Edges []dagSyncEdge `json:"edges"`
}
type dagSyncNode struct {
NodeKey string `json:"node_key"`
NodeType string `json:"node_type"`
PropertiesHash *string `json:"properties_hash,omitempty"`
ValidationState string `json:"validation_state,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
}
type dagSyncEdge struct {
SourceKey string `json:"source_key"`
TargetKey string `json:"target_key"`
EdgeType string `json:"edge_type,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
}
// HandleGetDAG returns the feature DAG for an item's current revision.
func (s *Server) HandleGetDAG(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil || item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
nodes, err := s.dag.GetNodes(ctx, item.ID, item.CurrentRevision)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get DAG nodes")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get DAG")
return
}
edges, err := s.dag.GetEdges(ctx, item.ID, item.CurrentRevision)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get DAG edges")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get DAG edges")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"item_id": item.ID,
"part_number": item.PartNumber,
"revision_number": item.CurrentRevision,
"nodes": nodes,
"edges": edges,
})
}
// HandleGetForwardCone returns all downstream dependents of a node.
func (s *Server) HandleGetForwardCone(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
nodeKey := chi.URLParam(r, "nodeKey")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil || item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
node, err := s.dag.GetNodeByKey(ctx, item.ID, item.CurrentRevision, nodeKey)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get DAG node")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get node")
return
}
if node == nil {
writeError(w, http.StatusNotFound, "not_found", "Node not found")
return
}
cone, err := s.dag.GetForwardCone(ctx, node.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get forward cone")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get forward cone")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"root_node": node,
"cone": cone,
})
}
// HandleGetDirtySubgraph returns all non-clean nodes for an item.
func (s *Server) HandleGetDirtySubgraph(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil || item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
nodes, err := s.dag.GetDirtySubgraph(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get dirty subgraph")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get dirty subgraph")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"item_id": item.ID,
"nodes": nodes,
})
}
// HandleSyncDAG accepts a full feature tree from a client or runner.
func (s *Server) HandleSyncDAG(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil || item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
var req dagSyncRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if req.RevisionNumber == 0 {
req.RevisionNumber = item.CurrentRevision
}
// Convert request nodes to DB nodes
nodes := make([]db.DAGNode, len(req.Nodes))
for i, n := range req.Nodes {
state := n.ValidationState
if state == "" {
state = "clean"
}
nodes[i] = db.DAGNode{
NodeKey: n.NodeKey,
NodeType: n.NodeType,
PropertiesHash: n.PropertiesHash,
ValidationState: state,
Metadata: n.Metadata,
}
}
// Sync nodes first to get IDs
if err := s.dag.SyncFeatureTree(ctx, item.ID, req.RevisionNumber, nodes, nil); err != nil {
s.logger.Error().Err(err).Msg("failed to sync DAG nodes")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to sync DAG")
return
}
// Build key→ID map from synced nodes
keyToID := make(map[string]string, len(nodes))
for _, n := range nodes {
keyToID[n.NodeKey] = n.ID
}
// Convert request edges, resolving keys to IDs
edges := make([]db.DAGEdge, len(req.Edges))
for i, e := range req.Edges {
sourceID, ok := keyToID[e.SourceKey]
if !ok {
writeError(w, http.StatusBadRequest, "invalid_edge",
"Unknown source_key: "+e.SourceKey)
return
}
targetID, ok := keyToID[e.TargetKey]
if !ok {
writeError(w, http.StatusBadRequest, "invalid_edge",
"Unknown target_key: "+e.TargetKey)
return
}
edgeType := e.EdgeType
if edgeType == "" {
edgeType = "depends_on"
}
edges[i] = db.DAGEdge{
SourceNodeID: sourceID,
TargetNodeID: targetID,
EdgeType: edgeType,
Metadata: e.Metadata,
}
}
// Sync edges (nodes already synced, so pass empty nodes to skip re-upsert)
if len(edges) > 0 {
// Delete old edges and insert new ones
if err := s.dag.DeleteEdgesForItem(ctx, item.ID, req.RevisionNumber); err != nil {
s.logger.Error().Err(err).Msg("failed to delete old edges")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to sync DAG edges")
return
}
for i := range edges {
if err := s.dag.CreateEdge(ctx, &edges[i]); err != nil {
s.logger.Error().Err(err).Msg("failed to create edge")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to create edge")
return
}
}
}
// Publish SSE event
s.broker.Publish("dag.updated", mustMarshal(map[string]any{
"item_id": item.ID,
"part_number": item.PartNumber,
"revision_number": req.RevisionNumber,
"node_count": len(req.Nodes),
"edge_count": len(req.Edges),
}))
writeJSON(w, http.StatusOK, map[string]any{
"synced": true,
"node_count": len(req.Nodes),
"edge_count": len(req.Edges),
})
}
// HandleMarkDirty marks a node and all its downstream dependents as dirty.
func (s *Server) HandleMarkDirty(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
nodeKey := chi.URLParam(r, "nodeKey")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil || item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
node, err := s.dag.GetNodeByKey(ctx, item.ID, item.CurrentRevision, nodeKey)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get DAG node")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get node")
return
}
if node == nil {
writeError(w, http.StatusNotFound, "not_found", "Node not found")
return
}
affected, err := s.dag.MarkDirty(ctx, node.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to mark dirty")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to mark dirty")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"node_key": nodeKey,
"nodes_affected": affected,
})
}

View File

@@ -0,0 +1,250 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/testutil"
"github.com/rs/zerolog"
)
func newDAGTestServer(t *testing.T) *Server {
t.Helper()
pool := testutil.MustConnectTestPool(t)
database := db.NewFromPool(pool)
broker := NewBroker(zerolog.Nop())
state := NewServerState(zerolog.Nop(), nil, broker)
return NewServer(
zerolog.Nop(),
database,
map[string]*schema.Schema{},
"",
nil, nil, nil, nil, nil,
broker, state,
nil, "",
modules.NewRegistry(), nil,
nil,
)
}
func newDAGRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Route("/api/items/{partNumber}", func(r chi.Router) {
r.Get("/dag", s.HandleGetDAG)
r.Get("/dag/forward-cone/{nodeKey}", s.HandleGetForwardCone)
r.Get("/dag/dirty", s.HandleGetDirtySubgraph)
r.Put("/dag", s.HandleSyncDAG)
r.Post("/dag/mark-dirty/{nodeKey}", s.HandleMarkDirty)
})
return r
}
func TestHandleGetDAG_Empty(t *testing.T) {
s := newDAGTestServer(t)
r := newDAGRouter(s)
// Create an item
item := &db.Item{PartNumber: "DAG-TEST-001", ItemType: "part", Description: "DAG test"}
if err := s.items.Create(context.Background(), item, nil); err != nil {
t.Fatalf("creating item: %v", err)
}
req := httptest.NewRequest("GET", "/api/items/DAG-TEST-001/dag", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
var resp map[string]any
json.Unmarshal(w.Body.Bytes(), &resp)
if resp["part_number"] != "DAG-TEST-001" {
t.Errorf("expected part_number DAG-TEST-001, got %v", resp["part_number"])
}
}
func TestHandleSyncDAG(t *testing.T) {
s := newDAGTestServer(t)
r := newDAGRouter(s)
// Create an item with a revision
item := &db.Item{PartNumber: "DAG-SYNC-001", ItemType: "part", Description: "sync test"}
if err := s.items.Create(context.Background(), item, nil); err != nil {
t.Fatalf("creating item: %v", err)
}
// Sync a feature tree
body := `{
"nodes": [
{"node_key": "Sketch001", "node_type": "sketch"},
{"node_key": "Pad001", "node_type": "pad"},
{"node_key": "Fillet001", "node_type": "fillet"}
],
"edges": [
{"source_key": "Sketch001", "target_key": "Pad001", "edge_type": "depends_on"},
{"source_key": "Pad001", "target_key": "Fillet001", "edge_type": "depends_on"}
]
}`
req := httptest.NewRequest("PUT", "/api/items/DAG-SYNC-001/dag", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
var resp map[string]any
json.Unmarshal(w.Body.Bytes(), &resp)
if resp["node_count"] != float64(3) {
t.Errorf("expected 3 nodes, got %v", resp["node_count"])
}
if resp["edge_count"] != float64(2) {
t.Errorf("expected 2 edges, got %v", resp["edge_count"])
}
// Verify we can read the DAG back
req2 := httptest.NewRequest("GET", "/api/items/DAG-SYNC-001/dag", nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("GET dag: expected 200, got %d", w2.Code)
}
var dagResp map[string]any
json.Unmarshal(w2.Body.Bytes(), &dagResp)
nodes, ok := dagResp["nodes"].([]any)
if !ok || len(nodes) != 3 {
t.Errorf("expected 3 nodes in GET, got %v", dagResp["nodes"])
}
}
func TestHandleForwardCone(t *testing.T) {
s := newDAGTestServer(t)
r := newDAGRouter(s)
item := &db.Item{PartNumber: "DAG-CONE-001", ItemType: "part", Description: "cone test"}
if err := s.items.Create(context.Background(), item, nil); err != nil {
t.Fatalf("creating item: %v", err)
}
// Sync a linear chain: A -> B -> C
body := `{
"nodes": [
{"node_key": "A", "node_type": "sketch"},
{"node_key": "B", "node_type": "pad"},
{"node_key": "C", "node_type": "fillet"}
],
"edges": [
{"source_key": "A", "target_key": "B"},
{"source_key": "B", "target_key": "C"}
]
}`
req := httptest.NewRequest("PUT", "/api/items/DAG-CONE-001/dag", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("sync: %d %s", w.Code, w.Body.String())
}
// Forward cone from A should include B and C
req2 := httptest.NewRequest("GET", "/api/items/DAG-CONE-001/dag/forward-cone/A", nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("forward-cone: %d %s", w2.Code, w2.Body.String())
}
var resp map[string]any
json.Unmarshal(w2.Body.Bytes(), &resp)
cone, ok := resp["cone"].([]any)
if !ok || len(cone) != 2 {
t.Errorf("expected 2 nodes in forward cone, got %v", resp["cone"])
}
}
func TestHandleMarkDirty(t *testing.T) {
s := newDAGTestServer(t)
r := newDAGRouter(s)
item := &db.Item{PartNumber: "DAG-DIRTY-001", ItemType: "part", Description: "dirty test"}
if err := s.items.Create(context.Background(), item, nil); err != nil {
t.Fatalf("creating item: %v", err)
}
// Sync: A -> B -> C
body := `{
"nodes": [
{"node_key": "X", "node_type": "sketch"},
{"node_key": "Y", "node_type": "pad"},
{"node_key": "Z", "node_type": "fillet"}
],
"edges": [
{"source_key": "X", "target_key": "Y"},
{"source_key": "Y", "target_key": "Z"}
]
}`
req := httptest.NewRequest("PUT", "/api/items/DAG-DIRTY-001/dag", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("sync: %d %s", w.Code, w.Body.String())
}
// Mark X dirty — should propagate to Y and Z
req2 := httptest.NewRequest("POST", "/api/items/DAG-DIRTY-001/dag/mark-dirty/X", nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("mark-dirty: %d %s", w2.Code, w2.Body.String())
}
var resp map[string]any
json.Unmarshal(w2.Body.Bytes(), &resp)
affected := resp["nodes_affected"].(float64)
if affected != 3 {
t.Errorf("expected 3 nodes affected, got %v", affected)
}
// Verify dirty subgraph
req3 := httptest.NewRequest("GET", "/api/items/DAG-DIRTY-001/dag/dirty", nil)
w3 := httptest.NewRecorder()
r.ServeHTTP(w3, req3)
if w3.Code != http.StatusOK {
t.Fatalf("dirty: %d %s", w3.Code, w3.Body.String())
}
var dirtyResp map[string]any
json.Unmarshal(w3.Body.Bytes(), &dirtyResp)
dirtyNodes, ok := dirtyResp["nodes"].([]any)
if !ok || len(dirtyNodes) != 3 {
t.Errorf("expected 3 dirty nodes, got %v", dirtyResp["nodes"])
}
}
func TestHandleGetDAG_NotFound(t *testing.T) {
s := newDAGTestServer(t)
r := newDAGRouter(s)
req := httptest.NewRequest("GET", "/api/items/NONEXISTENT-999/dag", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("expected 404, got %d", w.Code)
}
}

View File

@@ -0,0 +1,125 @@
package api
import (
"net/http"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/storage"
)
// DependencyResponse is the JSON representation for GET /dependencies.
type DependencyResponse struct {
UUID string `json:"uuid"`
PartNumber *string `json:"part_number"`
Revision *int `json:"revision"`
Quantity *float64 `json:"quantity"`
Label *string `json:"label"`
Relationship string `json:"relationship"`
}
// ResolvedDependencyResponse is the JSON representation for GET /dependencies/resolve.
type ResolvedDependencyResponse struct {
UUID string `json:"uuid"`
PartNumber *string `json:"part_number"`
Label *string `json:"label"`
Revision *int `json:"revision"`
Quantity *float64 `json:"quantity"`
Resolved bool `json:"resolved"`
FileAvailable bool `json:"file_available"`
}
// HandleGetDependencies returns the raw dependency list for an item.
// GET /api/items/{partNumber}/dependencies
func (s *Server) HandleGetDependencies(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
deps, err := s.deps.ListByItem(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list dependencies")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list dependencies")
return
}
resp := make([]DependencyResponse, len(deps))
for i, d := range deps {
resp[i] = DependencyResponse{
UUID: d.ChildUUID,
PartNumber: d.ChildPartNumber,
Revision: d.ChildRevision,
Quantity: d.Quantity,
Label: d.Label,
Relationship: d.Relationship,
}
}
writeJSON(w, http.StatusOK, resp)
}
// HandleResolveDependencies returns dependencies with UUIDs resolved to part numbers
// and file availability status.
// GET /api/items/{partNumber}/dependencies/resolve
func (s *Server) HandleResolveDependencies(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
deps, err := s.deps.Resolve(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to resolve dependencies")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to resolve dependencies")
return
}
resp := make([]ResolvedDependencyResponse, len(deps))
for i, d := range deps {
// Use resolved part number if available, fall back to .kc-provided value.
pn := d.ChildPartNumber
rev := d.ChildRevision
if d.Resolved {
pn = d.ResolvedPartNumber
rev = d.ResolvedRevision
}
fileAvailable := false
if d.Resolved && pn != nil && rev != nil && s.storage != nil {
key := storage.FileKey(*pn, *rev)
if exists, err := s.storage.Exists(ctx, key); err == nil {
fileAvailable = exists
}
}
resp[i] = ResolvedDependencyResponse{
UUID: d.ChildUUID,
PartNumber: pn,
Label: d.Label,
Revision: rev,
Quantity: d.Quantity,
Resolved: d.Resolved,
FileAvailable: fileAvailable,
}
}
writeJSON(w, http.StatusOK, resp)
}

View File

@@ -3,7 +3,9 @@ package api
import (
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
@@ -19,7 +21,7 @@ type presignUploadRequest struct {
Size int64 `json:"size"`
}
// HandlePresignUpload generates a presigned PUT URL for direct browser upload to MinIO.
// HandlePresignUpload generates a presigned PUT URL for direct browser upload.
func (s *Server) HandlePresignUpload(w http.ResponseWriter, r *http.Request) {
if s.storage == nil {
writeError(w, http.StatusServiceUnavailable, "storage_unavailable", "File storage not configured")
@@ -314,3 +316,185 @@ func (s *Server) HandleSetItemThumbnail(w http.ResponseWriter, r *http.Request)
w.WriteHeader(http.StatusNoContent)
}
// storageBackend returns the configured storage backend name.
func (s *Server) storageBackend() string {
return "filesystem"
}
// HandleUploadItemFile accepts a multipart file upload and stores it as an item attachment.
func (s *Server) HandleUploadItemFile(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
if s.storage == nil {
writeError(w, http.StatusServiceUnavailable, "storage_unavailable", "File storage not configured")
return
}
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
// Parse multipart form (max 500MB)
if err := r.ParseMultipartForm(500 << 20); err != nil {
writeError(w, http.StatusBadRequest, "invalid_form", err.Error())
return
}
file, header, err := r.FormFile("file")
if err != nil {
writeError(w, http.StatusBadRequest, "missing_file", "File is required")
return
}
defer file.Close()
contentType := header.Header.Get("Content-Type")
if contentType == "" {
contentType = "application/octet-stream"
}
// Generate permanent key
fileID := uuid.New().String()
permanentKey := fmt.Sprintf("items/%s/files/%s/%s", item.ID, fileID, header.Filename)
// Write directly to storage
result, err := s.storage.Put(ctx, permanentKey, file, header.Size, contentType)
if err != nil {
s.logger.Error().Err(err).Msg("failed to upload file")
writeError(w, http.StatusInternalServerError, "upload_failed", "Failed to store file")
return
}
// Create DB record
itemFile := &db.ItemFile{
ItemID: item.ID,
Filename: header.Filename,
ContentType: contentType,
Size: result.Size,
ObjectKey: permanentKey,
StorageBackend: s.storageBackend(),
}
if err := s.itemFiles.Create(ctx, itemFile); err != nil {
s.logger.Error().Err(err).Msg("failed to create item file record")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to save file record")
return
}
s.logger.Info().
Str("part_number", partNumber).
Str("file_id", itemFile.ID).
Str("filename", header.Filename).
Int64("size", result.Size).
Msg("file uploaded to item")
writeJSON(w, http.StatusCreated, itemFileToResponse(itemFile))
}
// HandleUploadItemThumbnail accepts a multipart file upload and sets it as the item thumbnail.
func (s *Server) HandleUploadItemThumbnail(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
if s.storage == nil {
writeError(w, http.StatusServiceUnavailable, "storage_unavailable", "File storage not configured")
return
}
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
// Parse multipart form (max 10MB for thumbnails)
if err := r.ParseMultipartForm(10 << 20); err != nil {
writeError(w, http.StatusBadRequest, "invalid_form", err.Error())
return
}
file, header, err := r.FormFile("file")
if err != nil {
writeError(w, http.StatusBadRequest, "missing_file", "File is required")
return
}
defer file.Close()
contentType := header.Header.Get("Content-Type")
if contentType == "" {
contentType = "image/png"
}
thumbnailKey := fmt.Sprintf("items/%s/thumbnail.png", item.ID)
if _, err := s.storage.Put(ctx, thumbnailKey, file, header.Size, contentType); err != nil {
s.logger.Error().Err(err).Msg("failed to upload thumbnail")
writeError(w, http.StatusInternalServerError, "upload_failed", "Failed to store thumbnail")
return
}
if err := s.items.SetThumbnailKey(ctx, item.ID, thumbnailKey); err != nil {
s.logger.Error().Err(err).Msg("failed to update thumbnail key")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to save thumbnail")
return
}
w.WriteHeader(http.StatusNoContent)
}
// HandleDownloadItemFile streams an item file attachment to the client.
func (s *Server) HandleDownloadItemFile(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
fileID := chi.URLParam(r, "fileId")
if s.storage == nil {
writeError(w, http.StatusServiceUnavailable, "storage_unavailable", "File storage not configured")
return
}
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil || item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
file, err := s.itemFiles.Get(ctx, fileID)
if err != nil {
writeError(w, http.StatusNotFound, "not_found", "File not found")
return
}
if file.ItemID != item.ID {
writeError(w, http.StatusNotFound, "not_found", "File not found")
return
}
reader, err := s.storage.Get(ctx, file.ObjectKey)
if err != nil {
s.logger.Error().Err(err).Str("key", file.ObjectKey).Msg("failed to get file")
writeError(w, http.StatusInternalServerError, "download_failed", "Failed to retrieve file")
return
}
defer reader.Close()
w.Header().Set("Content-Type", file.ContentType)
w.Header().Set("Content-Disposition", fmt.Sprintf(`attachment; filename="%s"`, file.Filename))
if file.Size > 0 {
w.Header().Set("Content-Length", strconv.FormatInt(file.Size, 10))
}
io.Copy(w, reader)
}

View File

@@ -0,0 +1,186 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/db"
)
// newFileRouter creates a chi router with file-related routes for testing.
func newFileRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Route("/api/items/{partNumber}", func(r chi.Router) {
r.Get("/files", s.HandleListItemFiles)
r.Post("/files", s.HandleAssociateItemFile)
r.Delete("/files/{fileId}", s.HandleDeleteItemFile)
r.Put("/thumbnail", s.HandleSetItemThumbnail)
r.Post("/file", s.HandleUploadFile)
r.Get("/file/{revision}", s.HandleDownloadFile)
})
r.Post("/api/uploads/presign", s.HandlePresignUpload)
return r
}
// createFileDirect creates a file record directly via the DB for test setup.
func createFileDirect(t *testing.T, s *Server, itemID, filename string) *db.ItemFile {
t.Helper()
f := &db.ItemFile{
ItemID: itemID,
Filename: filename,
ContentType: "application/octet-stream",
Size: 1024,
ObjectKey: "items/" + itemID + "/files/" + filename,
}
if err := s.itemFiles.Create(context.Background(), f); err != nil {
t.Fatalf("creating file %s: %v", filename, err)
}
return f
}
func TestHandleListItemFiles(t *testing.T) {
s := newTestServer(t)
router := newFileRouter(s)
createItemDirect(t, s, "FAPI-001", "file list item", nil)
item, _ := s.items.GetByPartNumber(context.Background(), "FAPI-001")
createFileDirect(t, s, item.ID, "drawing.pdf")
createFileDirect(t, s, item.ID, "model.step")
req := httptest.NewRequest("GET", "/api/items/FAPI-001/files", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var files []itemFileResponse
if err := json.Unmarshal(w.Body.Bytes(), &files); err != nil {
t.Fatalf("decoding response: %v", err)
}
if len(files) != 2 {
t.Errorf("expected 2 files, got %d", len(files))
}
}
func TestHandleListItemFilesNotFound(t *testing.T) {
s := newTestServer(t)
router := newFileRouter(s)
req := httptest.NewRequest("GET", "/api/items/NONEXISTENT/files", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("status: got %d, want %d", w.Code, http.StatusNotFound)
}
}
func TestHandleDeleteItemFile(t *testing.T) {
s := newTestServer(t)
router := newFileRouter(s)
createItemDirect(t, s, "FDEL-API-001", "delete file item", nil)
item, _ := s.items.GetByPartNumber(context.Background(), "FDEL-API-001")
f := createFileDirect(t, s, item.ID, "removable.bin")
req := authRequest(httptest.NewRequest("DELETE", "/api/items/FDEL-API-001/files/"+f.ID, nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNoContent {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusNoContent, w.Body.String())
}
}
func TestHandleDeleteItemFileCrossItem(t *testing.T) {
s := newTestServer(t)
router := newFileRouter(s)
// Create two items, attach file to item A
createItemDirect(t, s, "CROSS-A", "item A", nil)
createItemDirect(t, s, "CROSS-B", "item B", nil)
itemA, _ := s.items.GetByPartNumber(context.Background(), "CROSS-A")
f := createFileDirect(t, s, itemA.ID, "belongs-to-a.pdf")
// Try to delete via item B — should fail
req := authRequest(httptest.NewRequest("DELETE", "/api/items/CROSS-B/files/"+f.ID, nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("status: got %d, want %d", w.Code, http.StatusNotFound)
}
}
func TestHandlePresignUploadNoStorage(t *testing.T) {
s := newTestServer(t) // storage is nil
router := newFileRouter(s)
body := `{"filename":"test.bin","content_type":"application/octet-stream","size":1024}`
req := authRequest(httptest.NewRequest("POST", "/api/uploads/presign", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusServiceUnavailable {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusServiceUnavailable, w.Body.String())
}
}
func TestHandleUploadFileNoStorage(t *testing.T) {
s := newTestServer(t) // storage is nil
router := newFileRouter(s)
createItemDirect(t, s, "UPNS-001", "upload no storage", nil)
req := authRequest(httptest.NewRequest("POST", "/api/items/UPNS-001/file", strings.NewReader("fake")))
req.Header.Set("Content-Type", "multipart/form-data")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusServiceUnavailable {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusServiceUnavailable, w.Body.String())
}
}
func TestHandleAssociateFileNoStorage(t *testing.T) {
s := newTestServer(t) // storage is nil
router := newFileRouter(s)
createItemDirect(t, s, "ASSNS-001", "associate no storage", nil)
body := `{"object_key":"uploads/tmp/abc/test.bin","filename":"test.bin"}`
req := authRequest(httptest.NewRequest("POST", "/api/items/ASSNS-001/files", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusServiceUnavailable {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusServiceUnavailable, w.Body.String())
}
}
func TestHandleSetThumbnailNoStorage(t *testing.T) {
s := newTestServer(t) // storage is nil
router := newFileRouter(s)
createItemDirect(t, s, "THNS-001", "thumbnail no storage", nil)
body := `{"object_key":"uploads/tmp/abc/thumb.png"}`
req := authRequest(httptest.NewRequest("PUT", "/api/items/THNS-001/thumbnail", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusServiceUnavailable {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusServiceUnavailable, w.Body.String())
}
}

View File

@@ -5,9 +5,11 @@ import (
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
@@ -17,9 +19,13 @@ import (
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/config"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/jobdef"
"github.com/kindredsystems/silo/internal/kc"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/partnum"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/storage"
"github.com/kindredsystems/silo/internal/workflow"
"github.com/rs/zerolog"
"gopkg.in/yaml.v3"
)
@@ -34,7 +40,7 @@ type Server struct {
schemas map[string]*schema.Schema
schemasDir string
partgen *partnum.Generator
storage *storage.Storage
storage storage.FileStore
auth *auth.Service
sessions *scs.SessionManager
oidc *auth.OIDCBackend
@@ -42,6 +48,22 @@ type Server struct {
itemFiles *db.ItemFileRepository
broker *Broker
serverState *ServerState
dag *db.DAGRepository
jobs *db.JobRepository
locations *db.LocationRepository
jobDefs map[string]*jobdef.Definition
jobDefsDir string
modules *modules.Registry
cfg *config.Config
settings *db.SettingsRepository
metadata *db.ItemMetadataRepository
deps *db.ItemDependencyRepository
macros *db.ItemMacroRepository
approvals *db.ItemApprovalRepository
workflows map[string]*workflow.Workflow
solverResults *db.SolverResultRepository
workstations *db.WorkstationRepository
editSessions *db.EditSessionRepository
}
// NewServer creates a new API server.
@@ -50,18 +72,34 @@ func NewServer(
database *db.DB,
schemas map[string]*schema.Schema,
schemasDir string,
store *storage.Storage,
store storage.FileStore,
authService *auth.Service,
sessionManager *scs.SessionManager,
oidcBackend *auth.OIDCBackend,
authCfg *config.AuthConfig,
broker *Broker,
state *ServerState,
jobDefs map[string]*jobdef.Definition,
jobDefsDir string,
registry *modules.Registry,
cfg *config.Config,
workflows map[string]*workflow.Workflow,
) *Server {
items := db.NewItemRepository(database)
projects := db.NewProjectRepository(database)
relationships := db.NewRelationshipRepository(database)
itemFiles := db.NewItemFileRepository(database)
dag := db.NewDAGRepository(database)
jobs := db.NewJobRepository(database)
settings := db.NewSettingsRepository(database)
locations := db.NewLocationRepository(database)
metadata := db.NewItemMetadataRepository(database)
itemDeps := db.NewItemDependencyRepository(database)
itemMacros := db.NewItemMacroRepository(database)
itemApprovals := db.NewItemApprovalRepository(database)
solverResults := db.NewSolverResultRepository(database)
workstations := db.NewWorkstationRepository(database)
editSessions := db.NewEditSessionRepository(database)
seqStore := &dbSequenceStore{db: database, schemas: schemas}
partgen := partnum.NewGenerator(schemas, seqStore)
@@ -82,6 +120,22 @@ func NewServer(
itemFiles: itemFiles,
broker: broker,
serverState: state,
dag: dag,
jobs: jobs,
locations: locations,
jobDefs: jobDefs,
jobDefsDir: jobDefsDir,
modules: registry,
cfg: cfg,
settings: settings,
metadata: metadata,
deps: itemDeps,
macros: itemMacros,
approvals: itemApprovals,
workflows: workflows,
solverResults: solverResults,
workstations: workstations,
editSessions: editSessions,
}
}
@@ -152,6 +206,54 @@ func (s *Server) HandleReady(w http.ResponseWriter, r *http.Request) {
})
}
// HandleGetModules returns the public module discovery response.
// No authentication required — clients call this pre-login.
func (s *Server) HandleGetModules(w http.ResponseWriter, r *http.Request) {
mods := make(map[string]any, 10)
for _, m := range s.modules.All() {
entry := map[string]any{
"enabled": s.modules.IsEnabled(m.ID),
"required": m.Required,
"name": m.Name,
}
if m.Version != "" {
entry["version"] = m.Version
}
if len(m.DependsOn) > 0 {
entry["depends_on"] = m.DependsOn
}
// Public config (non-secret) for specific modules.
switch m.ID {
case "auth":
if s.cfg != nil {
entry["config"] = map[string]any{
"local_enabled": s.cfg.Auth.Local.Enabled,
"ldap_enabled": s.cfg.Auth.LDAP.Enabled,
"oidc_enabled": s.cfg.Auth.OIDC.Enabled,
"oidc_issuer_url": s.cfg.Auth.OIDC.IssuerURL,
}
}
case "freecad":
if s.cfg != nil {
entry["config"] = map[string]any{
"uri_scheme": s.cfg.FreeCAD.URIScheme,
}
}
}
mods[m.ID] = entry
}
writeJSON(w, http.StatusOK, map[string]any{
"modules": mods,
"server": map[string]any{
"version": "0.2",
"read_only": s.serverState.IsReadOnly(),
},
})
}
// Schema handlers
// SchemaResponse represents a schema in API responses.
@@ -195,10 +297,35 @@ func (s *Server) HandleGetSchema(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, schemaToResponse(sch))
}
// HandleGetPropertySchema returns the property schema for a category.
func (s *Server) HandleGetPropertySchema(w http.ResponseWriter, r *http.Request) {
// FormFieldDescriptor describes a single field in the form descriptor response.
type FormFieldDescriptor struct {
Name string `json:"name"`
Type string `json:"type"`
Widget string `json:"widget,omitempty"`
Label string `json:"label"`
Required bool `json:"required,omitempty"`
Default any `json:"default,omitempty"`
Unit string `json:"unit,omitempty"`
Description string `json:"description,omitempty"`
Options []string `json:"options,omitempty"`
Currency string `json:"currency,omitempty"`
// Item-field specific
DerivedFromCategory map[string]string `json:"derived_from_category,omitempty"`
SearchEndpoint string `json:"search_endpoint,omitempty"`
}
// FormFieldGroupDescriptor describes an ordered group of resolved fields.
type FormFieldGroupDescriptor struct {
Key string `json:"key"`
Label string `json:"label"`
Order int `json:"order"`
Fields []FormFieldDescriptor `json:"fields"`
}
// HandleGetFormDescriptor returns the full form descriptor for a schema.
func (s *Server) HandleGetFormDescriptor(w http.ResponseWriter, r *http.Request) {
schemaName := chi.URLParam(r, "name")
category := r.URL.Query().Get("category")
sch, ok := s.schemas[schemaName]
if !ok {
@@ -206,19 +333,194 @@ func (s *Server) HandleGetPropertySchema(w http.ResponseWriter, r *http.Request)
return
}
if sch.PropertySchemas == nil {
writeJSON(w, http.StatusOK, map[string]any{
"version": 0,
"properties": map[string]any{},
})
return
result := map[string]any{
"schema_name": sch.Name,
"format": sch.Format,
}
props := sch.PropertySchemas.GetPropertiesForCategory(category)
writeJSON(w, http.StatusOK, map[string]any{
"version": sch.PropertySchemas.Version,
"properties": props,
// Category picker with auto-derived values_by_domain
if sch.UI != nil && sch.UI.CategoryPicker != nil {
picker := map[string]any{
"style": sch.UI.CategoryPicker.Style,
}
vbd := sch.ValuesByDomain()
stages := make([]map[string]any, 0, len(sch.UI.CategoryPicker.Stages)+1)
for _, stage := range sch.UI.CategoryPicker.Stages {
stg := map[string]any{
"name": stage.Name,
"label": stage.Label,
}
if stage.Values != nil {
stg["values"] = stage.Values
}
stages = append(stages, stg)
}
// Auto-add subcategory stage from values_by_domain
if vbd != nil {
stages = append(stages, map[string]any{
"name": "subcategory",
"label": "Type",
"values_by_domain": vbd,
})
}
picker["stages"] = stages
result["category_picker"] = picker
}
// Item fields
if sch.UI != nil && sch.UI.ItemFields != nil {
result["item_fields"] = sch.UI.ItemFields
}
// Resolve field groups into ordered list with full field metadata
if sch.UI != nil && sch.UI.FieldGroups != nil {
groups := s.resolveFieldGroups(sch, sch.UI.FieldGroups)
result["field_groups"] = groups
}
// Category field groups
if sch.UI != nil && sch.UI.CategoryFieldGroups != nil {
catGroups := make(map[string][]FormFieldGroupDescriptor)
for prefix, groups := range sch.UI.CategoryFieldGroups {
catGroups[prefix] = s.resolveCategoryFieldGroups(sch, prefix, groups)
}
result["category_field_groups"] = catGroups
}
// Field overrides (pass through)
if sch.UI != nil && sch.UI.FieldOverrides != nil {
result["field_overrides"] = sch.UI.FieldOverrides
}
writeJSON(w, http.StatusOK, result)
}
// resolveFieldGroups converts field group definitions into fully resolved descriptors.
func (s *Server) resolveFieldGroups(sch *schema.Schema, groups map[string]schema.FieldGroup) []FormFieldGroupDescriptor {
result := make([]FormFieldGroupDescriptor, 0, len(groups))
for key, group := range groups {
desc := FormFieldGroupDescriptor{
Key: key,
Label: group.Label,
Order: group.Order,
}
for _, fieldName := range group.Fields {
fd := s.resolveField(sch, fieldName)
desc.Fields = append(desc.Fields, fd)
}
result = append(result, desc)
}
// Sort by order
sort.Slice(result, func(i, j int) bool {
return result[i].Order < result[j].Order
})
return result
}
// resolveCategoryFieldGroups resolves category-specific field groups.
func (s *Server) resolveCategoryFieldGroups(sch *schema.Schema, prefix string, groups map[string]schema.FieldGroup) []FormFieldGroupDescriptor {
result := make([]FormFieldGroupDescriptor, 0, len(groups))
for key, group := range groups {
desc := FormFieldGroupDescriptor{
Key: key,
Label: group.Label,
Order: group.Order,
}
for _, fieldName := range group.Fields {
fd := s.resolveCategoryField(sch, prefix, fieldName)
desc.Fields = append(desc.Fields, fd)
}
result = append(result, desc)
}
sort.Slice(result, func(i, j int) bool {
return result[i].Order < result[j].Order
})
return result
}
// resolveField builds a FormFieldDescriptor from item_fields or property_schemas.defaults.
func (s *Server) resolveField(sch *schema.Schema, name string) FormFieldDescriptor {
fd := FormFieldDescriptor{Name: name}
// Check item_fields first
if sch.UI != nil && sch.UI.ItemFields != nil {
if def, ok := sch.UI.ItemFields[name]; ok {
fd.Type = def.Type
fd.Widget = def.Widget
fd.Label = def.Label
fd.Required = def.Required
fd.Default = def.Default
fd.Options = def.Options
fd.DerivedFromCategory = def.DerivedFromCategory
fd.SearchEndpoint = def.SearchEndpoint
s.applyOverrides(sch, name, &fd)
return fd
}
}
// Check property_schemas.defaults
if sch.PropertySchemas != nil && sch.PropertySchemas.Defaults != nil {
if def, ok := sch.PropertySchemas.Defaults[name]; ok {
fd.Type = def.Type
fd.Label = name // Use field name as label if not overridden
fd.Default = def.Default
fd.Unit = def.Unit
fd.Description = def.Description
fd.Required = def.Required
s.applyOverrides(sch, name, &fd)
return fd
}
}
// Fallback — field name only
fd.Label = name
fd.Type = "string"
s.applyOverrides(sch, name, &fd)
return fd
}
// resolveCategoryField builds a FormFieldDescriptor from category-specific property schema.
func (s *Server) resolveCategoryField(sch *schema.Schema, prefix, name string) FormFieldDescriptor {
fd := FormFieldDescriptor{Name: name, Label: name, Type: "string"}
if sch.PropertySchemas != nil {
if catProps, ok := sch.PropertySchemas.Categories[prefix]; ok {
if def, ok := catProps[name]; ok {
fd.Type = def.Type
fd.Default = def.Default
fd.Unit = def.Unit
fd.Description = def.Description
fd.Required = def.Required
}
}
}
s.applyOverrides(sch, name, &fd)
return fd
}
// applyOverrides applies field_overrides to a field descriptor.
func (s *Server) applyOverrides(sch *schema.Schema, name string, fd *FormFieldDescriptor) {
if sch.UI == nil || sch.UI.FieldOverrides == nil {
return
}
ov, ok := sch.UI.FieldOverrides[name]
if !ok {
return
}
if ov.Widget != "" {
fd.Widget = ov.Widget
}
if ov.Currency != "" {
fd.Currency = ov.Currency
}
if len(ov.Options) > 0 {
fd.Options = ov.Options
}
}
func schemaToResponse(sch *schema.Schema) SchemaResponse {
@@ -256,10 +558,10 @@ type ItemResponse struct {
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
SourcingType string `json:"sourcing_type"`
SourcingLink *string `json:"sourcing_link,omitempty"`
LongDescription *string `json:"long_description,omitempty"`
StandardCost *float64 `json:"standard_cost,omitempty"`
ThumbnailKey *string `json:"thumbnail_key,omitempty"`
FileCount int `json:"file_count"`
FilesTotalSize int64 `json:"files_total_size"`
Properties map[string]any `json:"properties,omitempty"`
}
@@ -271,9 +573,7 @@ type CreateItemRequest struct {
Projects []string `json:"projects,omitempty"`
Properties map[string]any `json:"properties,omitempty"`
SourcingType string `json:"sourcing_type,omitempty"`
SourcingLink *string `json:"sourcing_link,omitempty"`
LongDescription *string `json:"long_description,omitempty"`
StandardCost *float64 `json:"standard_cost,omitempty"`
}
// HandleListItems lists items with optional filtering.
@@ -304,9 +604,20 @@ func (s *Server) HandleListItems(w http.ResponseWriter, r *http.Request) {
return
}
// Batch-fetch file attachment stats
ids := make([]string, len(items))
for i, item := range items {
ids[i] = item.ID
}
fileStats, _ := s.items.BatchGetFileStats(ctx, ids)
response := make([]ItemResponse, len(items))
for i, item := range items {
response[i] = itemToResponse(item)
if fs, ok := fileStats[item.ID]; ok {
response[i].FileCount = fs.Count
response[i].FilesTotalSize = fs.TotalSize
}
}
writeJSON(w, http.StatusOK, response)
@@ -411,14 +722,18 @@ func (s *Server) HandleCreateItem(w http.ResponseWriter, r *http.Request) {
return
}
if err := s.partgen.Validate(partNumber, schemaName); err != nil {
s.logger.Error().Err(err).Str("part_number", partNumber).Msg("generated part number failed validation")
writeError(w, http.StatusInternalServerError, "validation_failed", err.Error())
return
}
item = &db.Item{
PartNumber: partNumber,
ItemType: itemType,
Description: req.Description,
SourcingType: req.SourcingType,
SourcingLink: req.SourcingLink,
LongDescription: req.LongDescription,
StandardCost: req.StandardCost,
}
if user := auth.UserFromContext(ctx); user != nil {
item.CreatedBy = &user.Username
@@ -482,7 +797,15 @@ func (s *Server) HandleGetItemByUUID(w http.ResponseWriter, r *http.Request) {
return
}
writeJSON(w, http.StatusOK, itemToResponse(item))
response := itemToResponse(item)
if fileStats, err := s.items.BatchGetFileStats(ctx, []string{item.ID}); err == nil {
if fs, ok := fileStats[item.ID]; ok {
response.FileCount = fs.Count
response.FilesTotalSize = fs.TotalSize
}
}
writeJSON(w, http.StatusOK, response)
}
// HandleGetItem retrieves an item by part number.
@@ -504,6 +827,14 @@ func (s *Server) HandleGetItem(w http.ResponseWriter, r *http.Request) {
response := itemToResponse(item)
// File attachment stats
if fileStats, err := s.items.BatchGetFileStats(ctx, []string{item.ID}); err == nil {
if fs, ok := fileStats[item.ID]; ok {
response.FileCount = fs.Count
response.FilesTotalSize = fs.TotalSize
}
}
// Include properties from current revision if requested
if r.URL.Query().Get("include") == "properties" {
revisions, err := s.items.GetRevisions(ctx, item.ID)
@@ -528,9 +859,7 @@ type UpdateItemRequest struct {
Properties map[string]any `json:"properties,omitempty"`
Comment string `json:"comment,omitempty"`
SourcingType *string `json:"sourcing_type,omitempty"`
SourcingLink *string `json:"sourcing_link,omitempty"`
LongDescription *string `json:"long_description,omitempty"`
StandardCost *float64 `json:"standard_cost,omitempty"`
}
// HandleUpdateItem updates an item's fields and/or creates a new revision.
@@ -561,9 +890,7 @@ func (s *Server) HandleUpdateItem(w http.ResponseWriter, r *http.Request) {
ItemType: item.ItemType,
Description: item.Description,
SourcingType: req.SourcingType,
SourcingLink: req.SourcingLink,
LongDescription: req.LongDescription,
StandardCost: req.StandardCost,
}
if req.PartNumber != "" {
@@ -1175,9 +1502,7 @@ func itemToResponse(item *db.Item) ItemResponse {
CreatedAt: item.CreatedAt.Format("2006-01-02T15:04:05Z07:00"),
UpdatedAt: item.UpdatedAt.Format("2006-01-02T15:04:05Z07:00"),
SourcingType: item.SourcingType,
SourcingLink: item.SourcingLink,
LongDescription: item.LongDescription,
StandardCost: item.StandardCost,
ThumbnailKey: item.ThumbnailKey,
}
}
@@ -1252,6 +1577,9 @@ func (s *Server) HandleCreateRevision(w http.ResponseWriter, r *http.Request) {
"part_number": partNumber,
"revision_number": rev.RevisionNumber,
}))
// Trigger auto-jobs (e.g. validation, export)
go s.triggerJobs(context.Background(), "revision_created", item.ID, item)
}
// HandleUploadFile uploads a file and creates a new revision.
@@ -1351,10 +1679,14 @@ func (s *Server) HandleUploadFile(w http.ResponseWriter, r *http.Request) {
Int64("size", result.Size).
Msg("file uploaded")
// .kc metadata extraction (best-effort)
s.extractKCMetadata(ctx, item, fileKey, rev)
writeJSON(w, http.StatusCreated, revisionToResponse(rev))
}
// HandleDownloadFile downloads the file for a specific revision.
// For .kc files, silo/ entries are repacked with current DB state.
func (s *Server) HandleDownloadFile(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
@@ -1409,18 +1741,23 @@ func (s *Server) HandleDownloadFile(w http.ResponseWriter, r *http.Request) {
return
}
// Get file from storage
var reader interface {
Read(p []byte) (n int, err error)
Close() error
// ETag: computed from revision + metadata freshness.
meta, _ := s.metadata.Get(ctx, item.ID) // nil is ok (plain .fcstd)
etag := computeETag(revision, meta)
if match := r.Header.Get("If-None-Match"); match == etag {
w.Header().Set("ETag", etag)
w.WriteHeader(http.StatusNotModified)
return
}
// Get file from storage
var reader io.ReadCloser
if revision.FileVersion != nil && *revision.FileVersion != "" {
reader, err = s.storage.GetVersion(ctx, *revision.FileKey, *revision.FileVersion)
} else {
reader, err = s.storage.Get(ctx, *revision.FileKey)
}
if err != nil {
s.logger.Error().Err(err).Str("key", *revision.FileKey).Msg("failed to get file")
writeError(w, http.StatusInternalServerError, "download_failed", err.Error())
@@ -1428,28 +1765,37 @@ func (s *Server) HandleDownloadFile(w http.ResponseWriter, r *http.Request) {
}
defer reader.Close()
// Read entire file for potential .kc repacking.
data, err := io.ReadAll(reader)
if err != nil {
s.logger.Error().Err(err).Msg("failed to read file")
writeError(w, http.StatusInternalServerError, "download_failed", "Failed to read file")
return
}
// Repack silo/ entries for .kc files with indexed metadata.
output := data
if meta != nil {
if hasSilo, chkErr := kc.HasSiloDir(data); chkErr == nil && hasSilo {
if !canSkipRepack(revision, meta) {
if packed, packErr := s.packKCFile(ctx, data, item, revision, meta); packErr != nil {
s.logger.Warn().Err(packErr).Str("part_number", partNumber).Msg("kc: packing failed, serving original")
} else {
output = packed
}
}
}
}
// Set response headers
filename := partNumber + "_rev" + strconv.Itoa(revNum) + ".FCStd"
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", "attachment; filename=\""+filename+"\"")
if revision.FileSize != nil {
w.Header().Set("Content-Length", strconv.FormatInt(*revision.FileSize, 10))
}
w.Header().Set("Content-Length", strconv.Itoa(len(output)))
w.Header().Set("ETag", etag)
w.Header().Set("Cache-Control", "private, must-revalidate")
// Stream file to response
buf := make([]byte, 32*1024)
for {
n, readErr := reader.Read(buf)
if n > 0 {
if _, writeErr := w.Write(buf[:n]); writeErr != nil {
s.logger.Error().Err(writeErr).Msg("failed to write response")
return
}
}
if readErr != nil {
break
}
}
w.Write(output)
}
// HandleDownloadLatestFile downloads the file for the latest revision.

View File

@@ -0,0 +1,382 @@
package api
import (
"context"
"encoding/json"
"net/http"
"strconv"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
)
// HandleListJobs returns jobs filtered by status and/or item.
func (s *Server) HandleListJobs(w http.ResponseWriter, r *http.Request) {
status := r.URL.Query().Get("status")
itemID := r.URL.Query().Get("item_id")
limit := 50
if v := r.URL.Query().Get("limit"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 && n <= 200 {
limit = n
}
}
offset := 0
if v := r.URL.Query().Get("offset"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n >= 0 {
offset = n
}
}
jobs, err := s.jobs.ListJobs(r.Context(), status, itemID, limit, offset)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list jobs")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list jobs")
return
}
writeJSON(w, http.StatusOK, jobs)
}
// HandleGetJob returns a single job by ID.
func (s *Server) HandleGetJob(w http.ResponseWriter, r *http.Request) {
jobID := chi.URLParam(r, "jobID")
job, err := s.jobs.GetJob(r.Context(), jobID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get job")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get job")
return
}
if job == nil {
writeError(w, http.StatusNotFound, "not_found", "Job not found")
return
}
writeJSON(w, http.StatusOK, job)
}
// HandleGetJobLogs returns log entries for a job.
func (s *Server) HandleGetJobLogs(w http.ResponseWriter, r *http.Request) {
jobID := chi.URLParam(r, "jobID")
logs, err := s.jobs.GetJobLogs(r.Context(), jobID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get job logs")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get job logs")
return
}
writeJSON(w, http.StatusOK, logs)
}
// HandleCreateJob manually triggers a job.
func (s *Server) HandleCreateJob(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := auth.UserFromContext(ctx)
var req struct {
DefinitionName string `json:"definition_name"`
ItemID *string `json:"item_id,omitempty"`
ProjectID *string `json:"project_id,omitempty"`
ScopeMetadata map[string]any `json:"scope_metadata,omitempty"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if req.DefinitionName == "" {
writeError(w, http.StatusBadRequest, "missing_field", "definition_name is required")
return
}
// Look up definition
def, err := s.jobs.GetDefinition(ctx, req.DefinitionName)
if err != nil {
s.logger.Error().Err(err).Msg("failed to look up job definition")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to look up definition")
return
}
if def == nil {
writeError(w, http.StatusNotFound, "not_found", "Job definition not found: "+req.DefinitionName)
return
}
var createdBy *string
if user != nil {
createdBy = &user.Username
}
job := &db.Job{
JobDefinitionID: &def.ID,
DefinitionName: def.Name,
Priority: def.Priority,
ItemID: req.ItemID,
ProjectID: req.ProjectID,
ScopeMetadata: req.ScopeMetadata,
RunnerTags: def.RunnerTags,
TimeoutSeconds: def.TimeoutSeconds,
MaxRetries: def.MaxRetries,
CreatedBy: createdBy,
}
if err := s.jobs.CreateJob(ctx, job); err != nil {
s.logger.Error().Err(err).Msg("failed to create job")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to create job")
return
}
s.broker.Publish("job.created", mustMarshal(map[string]any{
"job_id": job.ID,
"definition_name": job.DefinitionName,
"item_id": job.ItemID,
}))
writeJSON(w, http.StatusCreated, job)
}
// HandleCancelJob cancels a pending or active job.
func (s *Server) HandleCancelJob(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
jobID := chi.URLParam(r, "jobID")
user := auth.UserFromContext(ctx)
cancelledBy := "system"
if user != nil {
cancelledBy = user.Username
}
if err := s.jobs.CancelJob(ctx, jobID, cancelledBy); err != nil {
writeError(w, http.StatusBadRequest, "cancel_failed", err.Error())
return
}
s.broker.Publish("job.cancelled", mustMarshal(map[string]any{
"job_id": jobID,
"cancelled_by": cancelledBy,
}))
writeJSON(w, http.StatusOK, map[string]string{"status": "cancelled"})
}
// HandleListJobDefinitions returns all loaded job definitions.
func (s *Server) HandleListJobDefinitions(w http.ResponseWriter, r *http.Request) {
defs, err := s.jobs.ListDefinitions(r.Context())
if err != nil {
s.logger.Error().Err(err).Msg("failed to list job definitions")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list definitions")
return
}
writeJSON(w, http.StatusOK, defs)
}
// HandleGetJobDefinition returns a single job definition by name.
func (s *Server) HandleGetJobDefinition(w http.ResponseWriter, r *http.Request) {
name := chi.URLParam(r, "name")
def, err := s.jobs.GetDefinition(r.Context(), name)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get job definition")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get definition")
return
}
if def == nil {
writeError(w, http.StatusNotFound, "not_found", "Job definition not found")
return
}
writeJSON(w, http.StatusOK, def)
}
// HandleReloadJobDefinitions re-reads YAML files from disk and upserts them.
func (s *Server) HandleReloadJobDefinitions(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
if s.jobDefsDir == "" {
writeError(w, http.StatusBadRequest, "no_directory", "Job definitions directory not configured")
return
}
defs, err := loadAndUpsertJobDefs(ctx, s.jobDefsDir, s.jobs)
if err != nil {
s.logger.Error().Err(err).Msg("failed to reload job definitions")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to reload definitions")
return
}
// Update in-memory map
s.jobDefs = defs
writeJSON(w, http.StatusOK, map[string]any{
"reloaded": len(defs),
})
}
// HandleListRunners returns all registered runners (admin).
func (s *Server) HandleListRunners(w http.ResponseWriter, r *http.Request) {
runners, err := s.jobs.ListRunners(r.Context())
if err != nil {
s.logger.Error().Err(err).Msg("failed to list runners")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list runners")
return
}
// Redact token hashes from response
type runnerResponse struct {
ID string `json:"id"`
Name string `json:"name"`
TokenPrefix string `json:"token_prefix"`
Tags []string `json:"tags"`
Status string `json:"status"`
LastHeartbeat *string `json:"last_heartbeat,omitempty"`
LastJobID *string `json:"last_job_id,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
CreatedAt string `json:"created_at"`
}
resp := make([]runnerResponse, len(runners))
for i, runner := range runners {
var hb *string
if runner.LastHeartbeat != nil {
s := runner.LastHeartbeat.Format("2006-01-02T15:04:05Z07:00")
hb = &s
}
resp[i] = runnerResponse{
ID: runner.ID,
Name: runner.Name,
TokenPrefix: runner.TokenPrefix,
Tags: runner.Tags,
Status: runner.Status,
LastHeartbeat: hb,
LastJobID: runner.LastJobID,
Metadata: runner.Metadata,
CreatedAt: runner.CreatedAt.Format("2006-01-02T15:04:05Z07:00"),
}
}
writeJSON(w, http.StatusOK, resp)
}
// HandleRegisterRunner creates a new runner and returns the token (admin).
func (s *Server) HandleRegisterRunner(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
var req struct {
Name string `json:"name"`
Tags []string `json:"tags"`
Metadata map[string]any `json:"metadata,omitempty"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if req.Name == "" {
writeError(w, http.StatusBadRequest, "missing_field", "name is required")
return
}
if len(req.Tags) == 0 {
writeError(w, http.StatusBadRequest, "missing_field", "tags is required (at least one)")
return
}
rawToken, tokenHash, tokenPrefix := generateRunnerToken()
runner := &db.Runner{
Name: req.Name,
TokenHash: tokenHash,
TokenPrefix: tokenPrefix,
Tags: req.Tags,
Metadata: req.Metadata,
}
if err := s.jobs.RegisterRunner(ctx, runner); err != nil {
s.logger.Error().Err(err).Msg("failed to register runner")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to register runner")
return
}
s.broker.Publish("runner.online", mustMarshal(map[string]any{
"runner_id": runner.ID,
"name": runner.Name,
}))
writeJSON(w, http.StatusCreated, map[string]any{
"id": runner.ID,
"name": runner.Name,
"token": rawToken,
"tags": runner.Tags,
})
}
// HandleDeleteRunner removes a runner (admin).
func (s *Server) HandleDeleteRunner(w http.ResponseWriter, r *http.Request) {
runnerID := chi.URLParam(r, "runnerID")
if err := s.jobs.DeleteRunner(r.Context(), runnerID); err != nil {
writeError(w, http.StatusNotFound, "not_found", err.Error())
return
}
w.WriteHeader(http.StatusNoContent)
}
// triggerJobs creates jobs for all enabled definitions matching the trigger type.
// It applies trigger filters (e.g. item_type) before creating each job.
func (s *Server) triggerJobs(ctx context.Context, triggerType string, itemID string, item *db.Item) {
if !s.modules.IsEnabled("jobs") {
return
}
defs, err := s.jobs.GetDefinitionsByTrigger(ctx, triggerType)
if err != nil {
s.logger.Error().Err(err).Str("trigger", triggerType).Msg("failed to get job definitions for trigger")
return
}
for _, def := range defs {
// Apply trigger filter (e.g. item_type == "assembly")
if def.Definition != nil {
if triggerCfg, ok := def.Definition["trigger"].(map[string]any); ok {
if filterCfg, ok := triggerCfg["filter"].(map[string]any); ok {
if reqType, ok := filterCfg["item_type"].(string); ok && item != nil {
if item.ItemType != reqType {
continue
}
}
}
}
}
job := &db.Job{
JobDefinitionID: &def.ID,
DefinitionName: def.Name,
Priority: def.Priority,
ItemID: &itemID,
RunnerTags: def.RunnerTags,
TimeoutSeconds: def.TimeoutSeconds,
MaxRetries: def.MaxRetries,
}
if err := s.jobs.CreateJob(ctx, job); err != nil {
s.logger.Error().Err(err).Str("definition", def.Name).Msg("failed to create triggered job")
continue
}
s.broker.Publish("job.created", mustMarshal(map[string]any{
"job_id": job.ID,
"definition_name": def.Name,
"trigger": triggerType,
"item_id": itemID,
}))
s.logger.Info().
Str("job_id", job.ID).
Str("definition", def.Name).
Str("trigger", triggerType).
Str("item_id", itemID).
Msg("triggered job")
}
}

View File

@@ -0,0 +1,596 @@
package api
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/testutil"
"github.com/rs/zerolog"
)
func newJobTestServer(t *testing.T) *Server {
t.Helper()
pool := testutil.MustConnectTestPool(t)
database := db.NewFromPool(pool)
broker := NewBroker(zerolog.Nop())
state := NewServerState(zerolog.Nop(), nil, broker)
return NewServer(
zerolog.Nop(),
database,
map[string]*schema.Schema{},
"",
nil, nil, nil, nil, nil,
broker, state,
nil, "",
modules.NewRegistry(), nil,
nil,
)
}
func newJobRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Route("/api/jobs", func(r chi.Router) {
r.Get("/", s.HandleListJobs)
r.Get("/{jobID}", s.HandleGetJob)
r.Get("/{jobID}/logs", s.HandleGetJobLogs)
r.Post("/", s.HandleCreateJob)
r.Post("/{jobID}/cancel", s.HandleCancelJob)
})
r.Route("/api/job-definitions", func(r chi.Router) {
r.Get("/", s.HandleListJobDefinitions)
r.Get("/{name}", s.HandleGetJobDefinition)
})
r.Route("/api/runners", func(r chi.Router) {
r.Get("/", s.HandleListRunners)
r.Post("/", s.HandleRegisterRunner)
r.Delete("/{runnerID}", s.HandleDeleteRunner)
})
return r
}
func seedJobDefinition(t *testing.T, s *Server) *db.JobDefinitionRecord {
t.Helper()
rec := &db.JobDefinitionRecord{
Name: "test-validate",
Version: 1,
TriggerType: "manual",
ScopeType: "item",
ComputeType: "validate",
RunnerTags: []string{"create"},
TimeoutSeconds: 300,
MaxRetries: 1,
Priority: 100,
Definition: map[string]any{"compute": map[string]any{"command": "create-validate"}},
Enabled: true,
}
if err := s.jobs.UpsertDefinition(context.Background(), rec); err != nil {
t.Fatalf("seeding job definition: %v", err)
}
return rec
}
func TestHandleListJobDefinitions(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
seedJobDefinition(t, s)
req := httptest.NewRequest("GET", "/api/job-definitions", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
var defs []map[string]any
json.Unmarshal(w.Body.Bytes(), &defs)
if len(defs) == 0 {
t.Error("expected at least one definition")
}
}
func TestHandleGetJobDefinition(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
seedJobDefinition(t, s)
req := httptest.NewRequest("GET", "/api/job-definitions/test-validate", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d: %s", w.Code, w.Body.String())
}
var def map[string]any
json.Unmarshal(w.Body.Bytes(), &def)
if def["name"] != "test-validate" {
t.Errorf("expected name test-validate, got %v", def["name"])
}
}
func TestHandleCreateAndGetJob(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
seedJobDefinition(t, s)
// Create a job
body := `{"definition_name": "test-validate"}`
req := httptest.NewRequest("POST", "/api/jobs", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create: expected 201, got %d: %s", w.Code, w.Body.String())
}
var job map[string]any
json.Unmarshal(w.Body.Bytes(), &job)
jobID := job["ID"].(string)
if jobID == "" {
t.Fatal("job ID is empty")
}
// Get the job
req2 := httptest.NewRequest("GET", "/api/jobs/"+jobID, nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("get: expected 200, got %d: %s", w2.Code, w2.Body.String())
}
}
func TestHandleCancelJob(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
seedJobDefinition(t, s)
// Create a job
body := `{"definition_name": "test-validate"}`
req := httptest.NewRequest("POST", "/api/jobs", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
var job map[string]any
json.Unmarshal(w.Body.Bytes(), &job)
jobID := job["ID"].(string)
// Cancel the job
req2 := httptest.NewRequest("POST", "/api/jobs/"+jobID+"/cancel", nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("cancel: expected 200, got %d: %s", w2.Code, w2.Body.String())
}
}
func TestHandleListJobs(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
seedJobDefinition(t, s)
// Create a job
body := `{"definition_name": "test-validate"}`
req := httptest.NewRequest("POST", "/api/jobs", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
// List jobs
req2 := httptest.NewRequest("GET", "/api/jobs", nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("list: expected 200, got %d: %s", w2.Code, w2.Body.String())
}
var jobs []map[string]any
json.Unmarshal(w2.Body.Bytes(), &jobs)
if len(jobs) == 0 {
t.Error("expected at least one job")
}
}
func TestHandleListJobs_FilterByStatus(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
seedJobDefinition(t, s)
// Create a job
body := `{"definition_name": "test-validate"}`
req := httptest.NewRequest("POST", "/api/jobs", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
// Filter by pending
req2 := httptest.NewRequest("GET", "/api/jobs?status=pending", nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("expected 200, got %d", w2.Code)
}
var jobs []map[string]any
json.Unmarshal(w2.Body.Bytes(), &jobs)
if len(jobs) == 0 {
t.Error("expected pending jobs")
}
// Filter by completed (should be empty)
req3 := httptest.NewRequest("GET", "/api/jobs?status=completed", nil)
w3 := httptest.NewRecorder()
r.ServeHTTP(w3, req3)
var completedJobs []map[string]any
json.Unmarshal(w3.Body.Bytes(), &completedJobs)
if len(completedJobs) != 0 {
t.Errorf("expected no completed jobs, got %d", len(completedJobs))
}
}
func TestHandleRegisterAndListRunners(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
// Register a runner
body := `{"name": "test-runner-1", "tags": ["create", "linux"]}`
req := httptest.NewRequest("POST", "/api/runners", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("register: expected 201, got %d: %s", w.Code, w.Body.String())
}
var resp map[string]any
json.Unmarshal(w.Body.Bytes(), &resp)
if resp["token"] == nil || resp["token"] == "" {
t.Error("expected a token in response")
}
if !strings.HasPrefix(resp["token"].(string), "silo_runner_") {
t.Errorf("expected token to start with silo_runner_, got %s", resp["token"])
}
// List runners
req2 := httptest.NewRequest("GET", "/api/runners", nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusOK {
t.Fatalf("list: expected 200, got %d", w2.Code)
}
var runners []map[string]any
json.Unmarshal(w2.Body.Bytes(), &runners)
if len(runners) == 0 {
t.Error("expected at least one runner")
}
// Token hash should not be exposed
for _, runner := range runners {
if runner["token_hash"] != nil {
t.Error("token_hash should not be in response")
}
}
}
func TestHandleDeleteRunner(t *testing.T) {
s := newJobTestServer(t)
r := newJobRouter(s)
// Register a runner
body := `{"name": "test-runner-delete", "tags": ["create"]}`
req := httptest.NewRequest("POST", "/api/runners", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
var resp map[string]any
json.Unmarshal(w.Body.Bytes(), &resp)
runnerID := resp["id"].(string)
// Delete the runner
req2 := httptest.NewRequest("DELETE", "/api/runners/"+runnerID, nil)
w2 := httptest.NewRecorder()
r.ServeHTTP(w2, req2)
if w2.Code != http.StatusNoContent {
t.Fatalf("delete: expected 204, got %d: %s", w2.Code, w2.Body.String())
}
}
// --- Trigger integration tests ---
// newTriggerRouter builds a router with items, revisions, BOM, and jobs routes
// so that HTTP-based actions can fire triggerJobs via goroutine.
func newTriggerRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Route("/api/items", func(r chi.Router) {
r.Post("/", s.HandleCreateItem)
r.Route("/{partNumber}", func(r chi.Router) {
r.Post("/revisions", s.HandleCreateRevision)
r.Post("/bom", s.HandleAddBOMEntry)
r.Put("/bom/{childPartNumber}", s.HandleUpdateBOMEntry)
r.Delete("/bom/{childPartNumber}", s.HandleDeleteBOMEntry)
})
})
r.Route("/api/jobs", func(r chi.Router) {
r.Get("/", s.HandleListJobs)
})
return r
}
func waitForJobs(t *testing.T, s *Server, itemID string, wantCount int) []*db.Job {
t.Helper()
// triggerJobs runs in a goroutine; poll up to 2 seconds.
for i := 0; i < 20; i++ {
jobs, err := s.jobs.ListJobs(context.Background(), "", itemID, 50, 0)
if err != nil {
t.Fatalf("listing jobs: %v", err)
}
if len(jobs) >= wantCount {
return jobs
}
time.Sleep(100 * time.Millisecond)
}
jobs, _ := s.jobs.ListJobs(context.Background(), "", itemID, 50, 0)
return jobs
}
func TestTriggerJobsOnRevisionCreate(t *testing.T) {
s := newJobTestServer(t)
if err := s.modules.SetEnabled("jobs", true); err != nil {
t.Fatalf("enabling jobs module: %v", err)
}
router := newTriggerRouter(s)
// Create an item.
createItemDirect(t, s, "TRIG-REV-001", "trigger test item", nil)
// Seed a job definition that triggers on revision_created.
def := &db.JobDefinitionRecord{
Name: "rev-trigger-test",
Version: 1,
TriggerType: "revision_created",
ScopeType: "item",
ComputeType: "validate",
RunnerTags: []string{"test"},
TimeoutSeconds: 60,
MaxRetries: 0,
Priority: 100,
Enabled: true,
}
if err := s.jobs.UpsertDefinition(context.Background(), def); err != nil {
t.Fatalf("seeding definition: %v", err)
}
// Create a revision via HTTP (fires triggerJobs in goroutine).
body := `{"properties":{"material":"steel"},"comment":"trigger test"}`
req := authRequest(httptest.NewRequest("POST", "/api/items/TRIG-REV-001/revisions", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create revision: expected 201, got %d: %s", w.Code, w.Body.String())
}
// Get the item ID to filter jobs.
item, _ := s.items.GetByPartNumber(context.Background(), "TRIG-REV-001")
if item == nil {
t.Fatal("item not found after creation")
}
jobs := waitForJobs(t, s, item.ID, 1)
if len(jobs) == 0 {
t.Fatal("expected at least 1 triggered job, got 0")
}
if jobs[0].DefinitionName != "rev-trigger-test" {
t.Errorf("expected definition name rev-trigger-test, got %s", jobs[0].DefinitionName)
}
}
func TestTriggerJobsOnBOMChange(t *testing.T) {
s := newJobTestServer(t)
if err := s.modules.SetEnabled("jobs", true); err != nil {
t.Fatalf("enabling jobs module: %v", err)
}
router := newTriggerRouter(s)
// Create parent and child items.
createItemDirect(t, s, "TRIG-BOM-P", "parent", nil)
createItemDirect(t, s, "TRIG-BOM-C", "child", nil)
// Seed a bom_changed job definition.
def := &db.JobDefinitionRecord{
Name: "bom-trigger-test",
Version: 1,
TriggerType: "bom_changed",
ScopeType: "item",
ComputeType: "validate",
RunnerTags: []string{"test"},
TimeoutSeconds: 60,
MaxRetries: 0,
Priority: 100,
Enabled: true,
}
if err := s.jobs.UpsertDefinition(context.Background(), def); err != nil {
t.Fatalf("seeding definition: %v", err)
}
// Add a BOM entry via HTTP.
body := `{"child_part_number":"TRIG-BOM-C","rel_type":"component","quantity":2}`
req := authRequest(httptest.NewRequest("POST", "/api/items/TRIG-BOM-P/bom", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("add BOM entry: expected 201, got %d: %s", w.Code, w.Body.String())
}
// Get the parent item ID.
parent, _ := s.items.GetByPartNumber(context.Background(), "TRIG-BOM-P")
if parent == nil {
t.Fatal("parent item not found")
}
jobs := waitForJobs(t, s, parent.ID, 1)
if len(jobs) == 0 {
t.Fatal("expected at least 1 triggered job, got 0")
}
if jobs[0].DefinitionName != "bom-trigger-test" {
t.Errorf("expected definition name bom-trigger-test, got %s", jobs[0].DefinitionName)
}
}
func TestTriggerJobsFilterMismatch(t *testing.T) {
s := newJobTestServer(t)
if err := s.modules.SetEnabled("jobs", true); err != nil {
t.Fatalf("enabling jobs module: %v", err)
}
router := newTriggerRouter(s)
// Create a "part" type item (not "assembly").
createItemDirect(t, s, "TRIG-FILT-P", "filter parent", nil)
createItemDirect(t, s, "TRIG-FILT-C", "filter child", nil)
// Seed a definition that only triggers for assembly items.
def := &db.JobDefinitionRecord{
Name: "assembly-only-test",
Version: 1,
TriggerType: "bom_changed",
ScopeType: "item",
ComputeType: "validate",
RunnerTags: []string{"test"},
TimeoutSeconds: 60,
MaxRetries: 0,
Priority: 100,
Enabled: true,
Definition: map[string]any{
"trigger": map[string]any{
"filter": map[string]any{
"item_type": "assembly",
},
},
},
}
if err := s.jobs.UpsertDefinition(context.Background(), def); err != nil {
t.Fatalf("seeding definition: %v", err)
}
// Add a BOM entry on a "part" item (should NOT match assembly filter).
body := `{"child_part_number":"TRIG-FILT-C","rel_type":"component","quantity":1}`
req := authRequest(httptest.NewRequest("POST", "/api/items/TRIG-FILT-P/bom", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("add BOM entry: expected 201, got %d: %s", w.Code, w.Body.String())
}
// Wait briefly, then verify no jobs were created.
parent, _ := s.items.GetByPartNumber(context.Background(), "TRIG-FILT-P")
time.Sleep(500 * time.Millisecond)
jobs, err := s.jobs.ListJobs(context.Background(), "", parent.ID, 50, 0)
if err != nil {
t.Fatalf("listing jobs: %v", err)
}
if len(jobs) != 0 {
t.Errorf("expected 0 jobs (filter mismatch), got %d", len(jobs))
}
}
func TestTriggerJobsModuleDisabled(t *testing.T) {
s := newJobTestServer(t)
// Jobs module is disabled by default in NewRegistry().
router := newTriggerRouter(s)
// Create items.
createItemDirect(t, s, "TRIG-DIS-P", "disabled parent", nil)
createItemDirect(t, s, "TRIG-DIS-C", "disabled child", nil)
// Seed a bom_changed definition (it exists in DB but module is off).
def := &db.JobDefinitionRecord{
Name: "disabled-trigger-test",
Version: 1,
TriggerType: "bom_changed",
ScopeType: "item",
ComputeType: "validate",
RunnerTags: []string{"test"},
TimeoutSeconds: 60,
MaxRetries: 0,
Priority: 100,
Enabled: true,
}
if err := s.jobs.UpsertDefinition(context.Background(), def); err != nil {
t.Fatalf("seeding definition: %v", err)
}
// Add a BOM entry with jobs module disabled.
body := `{"child_part_number":"TRIG-DIS-C","rel_type":"component","quantity":1}`
req := authRequest(httptest.NewRequest("POST", "/api/items/TRIG-DIS-P/bom", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("add BOM entry: expected 201, got %d: %s", w.Code, w.Body.String())
}
// Wait briefly, then verify no jobs were created.
parent, _ := s.items.GetByPartNumber(context.Background(), "TRIG-DIS-P")
time.Sleep(500 * time.Millisecond)
jobs, err := s.jobs.ListJobs(context.Background(), "", parent.ID, 50, 0)
if err != nil {
t.Fatalf("listing jobs: %v", err)
}
if len(jobs) != 0 {
t.Errorf("expected 0 jobs (module disabled), got %d", len(jobs))
}
}
func TestGenerateRunnerToken(t *testing.T) {
raw, hash, prefix := generateRunnerToken()
if !strings.HasPrefix(raw, "silo_runner_") {
t.Errorf("raw token should start with silo_runner_, got %s", raw[:20])
}
if len(hash) != 64 {
t.Errorf("hash should be 64 hex chars, got %d", len(hash))
}
if len(prefix) != 20 {
t.Errorf("prefix should be 20 chars, got %d: %s", len(prefix), prefix)
}
// Two tokens should be different
raw2, _, _ := generateRunnerToken()
if raw == raw2 {
t.Error("two generated tokens should be different")
}
}

View File

@@ -0,0 +1,234 @@
package api
import (
"encoding/json"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/db"
)
// LocationResponse is the API representation of a location.
type LocationResponse struct {
ID string `json:"id"`
Path string `json:"path"`
Name string `json:"name"`
ParentID *string `json:"parent_id,omitempty"`
LocationType string `json:"location_type"`
Depth int `json:"depth"`
Metadata map[string]any `json:"metadata,omitempty"`
CreatedAt string `json:"created_at"`
}
// CreateLocationRequest represents a request to create a location.
type CreateLocationRequest struct {
Path string `json:"path"`
Name string `json:"name"`
LocationType string `json:"location_type"`
Metadata map[string]any `json:"metadata,omitempty"`
}
// UpdateLocationRequest represents a request to update a location.
type UpdateLocationRequest struct {
Name string `json:"name"`
LocationType string `json:"location_type"`
Metadata map[string]any `json:"metadata,omitempty"`
}
func locationToResponse(loc *db.Location) LocationResponse {
return LocationResponse{
ID: loc.ID,
Path: loc.Path,
Name: loc.Name,
ParentID: loc.ParentID,
LocationType: loc.LocationType,
Depth: loc.Depth,
Metadata: loc.Metadata,
CreatedAt: loc.CreatedAt.Format("2006-01-02T15:04:05Z07:00"),
}
}
// HandleListLocations lists all locations. If ?tree={path} is set, returns that
// subtree. If ?root=true, returns only root-level locations (depth 0).
func (s *Server) HandleListLocations(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
treePath := r.URL.Query().Get("tree")
if treePath != "" {
locs, err := s.locations.GetTree(ctx, treePath)
if err != nil {
s.logger.Error().Err(err).Str("tree", treePath).Msg("failed to get location tree")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get location tree")
return
}
writeJSON(w, http.StatusOK, locationsToResponse(locs))
return
}
locs, err := s.locations.List(ctx)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list locations")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list locations")
return
}
writeJSON(w, http.StatusOK, locationsToResponse(locs))
}
// HandleCreateLocation creates a new location.
func (s *Server) HandleCreateLocation(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
var req CreateLocationRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_json", err.Error())
return
}
if req.Path == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Path is required")
return
}
if req.Name == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Name is required")
return
}
if req.LocationType == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Location type is required")
return
}
// Normalize: trim slashes
req.Path = strings.Trim(req.Path, "/")
loc := &db.Location{
Path: req.Path,
Name: req.Name,
LocationType: req.LocationType,
Metadata: req.Metadata,
}
if loc.Metadata == nil {
loc.Metadata = map[string]any{}
}
if err := s.locations.Create(ctx, loc); err != nil {
if strings.Contains(err.Error(), "parent location") || strings.Contains(err.Error(), "does not exist") {
writeError(w, http.StatusBadRequest, "invalid_parent", err.Error())
return
}
if strings.Contains(err.Error(), "duplicate key") || strings.Contains(err.Error(), "unique") {
writeError(w, http.StatusConflict, "already_exists", "Location path already exists")
return
}
s.logger.Error().Err(err).Str("path", req.Path).Msg("failed to create location")
writeError(w, http.StatusInternalServerError, "create_failed", err.Error())
return
}
writeJSON(w, http.StatusCreated, locationToResponse(loc))
}
// HandleGetLocation retrieves a location by path. The path is the rest of the
// URL after /api/locations/, which chi captures as a wildcard.
func (s *Server) HandleGetLocation(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
path := strings.Trim(chi.URLParam(r, "*"), "/")
if path == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Location path is required")
return
}
loc, err := s.locations.GetByPath(ctx, path)
if err != nil {
s.logger.Error().Err(err).Str("path", path).Msg("failed to get location")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get location")
return
}
if loc == nil {
writeError(w, http.StatusNotFound, "not_found", "Location not found")
return
}
writeJSON(w, http.StatusOK, locationToResponse(loc))
}
// HandleUpdateLocation updates a location by path.
func (s *Server) HandleUpdateLocation(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
path := strings.Trim(chi.URLParam(r, "*"), "/")
if path == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Location path is required")
return
}
var req UpdateLocationRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_json", err.Error())
return
}
if req.Name == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Name is required")
return
}
if req.LocationType == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Location type is required")
return
}
meta := req.Metadata
if meta == nil {
meta = map[string]any{}
}
if err := s.locations.Update(ctx, path, req.Name, req.LocationType, meta); err != nil {
if strings.Contains(err.Error(), "not found") {
writeError(w, http.StatusNotFound, "not_found", "Location not found")
return
}
s.logger.Error().Err(err).Str("path", path).Msg("failed to update location")
writeError(w, http.StatusInternalServerError, "update_failed", err.Error())
return
}
loc, _ := s.locations.GetByPath(ctx, path)
writeJSON(w, http.StatusOK, locationToResponse(loc))
}
// HandleDeleteLocation deletes a location by path. Rejects if inventory exists.
func (s *Server) HandleDeleteLocation(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
path := strings.Trim(chi.URLParam(r, "*"), "/")
if path == "" {
writeError(w, http.StatusBadRequest, "invalid_request", "Location path is required")
return
}
if err := s.locations.Delete(ctx, path); err != nil {
if strings.Contains(err.Error(), "inventory record") {
writeError(w, http.StatusConflict, "has_inventory", err.Error())
return
}
if strings.Contains(err.Error(), "not found") {
writeError(w, http.StatusNotFound, "not_found", "Location not found")
return
}
s.logger.Error().Err(err).Str("path", path).Msg("failed to delete location")
writeError(w, http.StatusInternalServerError, "delete_failed", err.Error())
return
}
w.WriteHeader(http.StatusNoContent)
}
func locationsToResponse(locs []*db.Location) []LocationResponse {
result := make([]LocationResponse, len(locs))
for i, l := range locs {
result[i] = locationToResponse(l)
}
return result
}

View File

@@ -0,0 +1,323 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
)
func newLocationRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Get("/api/locations", s.HandleListLocations)
r.Post("/api/locations", s.HandleCreateLocation)
r.Get("/api/locations/*", s.HandleGetLocation)
r.Put("/api/locations/*", s.HandleUpdateLocation)
r.Delete("/api/locations/*", s.HandleDeleteLocation)
return r
}
func TestHandleListLocationsEmpty(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
req := httptest.NewRequest("GET", "/api/locations", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var locs []LocationResponse
if err := json.Unmarshal(w.Body.Bytes(), &locs); err != nil {
t.Fatalf("decoding response: %v", err)
}
if len(locs) != 0 {
t.Fatalf("expected 0 locations, got %d", len(locs))
}
}
func TestHandleCreateAndGetLocation(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
// Create root location
body := `{"path": "lab", "name": "Lab", "location_type": "building"}`
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create status: got %d, want %d; body: %s", w.Code, http.StatusCreated, w.Body.String())
}
var created LocationResponse
if err := json.Unmarshal(w.Body.Bytes(), &created); err != nil {
t.Fatalf("decoding create response: %v", err)
}
if created.Path != "lab" {
t.Errorf("path: got %q, want %q", created.Path, "lab")
}
if created.Name != "Lab" {
t.Errorf("name: got %q, want %q", created.Name, "Lab")
}
if created.Depth != 0 {
t.Errorf("depth: got %d, want 0", created.Depth)
}
if created.ID == "" {
t.Error("expected ID to be set")
}
// Get by path
req = httptest.NewRequest("GET", "/api/locations/lab", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("get status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var got LocationResponse
if err := json.Unmarshal(w.Body.Bytes(), &got); err != nil {
t.Fatalf("decoding get response: %v", err)
}
if got.ID != created.ID {
t.Errorf("ID mismatch: got %q, want %q", got.ID, created.ID)
}
}
func TestHandleCreateNestedLocation(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
// Create root
body := `{"path": "warehouse", "name": "Warehouse", "location_type": "building"}`
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create root: got %d; body: %s", w.Code, w.Body.String())
}
// Create child
body = `{"path": "warehouse/shelf-a", "name": "Shelf A", "location_type": "shelf"}`
req = httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create child: got %d; body: %s", w.Code, w.Body.String())
}
var child LocationResponse
json.Unmarshal(w.Body.Bytes(), &child)
if child.Depth != 1 {
t.Errorf("child depth: got %d, want 1", child.Depth)
}
if child.ParentID == nil {
t.Error("expected parent_id to be set")
}
// Create grandchild
body = `{"path": "warehouse/shelf-a/bin-3", "name": "Bin 3", "location_type": "bin"}`
req = httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create grandchild: got %d; body: %s", w.Code, w.Body.String())
}
var gc LocationResponse
json.Unmarshal(w.Body.Bytes(), &gc)
if gc.Depth != 2 {
t.Errorf("grandchild depth: got %d, want 2", gc.Depth)
}
// Get nested path
req = httptest.NewRequest("GET", "/api/locations/warehouse/shelf-a/bin-3", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("get nested: got %d; body: %s", w.Code, w.Body.String())
}
}
func TestHandleCreateLocationMissingParent(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
body := `{"path": "nonexistent/child", "name": "Child", "location_type": "shelf"}`
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Fatalf("expected 400, got %d; body: %s", w.Code, w.Body.String())
}
}
func TestHandleUpdateLocation(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
// Create
body := `{"path": "office", "name": "Office", "location_type": "room"}`
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create: got %d; body: %s", w.Code, w.Body.String())
}
// Update
body = `{"name": "Main Office", "location_type": "building", "metadata": {"floor": 2}}`
req = httptest.NewRequest("PUT", "/api/locations/office", strings.NewReader(body))
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("update: got %d; body: %s", w.Code, w.Body.String())
}
var updated LocationResponse
json.Unmarshal(w.Body.Bytes(), &updated)
if updated.Name != "Main Office" {
t.Errorf("name: got %q, want %q", updated.Name, "Main Office")
}
if updated.LocationType != "building" {
t.Errorf("type: got %q, want %q", updated.LocationType, "building")
}
}
func TestHandleDeleteLocation(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
// Create
body := `{"path": "temp", "name": "Temp", "location_type": "area"}`
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create: got %d; body: %s", w.Code, w.Body.String())
}
// Delete
req = httptest.NewRequest("DELETE", "/api/locations/temp", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNoContent {
t.Fatalf("delete: got %d, want %d; body: %s", w.Code, http.StatusNoContent, w.Body.String())
}
// Verify gone
req = httptest.NewRequest("GET", "/api/locations/temp", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Fatalf("get after delete: got %d, want %d", w.Code, http.StatusNotFound)
}
}
func TestHandleDeleteLocationNotFound(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
req := httptest.NewRequest("DELETE", "/api/locations/doesnotexist", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Fatalf("delete missing: got %d, want %d; body: %s", w.Code, http.StatusNotFound, w.Body.String())
}
}
func TestHandleListLocationsTree(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
// Create hierarchy
for _, loc := range []string{
`{"path": "site", "name": "Site", "location_type": "site"}`,
`{"path": "site/bldg", "name": "Building", "location_type": "building"}`,
`{"path": "site/bldg/room1", "name": "Room 1", "location_type": "room"}`,
`{"path": "other", "name": "Other", "location_type": "site"}`,
} {
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(loc))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create: got %d; body: %s", w.Code, w.Body.String())
}
}
// List tree under "site"
req := httptest.NewRequest("GET", "/api/locations?tree=site", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("tree: got %d; body: %s", w.Code, w.Body.String())
}
var locs []LocationResponse
json.Unmarshal(w.Body.Bytes(), &locs)
if len(locs) != 3 {
t.Fatalf("tree count: got %d, want 3 (site + bldg + room1)", len(locs))
}
// Full list should have 4
req = httptest.NewRequest("GET", "/api/locations", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
json.Unmarshal(w.Body.Bytes(), &locs)
if len(locs) != 4 {
t.Fatalf("full list: got %d, want 4", len(locs))
}
}
func TestHandleCreateLocationDuplicate(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
body := `{"path": "dup", "name": "Dup", "location_type": "area"}`
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("first create: got %d; body: %s", w.Code, w.Body.String())
}
// Duplicate
req = httptest.NewRequest("POST", "/api/locations", strings.NewReader(body))
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusConflict {
t.Fatalf("duplicate: got %d, want %d; body: %s", w.Code, http.StatusConflict, w.Body.String())
}
}
func TestHandleCreateLocationValidation(t *testing.T) {
s := newTestServer(t)
router := newLocationRouter(s)
tests := []struct {
name string
body string
}{
{"missing path", `{"name": "X", "location_type": "area"}`},
{"missing name", `{"path": "x", "location_type": "area"}`},
{"missing type", `{"path": "x", "name": "X"}`},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest("POST", "/api/locations", strings.NewReader(tc.body))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Fatalf("got %d, want 400; body: %s", w.Code, w.Body.String())
}
})
}
}

View File

@@ -0,0 +1,95 @@
package api
import (
"net/http"
"github.com/go-chi/chi/v5"
)
// MacroListItem is the JSON representation for GET /macros list entries.
type MacroListItem struct {
Filename string `json:"filename"`
Trigger string `json:"trigger"`
RevisionNumber int `json:"revision_number"`
}
// MacroResponse is the JSON representation for GET /macros/{filename}.
type MacroResponse struct {
Filename string `json:"filename"`
Trigger string `json:"trigger"`
Content string `json:"content"`
RevisionNumber int `json:"revision_number"`
}
// HandleGetMacros returns the list of registered macros for an item.
// GET /api/items/{partNumber}/macros
func (s *Server) HandleGetMacros(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
macros, err := s.macros.ListByItem(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list macros")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list macros")
return
}
resp := make([]MacroListItem, len(macros))
for i, m := range macros {
resp[i] = MacroListItem{
Filename: m.Filename,
Trigger: m.Trigger,
RevisionNumber: m.RevisionNumber,
}
}
writeJSON(w, http.StatusOK, resp)
}
// HandleGetMacro returns a single macro's source content.
// GET /api/items/{partNumber}/macros/{filename}
func (s *Server) HandleGetMacro(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
filename := chi.URLParam(r, "filename")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
macro, err := s.macros.GetByFilename(ctx, item.ID, filename)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get macro")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get macro")
return
}
if macro == nil {
writeError(w, http.StatusNotFound, "not_found", "Macro not found")
return
}
writeJSON(w, http.StatusOK, MacroResponse{
Filename: macro.Filename,
Trigger: macro.Trigger,
Content: macro.Content,
RevisionNumber: macro.RevisionNumber,
})
}

View File

@@ -0,0 +1,472 @@
package api
import (
"context"
"encoding/json"
"io"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/kc"
)
// validTransitions defines allowed lifecycle state transitions for Phase 1.
var validTransitions = map[string][]string{
"draft": {"review"},
"review": {"draft", "released"},
"released": {"obsolete"},
"obsolete": {},
}
// MetadataResponse is the JSON representation returned by GET /metadata.
type MetadataResponse struct {
SchemaName *string `json:"schema_name"`
LifecycleState string `json:"lifecycle_state"`
Tags []string `json:"tags"`
Fields map[string]any `json:"fields"`
Manifest *ManifestInfo `json:"manifest,omitempty"`
UpdatedAt string `json:"updated_at"`
UpdatedBy *string `json:"updated_by,omitempty"`
}
// ManifestInfo is the manifest subset included in MetadataResponse.
type ManifestInfo struct {
UUID *string `json:"uuid,omitempty"`
SiloInstance *string `json:"silo_instance,omitempty"`
RevisionHash *string `json:"revision_hash,omitempty"`
KCVersion *string `json:"kc_version,omitempty"`
}
func metadataToResponse(m *db.ItemMetadata) MetadataResponse {
resp := MetadataResponse{
SchemaName: m.SchemaName,
LifecycleState: m.LifecycleState,
Tags: m.Tags,
Fields: m.Fields,
UpdatedAt: m.UpdatedAt.UTC().Format("2006-01-02T15:04:05Z"),
UpdatedBy: m.UpdatedBy,
}
if m.ManifestUUID != nil || m.SiloInstance != nil || m.RevisionHash != nil || m.KCVersion != nil {
resp.Manifest = &ManifestInfo{
UUID: m.ManifestUUID,
SiloInstance: m.SiloInstance,
RevisionHash: m.RevisionHash,
KCVersion: m.KCVersion,
}
}
return resp
}
// HandleGetMetadata returns indexed metadata for an item.
// GET /api/items/{partNumber}/metadata
func (s *Server) HandleGetMetadata(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
meta, err := s.metadata.Get(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get metadata")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get metadata")
return
}
if meta == nil {
writeError(w, http.StatusNotFound, "not_found", "No metadata indexed for this item")
return
}
writeJSON(w, http.StatusOK, metadataToResponse(meta))
}
// HandleUpdateMetadata merges fields into the metadata JSONB.
// PUT /api/items/{partNumber}/metadata
func (s *Server) HandleUpdateMetadata(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
var req struct {
Fields map[string]any `json:"fields"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if len(req.Fields) == 0 {
writeError(w, http.StatusBadRequest, "invalid_body", "Fields must not be empty")
return
}
username := ""
if user := auth.UserFromContext(ctx); user != nil {
username = user.Username
}
if err := s.metadata.UpdateFields(ctx, item.ID, req.Fields, username); err != nil {
s.logger.Error().Err(err).Msg("failed to update metadata fields")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to update metadata")
return
}
meta, err := s.metadata.Get(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to read back metadata")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to read metadata")
return
}
s.broker.Publish("metadata.updated", mustMarshal(map[string]any{
"part_number": partNumber,
"changed_fields": fieldKeys(req.Fields),
"lifecycle_state": meta.LifecycleState,
"updated_by": username,
}))
writeJSON(w, http.StatusOK, metadataToResponse(meta))
}
// HandleUpdateLifecycle transitions the lifecycle state.
// PATCH /api/items/{partNumber}/metadata/lifecycle
func (s *Server) HandleUpdateLifecycle(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
var req struct {
State string `json:"state"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if req.State == "" {
writeError(w, http.StatusBadRequest, "invalid_body", "State is required")
return
}
meta, err := s.metadata.Get(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get metadata")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get metadata")
return
}
if meta == nil {
writeError(w, http.StatusNotFound, "not_found", "No metadata indexed for this item")
return
}
// Validate transition
allowed := validTransitions[meta.LifecycleState]
valid := false
for _, s := range allowed {
if s == req.State {
valid = true
break
}
}
if !valid {
writeError(w, http.StatusUnprocessableEntity, "invalid_transition",
"Cannot transition from '"+meta.LifecycleState+"' to '"+req.State+"'")
return
}
username := ""
if user := auth.UserFromContext(ctx); user != nil {
username = user.Username
}
fromState := meta.LifecycleState
if err := s.metadata.UpdateLifecycle(ctx, item.ID, req.State, username); err != nil {
s.logger.Error().Err(err).Msg("failed to update lifecycle")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to update lifecycle")
return
}
s.broker.Publish("metadata.lifecycle", mustMarshal(map[string]any{
"part_number": partNumber,
"from_state": fromState,
"to_state": req.State,
"updated_by": username,
}))
writeJSON(w, http.StatusOK, map[string]string{"lifecycle_state": req.State})
}
// HandleUpdateTags adds/removes tags.
// PATCH /api/items/{partNumber}/metadata/tags
func (s *Server) HandleUpdateTags(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
var req struct {
Add []string `json:"add"`
Remove []string `json:"remove"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if len(req.Add) == 0 && len(req.Remove) == 0 {
writeError(w, http.StatusBadRequest, "invalid_body", "Must provide 'add' or 'remove'")
return
}
meta, err := s.metadata.Get(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get metadata")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get metadata")
return
}
if meta == nil {
writeError(w, http.StatusNotFound, "not_found", "No metadata indexed for this item")
return
}
// Compute new tag set: (existing + add) - remove
tagSet := make(map[string]struct{})
for _, t := range meta.Tags {
tagSet[t] = struct{}{}
}
for _, t := range req.Add {
tagSet[t] = struct{}{}
}
removeSet := make(map[string]struct{})
for _, t := range req.Remove {
removeSet[t] = struct{}{}
}
var newTags []string
for t := range tagSet {
if _, removed := removeSet[t]; !removed {
newTags = append(newTags, t)
}
}
if newTags == nil {
newTags = []string{}
}
username := ""
if user := auth.UserFromContext(ctx); user != nil {
username = user.Username
}
if err := s.metadata.SetTags(ctx, item.ID, newTags, username); err != nil {
s.logger.Error().Err(err).Msg("failed to update tags")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to update tags")
return
}
s.broker.Publish("metadata.tags", mustMarshal(map[string]any{
"part_number": partNumber,
"added": req.Add,
"removed": req.Remove,
}))
writeJSON(w, http.StatusOK, map[string]any{"tags": newTags})
}
// extractKCMetadata attempts to extract and index silo/ metadata from an
// uploaded .kc file. Failures are logged but non-fatal for Phase 1.
func (s *Server) extractKCMetadata(ctx context.Context, item *db.Item, fileKey string, rev *db.Revision) {
if s.storage == nil {
return
}
reader, err := s.storage.Get(ctx, fileKey)
if err != nil {
s.logger.Warn().Err(err).Str("file_key", fileKey).Msg("kc: failed to read back file for extraction")
return
}
defer reader.Close()
data, err := io.ReadAll(reader)
if err != nil {
s.logger.Warn().Err(err).Msg("kc: failed to read file bytes")
return
}
result, err := kc.Extract(data)
if err != nil {
s.logger.Warn().Err(err).Str("part_number", item.PartNumber).Msg("kc: extraction failed")
return
}
if result == nil {
return // plain .fcstd, no silo/ directory
}
// Validate manifest UUID matches item
if result.Manifest != nil && result.Manifest.UUID != "" && result.Manifest.UUID != item.ID {
s.logger.Warn().
Str("manifest_uuid", result.Manifest.UUID).
Str("item_id", item.ID).
Msg("kc: manifest UUID does not match item, skipping indexing")
return
}
// Check for no-op (revision_hash unchanged)
if result.Manifest != nil && result.Manifest.RevisionHash != "" {
existing, _ := s.metadata.Get(ctx, item.ID)
if existing != nil && existing.RevisionHash != nil && *existing.RevisionHash == result.Manifest.RevisionHash {
s.logger.Debug().Str("part_number", item.PartNumber).Msg("kc: revision_hash unchanged, skipping")
return
}
}
username := ""
if rev.CreatedBy != nil {
username = *rev.CreatedBy
}
meta := &db.ItemMetadata{
ItemID: item.ID,
LifecycleState: "draft",
Fields: make(map[string]any),
Tags: []string{},
UpdatedBy: strPtr(username),
}
if result.Manifest != nil {
meta.KCVersion = strPtr(result.Manifest.KCVersion)
meta.ManifestUUID = strPtr(result.Manifest.UUID)
meta.SiloInstance = strPtr(result.Manifest.SiloInstance)
meta.RevisionHash = strPtr(result.Manifest.RevisionHash)
}
if result.Metadata != nil {
meta.SchemaName = strPtr(result.Metadata.SchemaName)
if result.Metadata.Tags != nil {
meta.Tags = result.Metadata.Tags
}
if result.Metadata.LifecycleState != "" {
meta.LifecycleState = result.Metadata.LifecycleState
}
if result.Metadata.Fields != nil {
meta.Fields = result.Metadata.Fields
}
}
if err := s.metadata.Upsert(ctx, meta); err != nil {
s.logger.Warn().Err(err).Str("part_number", item.PartNumber).Msg("kc: failed to upsert metadata")
return
}
s.broker.Publish("metadata.updated", mustMarshal(map[string]any{
"part_number": item.PartNumber,
"lifecycle_state": meta.LifecycleState,
"updated_by": username,
}))
// Index dependencies from silo/dependencies.json.
if result.Dependencies != nil {
dbDeps := make([]*db.ItemDependency, len(result.Dependencies))
for i, d := range result.Dependencies {
pn := d.PartNumber
rev := d.Revision
qty := d.Quantity
label := d.Label
rel := d.Relationship
if rel == "" {
rel = "component"
}
dbDeps[i] = &db.ItemDependency{
ParentItemID: item.ID,
ChildUUID: d.UUID,
ChildPartNumber: &pn,
ChildRevision: &rev,
Quantity: &qty,
Label: &label,
Relationship: rel,
}
}
if err := s.deps.ReplaceForRevision(ctx, item.ID, rev.RevisionNumber, dbDeps); err != nil {
s.logger.Warn().Err(err).Str("part_number", item.PartNumber).Msg("kc: failed to index dependencies")
} else {
s.broker.Publish("dependencies.changed", mustMarshal(map[string]any{
"part_number": item.PartNumber,
"count": len(dbDeps),
}))
}
}
// Index macros from silo/macros/*.
if len(result.Macros) > 0 {
dbMacros := make([]*db.ItemMacro, len(result.Macros))
for i, m := range result.Macros {
dbMacros[i] = &db.ItemMacro{
ItemID: item.ID,
Filename: m.Filename,
Trigger: "manual",
Content: m.Content,
}
}
if err := s.macros.ReplaceForItem(ctx, item.ID, rev.RevisionNumber, dbMacros); err != nil {
s.logger.Warn().Err(err).Str("part_number", item.PartNumber).Msg("kc: failed to index macros")
} else {
s.broker.Publish("macros.changed", mustMarshal(map[string]any{
"part_number": item.PartNumber,
"count": len(dbMacros),
}))
}
}
s.logger.Info().Str("part_number", item.PartNumber).Msg("kc: metadata indexed successfully")
}
// strPtr returns a pointer to s, or nil if s is empty.
func strPtr(s string) *string {
if s == "" {
return nil
}
return &s
}
// fieldKeys returns the keys from a map.
func fieldKeys(m map[string]any) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
return keys
}

View File

@@ -2,6 +2,8 @@
package api
import (
"crypto/sha256"
"encoding/hex"
"net/http"
"strings"
"time"
@@ -148,6 +150,55 @@ func (s *Server) RequireWritable(next http.Handler) http.Handler {
})
}
// RequireRunnerAuth extracts and validates a runner token from the
// Authorization header. On success, injects RunnerIdentity into context
// and updates the runner's heartbeat.
func (s *Server) RequireRunnerAuth(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := extractBearerToken(r)
if token == "" || !strings.HasPrefix(token, "silo_runner_") {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner token required")
return
}
hash := sha256.Sum256([]byte(token))
tokenHash := hex.EncodeToString(hash[:])
runner, err := s.jobs.GetRunnerByToken(r.Context(), tokenHash)
if err != nil || runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Invalid runner token")
return
}
// Update heartbeat on every authenticated request
_ = s.jobs.Heartbeat(r.Context(), runner.ID)
identity := &auth.RunnerIdentity{
ID: runner.ID,
Name: runner.Name,
Tags: runner.Tags,
}
ctx := auth.ContextWithRunner(r.Context(), identity)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// RequireModule returns middleware that rejects requests with 404 when
// the named module is not enabled.
func (s *Server) RequireModule(id string) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !s.modules.IsEnabled(id) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusNotFound)
w.Write([]byte(`{"error":"module '` + id + `' is not enabled"}`))
return
}
next.ServeHTTP(w, r)
})
}
}
func extractBearerToken(r *http.Request) string {
h := r.Header.Get("Authorization")
if strings.HasPrefix(h, "Bearer ") {

View File

@@ -138,21 +138,11 @@ func (s *Server) HandleExportODS(w http.ResponseWriter, r *http.Request) {
ods.StringCell(item.SourcingType),
}
if item.SourcingLink != nil {
cells = append(cells, ods.StringCell(*item.SourcingLink))
} else {
cells = append(cells, ods.EmptyCell())
}
if item.LongDescription != nil {
cells = append(cells, ods.StringCell(*item.LongDescription))
} else {
cells = append(cells, ods.EmptyCell())
}
if item.StandardCost != nil {
cells = append(cells, ods.CurrencyCell(*item.StandardCost))
} else {
cells = append(cells, ods.EmptyCell())
}
// Property columns
if includeProps {
@@ -419,6 +409,16 @@ func (s *Server) HandleImportODS(w http.ResponseWriter, r *http.Request) {
longDesc := getCellValue("long_description")
stdCostStr := getCellValue("standard_cost")
// Put sourcing_link and standard_cost into properties
if sourcingLink != "" {
properties["sourcing_link"] = sourcingLink
}
if stdCostStr != "" {
if cost, err := strconv.ParseFloat(strings.TrimLeft(stdCostStr, "$"), 64); err == nil {
properties["standard_cost"] = cost
}
}
item := &db.Item{
PartNumber: partNumber,
ItemType: itemType,
@@ -430,17 +430,9 @@ func (s *Server) HandleImportODS(w http.ResponseWriter, r *http.Request) {
if sourcingType != "" {
item.SourcingType = sourcingType
}
if sourcingLink != "" {
item.SourcingLink = &sourcingLink
}
if longDesc != "" {
item.LongDescription = &longDesc
}
if stdCostStr != "" {
if cost, err := strconv.ParseFloat(strings.TrimLeft(stdCostStr, "$"), 64); err == nil {
item.StandardCost = &cost
}
}
if err := s.items.Create(ctx, item, properties); err != nil {
result.Errors = append(result.Errors, CSVImportErr{
@@ -580,9 +572,16 @@ func (s *Server) HandleExportBOMODS(w http.ResponseWriter, r *http.Request) {
childItem, _ := s.items.GetByPartNumber(ctx, e.ChildPartNumber)
unitCost, hasUnitCost := getMetaFloat(e.Metadata, "unit_cost")
if !hasUnitCost && childItem != nil && childItem.StandardCost != nil {
unitCost = *childItem.StandardCost
hasUnitCost = true
if !hasUnitCost && childItem != nil {
// Fall back to standard_cost from revision properties
if childProps := itemPropsCache[e.ChildPartNumber]; childProps != nil {
if sc, ok := childProps["standard_cost"]; ok {
if cost, cok := sc.(float64); cok {
unitCost = cost
hasUnitCost = true
}
}
}
}
qty := 0.0
@@ -682,6 +681,21 @@ func (s *Server) HandleProjectSheetODS(w http.ResponseWriter, r *http.Request) {
return
}
// Build item properties cache for sourcing_link / standard_cost
itemPropsMap := make(map[string]map[string]any)
for _, item := range items {
revisions, err := s.items.GetRevisions(ctx, item.ID)
if err != nil {
continue
}
for _, rev := range revisions {
if rev.RevisionNumber == item.CurrentRevision && rev.Properties != nil {
itemPropsMap[item.ID] = rev.Properties
break
}
}
}
// Sheet 1: Items list
itemHeaders := []string{
"PN", "Type", "Description", "Revision", "Category",
@@ -696,6 +710,8 @@ func (s *Server) HandleProjectSheetODS(w http.ResponseWriter, r *http.Request) {
itemRows = append(itemRows, ods.Row{Cells: itemHeaderCells})
for _, item := range items {
props := itemPropsMap[item.ID]
cells := []ods.Cell{
ods.StringCell(item.PartNumber),
ods.StringCell(item.ItemType),
@@ -704,13 +720,17 @@ func (s *Server) HandleProjectSheetODS(w http.ResponseWriter, r *http.Request) {
ods.StringCell(parseCategory(item.PartNumber)),
ods.StringCell(item.SourcingType),
}
if item.SourcingLink != nil {
cells = append(cells, ods.StringCell(*item.SourcingLink))
if sl, ok := props["sourcing_link"]; ok {
cells = append(cells, ods.StringCell(formatPropertyValue(sl)))
} else {
cells = append(cells, ods.EmptyCell())
}
if item.StandardCost != nil {
cells = append(cells, ods.CurrencyCell(*item.StandardCost))
if sc, ok := props["standard_cost"]; ok {
if cost, cok := sc.(float64); cok {
cells = append(cells, ods.CurrencyCell(cost))
} else {
cells = append(cells, ods.StringCell(formatPropertyValue(sc)))
}
} else {
cells = append(cells, ods.EmptyCell())
}
@@ -746,9 +766,27 @@ func (s *Server) HandleProjectSheetODS(w http.ResponseWriter, r *http.Request) {
for _, e := range bomEntries {
childItem, _ := s.items.GetByPartNumber(ctx, e.ChildPartNumber)
unitCost, hasUnitCost := getMetaFloat(e.Metadata, "unit_cost")
if !hasUnitCost && childItem != nil && childItem.StandardCost != nil {
unitCost = *childItem.StandardCost
hasUnitCost = true
if !hasUnitCost && childItem != nil {
// Fall back to standard_cost from revision properties
// Ensure child item props are cached
if _, cached := itemPropsMap[childItem.ID]; !cached {
if revs, err := s.items.GetRevisions(ctx, childItem.ID); err == nil {
for _, rev := range revs {
if rev.RevisionNumber == childItem.CurrentRevision && rev.Properties != nil {
itemPropsMap[childItem.ID] = rev.Properties
break
}
}
}
}
if childRevProps := itemPropsMap[childItem.ID]; childRevProps != nil {
if sc, ok := childRevProps["standard_cost"]; ok {
if cost, cok := sc.(float64); cok {
unitCost = cost
hasUnitCost = true
}
}
}
}
qty := 0.0
if e.Quantity != nil {
@@ -957,7 +995,20 @@ func (s *Server) HandleSheetDiff(w http.ResponseWriter, r *http.Request) {
if costStr != "" {
costStr = strings.TrimLeft(costStr, "$")
if cost, err := strconv.ParseFloat(costStr, 64); err == nil {
if dbItem.StandardCost == nil || *dbItem.StandardCost != cost {
// Compare against standard_cost in revision properties
revisions, _ := s.items.GetRevisions(ctx, dbItem.ID)
var dbCost *float64
for _, rev := range revisions {
if rev.RevisionNumber == dbItem.CurrentRevision && rev.Properties != nil {
if sc, ok := rev.Properties["standard_cost"]; ok {
if c, cok := sc.(float64); cok {
dbCost = &c
}
}
break
}
}
if dbCost == nil || *dbCost != cost {
changes["standard_cost"] = cost
}
}
@@ -986,8 +1037,11 @@ func buildBOMRow(itemLabel string, depth int, source, pn string, item *db.Item,
if item != nil {
description = item.Description
if sourcingLink == "" && item.SourcingLink != nil {
sourcingLink = *item.SourcingLink
}
// Fall back to sourcing_link from revision properties
if sourcingLink == "" && props != nil {
if sl, ok := props["sourcing_link"]; ok {
sourcingLink = formatPropertyValue(sl)
}
}

View File

@@ -0,0 +1,90 @@
package api
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/db"
)
func newODSRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Get("/api/items/export.ods", s.HandleExportODS)
r.Get("/api/items/template.ods", s.HandleODSTemplate)
r.Post("/api/items/import.ods", s.HandleImportODS)
r.Get("/api/projects/{code}/sheet.ods", s.HandleProjectSheetODS)
return r
}
func TestHandleExportODS(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newODSRouter(s)
createItemDirect(t, s, "ODS-001", "ods export item", nil)
req := httptest.NewRequest("GET", "/api/items/export.ods", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
ct := w.Header().Get("Content-Type")
if !strings.Contains(ct, "application/vnd.oasis.opendocument.spreadsheet") {
t.Errorf("content-type: got %q, want ODS type", ct)
}
// ODS is a ZIP file — first 2 bytes should be PK
body := w.Body.Bytes()
if len(body) < 2 || body[0] != 'P' || body[1] != 'K' {
t.Error("response body does not start with PK (ZIP magic)")
}
}
func TestHandleODSTemplate(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newODSRouter(s)
req := httptest.NewRequest("GET", "/api/items/template.ods?schema=kindred-rd", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
ct := w.Header().Get("Content-Type")
if !strings.Contains(ct, "application/vnd.oasis.opendocument.spreadsheet") {
t.Errorf("content-type: got %q, want ODS type", ct)
}
}
func TestHandleProjectSheetODS(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newODSRouter(s)
// Create project and item
ctx := httptest.NewRequest("GET", "/", nil).Context()
proj := &db.Project{Code: "ODSPR", Name: "ODS Project"}
s.projects.Create(ctx, proj)
createItemDirect(t, s, "ODSPR-001", "project sheet item", nil)
item, _ := s.items.GetByPartNumber(ctx, "ODSPR-001")
s.projects.AddItemToProject(ctx, item.ID, proj.ID)
req := httptest.NewRequest("GET", "/api/projects/ODSPR/sheet.ods", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
ct := w.Header().Get("Content-Type")
if !strings.Contains(ct, "application/vnd.oasis.opendocument.spreadsheet") {
t.Errorf("content-type: got %q, want ODS type", ct)
}
}

View File

@@ -0,0 +1,182 @@
package api
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"time"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/kc"
)
// packKCFile gathers DB state and repacks silo/ entries in a .kc file.
func (s *Server) packKCFile(ctx context.Context, data []byte, item *db.Item, rev *db.Revision, meta *db.ItemMetadata) ([]byte, error) {
manifest := &kc.Manifest{
UUID: item.ID,
KCVersion: derefStr(meta.KCVersion, "1.0"),
RevisionHash: derefStr(meta.RevisionHash, ""),
SiloInstance: derefStr(meta.SiloInstance, ""),
}
metadata := &kc.Metadata{
SchemaName: derefStr(meta.SchemaName, ""),
Tags: meta.Tags,
LifecycleState: meta.LifecycleState,
Fields: meta.Fields,
}
// Build history from last 20 revisions.
revisions, err := s.items.GetRevisions(ctx, item.ID)
if err != nil {
return nil, fmt.Errorf("getting revisions: %w", err)
}
limit := 20
if len(revisions) < limit {
limit = len(revisions)
}
history := make([]kc.HistoryEntry, limit)
for i, r := range revisions[:limit] {
labels := r.Labels
if labels == nil {
labels = []string{}
}
history[i] = kc.HistoryEntry{
RevisionNumber: r.RevisionNumber,
CreatedAt: r.CreatedAt.UTC().Format(time.RFC3339),
CreatedBy: r.CreatedBy,
Comment: r.Comment,
Status: r.Status,
Labels: labels,
}
}
// Build dependencies from item_dependencies table.
var deps []kc.Dependency
dbDeps, err := s.deps.ListByItem(ctx, item.ID)
if err != nil {
s.logger.Warn().Err(err).Str("part_number", item.PartNumber).Msg("kc: failed to query dependencies for packing")
} else {
deps = make([]kc.Dependency, len(dbDeps))
for i, d := range dbDeps {
deps[i] = kc.Dependency{
UUID: d.ChildUUID,
PartNumber: derefStr(d.ChildPartNumber, ""),
Revision: derefInt(d.ChildRevision, 0),
Quantity: derefFloat(d.Quantity, 0),
Label: derefStr(d.Label, ""),
Relationship: d.Relationship,
}
}
}
if deps == nil {
deps = []kc.Dependency{}
}
// Build approvals from item_approvals table.
var approvals []kc.ApprovalEntry
dbApprovals, err := s.approvals.ListByItemWithSignatures(ctx, item.ID)
if err != nil {
s.logger.Warn().Err(err).Str("part_number", item.PartNumber).Msg("kc: failed to query approvals for packing")
} else {
approvals = make([]kc.ApprovalEntry, len(dbApprovals))
for i, a := range dbApprovals {
sigs := make([]kc.SignatureEntry, len(a.Signatures))
for j, sig := range a.Signatures {
var signedAt string
if sig.SignedAt != nil {
signedAt = sig.SignedAt.UTC().Format("2006-01-02T15:04:05Z")
}
var comment string
if sig.Comment != nil {
comment = *sig.Comment
}
sigs[j] = kc.SignatureEntry{
Username: sig.Username,
Role: sig.Role,
Status: sig.Status,
SignedAt: signedAt,
Comment: comment,
}
}
var ecoNumber string
if a.ECONumber != nil {
ecoNumber = *a.ECONumber
}
var updatedBy string
if a.UpdatedBy != nil {
updatedBy = *a.UpdatedBy
}
approvals[i] = kc.ApprovalEntry{
ID: a.ID,
WorkflowName: a.WorkflowName,
ECONumber: ecoNumber,
State: a.State,
UpdatedAt: a.UpdatedAt.UTC().Format("2006-01-02T15:04:05Z"),
UpdatedBy: updatedBy,
Signatures: sigs,
}
}
}
input := &kc.PackInput{
Manifest: manifest,
Metadata: metadata,
History: history,
Dependencies: deps,
Approvals: approvals,
}
return kc.Pack(data, input)
}
// computeETag generates a quoted ETag from the revision number and metadata freshness.
func computeETag(rev *db.Revision, meta *db.ItemMetadata) string {
var ts int64
if meta != nil {
ts = meta.UpdatedAt.UnixNano()
} else {
ts = rev.CreatedAt.UnixNano()
}
raw := fmt.Sprintf("%d:%d", rev.RevisionNumber, ts)
h := sha256.Sum256([]byte(raw))
return `"` + hex.EncodeToString(h[:8]) + `"`
}
// canSkipRepack returns true if the stored blob already has up-to-date silo/ data.
func canSkipRepack(rev *db.Revision, meta *db.ItemMetadata) bool {
if meta == nil {
return true // no metadata row = plain .fcstd
}
if meta.RevisionHash != nil && rev.FileChecksum != nil &&
*meta.RevisionHash == *rev.FileChecksum &&
meta.UpdatedAt.Before(rev.CreatedAt) {
return true
}
return false
}
// derefStr returns the value of a *string pointer, or fallback if nil.
func derefStr(p *string, fallback string) string {
if p != nil {
return *p
}
return fallback
}
// derefInt returns the value of a *int pointer, or fallback if nil.
func derefInt(p *int, fallback int) int {
if p != nil {
return *p
}
return fallback
}
// derefFloat returns the value of a *float64 pointer, or fallback if nil.
func derefFloat(p *float64, fallback float64) float64 {
if p != nil {
return *p
}
return fallback
}

View File

@@ -0,0 +1,222 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
)
func newRevisionRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Route("/api/items/{partNumber}", func(r chi.Router) {
r.Get("/revisions", s.HandleListRevisions)
r.Get("/revisions/compare", s.HandleCompareRevisions)
r.Get("/revisions/{revision}", s.HandleGetRevision)
r.Post("/revisions", s.HandleCreateRevision)
r.Patch("/revisions/{revision}", s.HandleUpdateRevision)
r.Post("/revisions/{revision}/rollback", s.HandleRollbackRevision)
})
return r
}
func TestHandleListRevisions(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
createItemDirect(t, s, "REV-API-001", "revision list", nil)
req := httptest.NewRequest("GET", "/api/items/REV-API-001/revisions", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var revisions []RevisionResponse
if err := json.Unmarshal(w.Body.Bytes(), &revisions); err != nil {
t.Fatalf("decoding response: %v", err)
}
if len(revisions) != 1 {
t.Errorf("expected 1 revision (initial), got %d", len(revisions))
}
}
func TestHandleListRevisionsNotFound(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
req := httptest.NewRequest("GET", "/api/items/NOEXIST/revisions", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("status: got %d, want %d", w.Code, http.StatusNotFound)
}
}
func TestHandleGetRevision(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
createItemDirect(t, s, "REVGET-001", "get revision", nil)
req := httptest.NewRequest("GET", "/api/items/REVGET-001/revisions/1", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var rev RevisionResponse
if err := json.Unmarshal(w.Body.Bytes(), &rev); err != nil {
t.Fatalf("decoding response: %v", err)
}
if rev.RevisionNumber != 1 {
t.Errorf("revision_number: got %d, want 1", rev.RevisionNumber)
}
}
func TestHandleGetRevisionNotFound(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
createItemDirect(t, s, "REVNF-001", "rev not found", nil)
req := httptest.NewRequest("GET", "/api/items/REVNF-001/revisions/99", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("status: got %d, want %d", w.Code, http.StatusNotFound)
}
}
func TestHandleCreateRevision(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
createItemDirect(t, s, "REVCR-001", "create revision", nil)
body := `{"properties":{"material":"steel"},"comment":"added material"}`
req := authRequest(httptest.NewRequest("POST", "/api/items/REVCR-001/revisions", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusCreated, w.Body.String())
}
var rev RevisionResponse
if err := json.Unmarshal(w.Body.Bytes(), &rev); err != nil {
t.Fatalf("decoding response: %v", err)
}
if rev.RevisionNumber != 2 {
t.Errorf("revision_number: got %d, want 2", rev.RevisionNumber)
}
}
func TestHandleUpdateRevision(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
createItemDirect(t, s, "REVUP-001", "update revision", nil)
body := `{"status":"released","labels":["production"]}`
req := authRequest(httptest.NewRequest("PATCH", "/api/items/REVUP-001/revisions/1", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var rev RevisionResponse
if err := json.Unmarshal(w.Body.Bytes(), &rev); err != nil {
t.Fatalf("decoding response: %v", err)
}
if rev.Status != "released" {
t.Errorf("status: got %q, want %q", rev.Status, "released")
}
if len(rev.Labels) != 1 || rev.Labels[0] != "production" {
t.Errorf("labels: got %v, want [production]", rev.Labels)
}
}
func TestHandleCompareRevisions(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
// Create item with properties, then create second revision with changed properties
cost := 10.0
createItemDirect(t, s, "REVCMP-001", "compare revisions", &cost)
body := `{"properties":{"standard_cost":20,"material":"aluminum"},"comment":"updated cost"}`
req := authRequest(httptest.NewRequest("POST", "/api/items/REVCMP-001/revisions", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create rev 2: status %d; body: %s", w.Code, w.Body.String())
}
// Compare rev 1 vs rev 2
req = httptest.NewRequest("GET", "/api/items/REVCMP-001/revisions/compare?from=1&to=2", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var diff RevisionDiffResponse
if err := json.Unmarshal(w.Body.Bytes(), &diff); err != nil {
t.Fatalf("decoding response: %v", err)
}
if diff.FromRevision != 1 || diff.ToRevision != 2 {
t.Errorf("revisions: got from=%d to=%d, want from=1 to=2", diff.FromRevision, diff.ToRevision)
}
}
func TestHandleRollbackRevision(t *testing.T) {
s := newTestServer(t)
router := newRevisionRouter(s)
createItemDirect(t, s, "REVRB-001", "rollback test", nil)
// Create rev 2
body := `{"properties":{"version":"v2"},"comment":"version 2"}`
req := authRequest(httptest.NewRequest("POST", "/api/items/REVRB-001/revisions", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("create rev 2: status %d; body: %s", w.Code, w.Body.String())
}
// Rollback to rev 1 — should create rev 3
body = `{"comment":"rolling back"}`
req = authRequest(httptest.NewRequest("POST", "/api/items/REVRB-001/revisions/1/rollback", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusCreated {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusCreated, w.Body.String())
}
var rev RevisionResponse
if err := json.Unmarshal(w.Body.Bytes(), &rev); err != nil {
t.Fatalf("decoding response: %v", err)
}
if rev.RevisionNumber != 3 {
t.Errorf("revision_number: got %d, want 3", rev.RevisionNumber)
}
}

View File

@@ -58,6 +58,7 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
r.Get("/auth/callback", server.HandleOIDCCallback)
// Public API endpoints (no auth required)
r.Get("/api/modules", server.HandleGetModules)
r.Get("/api/auth/config", server.HandleAuthConfig)
// API routes (require auth, no CSRF — token auth instead)
@@ -67,6 +68,23 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
// SSE event stream (viewer+)
r.Get("/events", server.HandleEvents)
// Workflows (viewer+)
r.Get("/workflows", server.HandleListWorkflows)
// Workstations (gated by sessions module)
r.Route("/workstations", func(r chi.Router) {
r.Use(server.RequireModule("sessions"))
r.Get("/", server.HandleListWorkstations)
r.Post("/", server.HandleRegisterWorkstation)
r.Delete("/{id}", server.HandleDeleteWorkstation)
})
// Edit sessions — current user's active sessions (gated by sessions module)
r.Route("/edit-sessions", func(r chi.Router) {
r.Use(server.RequireModule("sessions"))
r.Get("/", server.HandleListUserEditSessions)
})
// Auth endpoints
r.Get("/auth/me", server.HandleGetCurrentUser)
r.Route("/auth/tokens", func(r chi.Router) {
@@ -86,7 +104,7 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
r.Route("/schemas", func(r chi.Router) {
r.Get("/", server.HandleListSchemas)
r.Get("/{name}", server.HandleGetSchema)
r.Get("/{name}/properties", server.HandleGetPropertySchema)
r.Get("/{name}/form", server.HandleGetFormDescriptor)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
@@ -101,6 +119,7 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
// Projects (read: viewer, write: editor)
r.Route("/projects", func(r chi.Router) {
r.Use(server.RequireModule("projects"))
r.Get("/", server.HandleListProjects)
r.Get("/{code}", server.HandleGetProject)
r.Get("/{code}/items", server.HandleGetProjectItems)
@@ -115,6 +134,26 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
})
})
// Locations (read: viewer, write: editor)
r.Route("/locations", func(r chi.Router) {
r.Get("/", server.HandleListLocations)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
r.Use(server.RequireRole(auth.RoleEditor))
r.Post("/", server.HandleCreateLocation)
})
// Wildcard routes for path-based lookup (e.g., /api/locations/lab/shelf-a/bin-3)
r.Get("/*", server.HandleGetLocation)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
r.Use(server.RequireRole(auth.RoleEditor))
r.Put("/*", server.HandleUpdateLocation)
r.Delete("/*", server.HandleDeleteLocation)
})
})
// Items (read: viewer, write: editor)
r.Route("/items", func(r chi.Router) {
r.Get("/", server.HandleListItems)
@@ -140,6 +179,7 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
r.Get("/revisions/compare", server.HandleCompareRevisions)
r.Get("/revisions/{revision}", server.HandleGetRevision)
r.Get("/files", server.HandleListItemFiles)
r.Get("/files/{fileId}/download", server.HandleDownloadItemFile)
r.Get("/file", server.HandleDownloadLatestFile)
r.Get("/file/{revision}", server.HandleDownloadFile)
r.Get("/bom", server.HandleGetBOM)
@@ -149,6 +189,41 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
r.Get("/bom/where-used", server.HandleGetWhereUsed)
r.Get("/bom/export.csv", server.HandleExportBOMCSV)
r.Get("/bom/export.ods", server.HandleExportBOMODS)
r.Get("/metadata", server.HandleGetMetadata)
r.Get("/dependencies", server.HandleGetDependencies)
r.Get("/dependencies/resolve", server.HandleResolveDependencies)
r.Get("/macros", server.HandleGetMacros)
r.Get("/macros/{filename}", server.HandleGetMacro)
r.Get("/approvals", server.HandleGetApprovals)
r.Get("/solver/results", server.HandleGetSolverResults)
// DAG (gated by dag module)
r.Route("/dag", func(r chi.Router) {
r.Use(server.RequireModule("dag"))
r.Get("/", server.HandleGetDAG)
r.Get("/forward-cone/{nodeKey}", server.HandleGetForwardCone)
r.Get("/dirty", server.HandleGetDirtySubgraph)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
r.Use(server.RequireRole(auth.RoleEditor))
r.Put("/", server.HandleSyncDAG)
r.Post("/mark-dirty/{nodeKey}", server.HandleMarkDirty)
})
})
// Edit sessions (gated by sessions module)
r.Route("/edit-sessions", func(r chi.Router) {
r.Use(server.RequireModule("sessions"))
r.Get("/", server.HandleListItemEditSessions)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
r.Use(server.RequireRole(auth.RoleEditor))
r.Post("/", server.HandleAcquireEditSession)
r.Delete("/{sessionID}", server.HandleReleaseEditSession)
})
})
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
@@ -162,24 +237,34 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
r.Post("/revisions/{revision}/rollback", server.HandleRollbackRevision)
r.Post("/file", server.HandleUploadFile)
r.Post("/files", server.HandleAssociateItemFile)
r.Post("/files/upload", server.HandleUploadItemFile)
r.Delete("/files/{fileId}", server.HandleDeleteItemFile)
r.Put("/thumbnail", server.HandleSetItemThumbnail)
r.Post("/thumbnail/upload", server.HandleUploadItemThumbnail)
r.Post("/bom", server.HandleAddBOMEntry)
r.Post("/bom/import", server.HandleImportBOMCSV)
r.Post("/bom/merge", server.HandleMergeBOM)
r.Put("/bom/{childPartNumber}", server.HandleUpdateBOMEntry)
r.Delete("/bom/{childPartNumber}", server.HandleDeleteBOMEntry)
r.Put("/metadata", server.HandleUpdateMetadata)
r.Patch("/metadata/lifecycle", server.HandleUpdateLifecycle)
r.Patch("/metadata/tags", server.HandleUpdateTags)
r.Post("/approvals", server.HandleCreateApproval)
r.Post("/approvals/{id}/sign", server.HandleSignApproval)
})
})
})
// Audit (read-only, viewer role)
r.Route("/audit", func(r chi.Router) {
r.Use(server.RequireModule("audit"))
r.Get("/completeness", server.HandleAuditCompleteness)
r.Get("/completeness/{partNumber}", server.HandleAuditItemDetail)
})
// Integrations (read: viewer, write: editor)
r.Route("/integrations/odoo", func(r chi.Router) {
r.Use(server.RequireModule("odoo"))
r.Get("/config", server.HandleGetOdooConfig)
r.Get("/sync-log", server.HandleGetOdooSyncLog)
@@ -193,6 +278,21 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
})
})
// Solver (gated by solver module)
r.Route("/solver", func(r chi.Router) {
r.Use(server.RequireModule("solver"))
r.Get("/solvers", server.HandleGetSolverRegistry)
r.Get("/jobs", server.HandleListSolverJobs)
r.Get("/jobs/{jobID}", server.HandleGetSolverJob)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
r.Use(server.RequireRole(auth.RoleEditor))
r.Post("/jobs", server.HandleSubmitSolverJob)
r.Post("/jobs/{jobID}/cancel", server.HandleCancelSolverJob)
})
})
// Sheets (editor)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
@@ -200,12 +300,71 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
r.Post("/sheets/diff", server.HandleSheetDiff)
})
// Jobs (read: viewer, write: editor)
r.Route("/jobs", func(r chi.Router) {
r.Use(server.RequireModule("jobs"))
r.Get("/", server.HandleListJobs)
r.Get("/{jobID}", server.HandleGetJob)
r.Get("/{jobID}/logs", server.HandleGetJobLogs)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
r.Use(server.RequireRole(auth.RoleEditor))
r.Post("/", server.HandleCreateJob)
r.Post("/{jobID}/cancel", server.HandleCancelJob)
})
})
// Job definitions (read: viewer, reload: admin)
r.Route("/job-definitions", func(r chi.Router) {
r.Use(server.RequireModule("jobs"))
r.Get("/", server.HandleListJobDefinitions)
r.Get("/{name}", server.HandleGetJobDefinition)
r.Group(func(r chi.Router) {
r.Use(server.RequireRole(auth.RoleAdmin))
r.Post("/reload", server.HandleReloadJobDefinitions)
})
})
// Runners (admin)
r.Route("/runners", func(r chi.Router) {
r.Use(server.RequireModule("jobs"))
r.Use(server.RequireRole(auth.RoleAdmin))
r.Get("/", server.HandleListRunners)
r.Post("/", server.HandleRegisterRunner)
r.Delete("/{runnerID}", server.HandleDeleteRunner)
})
// Part number generation (editor)
r.Group(func(r chi.Router) {
r.Use(server.RequireWritable)
r.Use(server.RequireRole(auth.RoleEditor))
r.Post("/generate-part-number", server.HandleGeneratePartNumber)
})
// Admin settings (admin only)
r.Route("/admin/settings", func(r chi.Router) {
r.Use(server.RequireRole(auth.RoleAdmin))
r.Get("/", server.HandleGetAllSettings)
r.Get("/{module}", server.HandleGetModuleSettings)
r.Put("/{module}", server.HandleUpdateModuleSettings)
r.Post("/{module}/test", server.HandleTestModuleConnectivity)
})
})
// Runner-facing API (runner token auth, not user auth)
r.Route("/api/runner", func(r chi.Router) {
r.Use(server.RequireModule("jobs"))
r.Use(server.RequireRunnerAuth)
r.Post("/heartbeat", server.HandleRunnerHeartbeat)
r.Post("/claim", server.HandleRunnerClaim)
r.Post("/jobs/{jobID}/start", server.HandleRunnerStartJob)
r.Put("/jobs/{jobID}/progress", server.HandleRunnerUpdateProgress)
r.Post("/jobs/{jobID}/complete", server.HandleRunnerCompleteJob)
r.Post("/jobs/{jobID}/fail", server.HandleRunnerFailJob)
r.Post("/jobs/{jobID}/log", server.HandleRunnerAppendLog)
r.Put("/jobs/{jobID}/dag", server.HandleRunnerSyncDAG)
})
// React SPA — serve from web/dist at root, fallback to index.html

View File

@@ -0,0 +1,388 @@
package api
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/jobdef"
)
// HandleRunnerHeartbeat updates the runner's heartbeat timestamp.
func (s *Server) HandleRunnerHeartbeat(w http.ResponseWriter, r *http.Request) {
runner := auth.RunnerFromContext(r.Context())
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
// Heartbeat already updated by RequireRunnerAuth middleware
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
// HandleRunnerClaim claims the next available job matching the runner's tags.
func (s *Server) HandleRunnerClaim(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
runner := auth.RunnerFromContext(ctx)
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
job, err := s.jobs.ClaimJob(ctx, runner.ID, runner.Tags)
if err != nil {
s.logger.Error().Err(err).Msg("failed to claim job")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to claim job")
return
}
if job == nil {
writeJSON(w, http.StatusNoContent, nil)
return
}
// Look up the full definition to send to the runner
var defPayload map[string]any
if job.JobDefinitionID != nil {
rec, err := s.jobs.GetDefinitionByID(ctx, *job.JobDefinitionID)
if err == nil && rec != nil {
defPayload = rec.Definition
}
}
s.broker.Publish("job.claimed", mustMarshal(map[string]any{
"job_id": job.ID,
"runner_id": runner.ID,
"runner": runner.Name,
}))
writeJSON(w, http.StatusOK, map[string]any{
"job": job,
"definition": defPayload,
})
}
// HandleRunnerStartJob transitions a claimed job to running.
func (s *Server) HandleRunnerStartJob(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
runner := auth.RunnerFromContext(ctx)
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
jobID := chi.URLParam(r, "jobID")
if err := s.jobs.StartJob(ctx, jobID, runner.ID); err != nil {
writeError(w, http.StatusBadRequest, "start_failed", err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "running"})
}
// HandleRunnerUpdateProgress updates a running job's progress.
func (s *Server) HandleRunnerUpdateProgress(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
runner := auth.RunnerFromContext(ctx)
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
jobID := chi.URLParam(r, "jobID")
var req struct {
Progress int `json:"progress"`
Message string `json:"message,omitempty"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if err := s.jobs.UpdateProgress(ctx, jobID, runner.ID, req.Progress, req.Message); err != nil {
writeError(w, http.StatusBadRequest, "update_failed", err.Error())
return
}
s.broker.Publish("job.progress", mustMarshal(map[string]any{
"job_id": jobID,
"progress": req.Progress,
"message": req.Message,
}))
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
// HandleRunnerCompleteJob marks a job as completed.
func (s *Server) HandleRunnerCompleteJob(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
runner := auth.RunnerFromContext(ctx)
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
jobID := chi.URLParam(r, "jobID")
var req struct {
Result map[string]any `json:"result,omitempty"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if err := s.jobs.CompleteJob(ctx, jobID, runner.ID, req.Result); err != nil {
writeError(w, http.StatusBadRequest, "complete_failed", err.Error())
return
}
// Cache solver results asynchronously (no-op for non-solver jobs).
go s.maybeCacheSolverResult(context.Background(), jobID)
s.broker.Publish("job.completed", mustMarshal(map[string]any{
"job_id": jobID,
"runner_id": runner.ID,
}))
writeJSON(w, http.StatusOK, map[string]string{"status": "completed"})
}
// HandleRunnerFailJob marks a job as failed.
func (s *Server) HandleRunnerFailJob(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
runner := auth.RunnerFromContext(ctx)
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
jobID := chi.URLParam(r, "jobID")
var req struct {
Error string `json:"error"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if err := s.jobs.FailJob(ctx, jobID, runner.ID, req.Error); err != nil {
writeError(w, http.StatusBadRequest, "fail_failed", err.Error())
return
}
s.broker.Publish("job.failed", mustMarshal(map[string]any{
"job_id": jobID,
"runner_id": runner.ID,
"error": req.Error,
}))
writeJSON(w, http.StatusOK, map[string]string{"status": "failed"})
}
// HandleRunnerAppendLog appends a log entry to a job.
func (s *Server) HandleRunnerAppendLog(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
runner := auth.RunnerFromContext(ctx)
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
jobID := chi.URLParam(r, "jobID")
var req struct {
Level string `json:"level"`
Message string `json:"message"`
Metadata map[string]any `json:"metadata,omitempty"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if req.Level == "" {
req.Level = "info"
}
entry := &db.JobLogEntry{
JobID: jobID,
Level: req.Level,
Message: req.Message,
Metadata: req.Metadata,
}
if err := s.jobs.AppendLog(ctx, entry); err != nil {
s.logger.Error().Err(err).Msg("failed to append job log")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to append log")
return
}
writeJSON(w, http.StatusCreated, entry)
}
// HandleRunnerSyncDAG allows a runner to push DAG results for a job's item.
func (s *Server) HandleRunnerSyncDAG(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
runner := auth.RunnerFromContext(ctx)
if runner == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Runner identity required")
return
}
jobID := chi.URLParam(r, "jobID")
// Get the job to find the item
job, err := s.jobs.GetJob(ctx, jobID)
if err != nil || job == nil {
writeError(w, http.StatusNotFound, "not_found", "Job not found")
return
}
if job.ItemID == nil {
writeError(w, http.StatusBadRequest, "no_item", "Job has no associated item")
return
}
// Delegate to the DAG sync handler logic
var req dagSyncRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
if req.RevisionNumber == 0 {
// Look up current revision
item, err := s.items.GetByID(ctx, *job.ItemID)
if err != nil || item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
req.RevisionNumber = item.CurrentRevision
}
// Convert and sync nodes
nodes := make([]db.DAGNode, len(req.Nodes))
for i, n := range req.Nodes {
state := n.ValidationState
if state == "" {
state = "clean"
}
nodes[i] = db.DAGNode{
NodeKey: n.NodeKey,
NodeType: n.NodeType,
PropertiesHash: n.PropertiesHash,
ValidationState: state,
Metadata: n.Metadata,
}
}
if err := s.dag.SyncFeatureTree(ctx, *job.ItemID, req.RevisionNumber, nodes, nil); err != nil {
s.logger.Error().Err(err).Msg("failed to sync DAG from runner")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to sync DAG")
return
}
// Build key→ID map and sync edges
keyToID := make(map[string]string, len(nodes))
for _, n := range nodes {
keyToID[n.NodeKey] = n.ID
}
if len(req.Edges) > 0 {
if err := s.dag.DeleteEdgesForItem(ctx, *job.ItemID, req.RevisionNumber); err != nil {
s.logger.Error().Err(err).Msg("failed to delete old edges")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to sync DAG edges")
return
}
for _, e := range req.Edges {
sourceID, ok := keyToID[e.SourceKey]
if !ok {
continue
}
targetID, ok := keyToID[e.TargetKey]
if !ok {
continue
}
edgeType := e.EdgeType
if edgeType == "" {
edgeType = "depends_on"
}
edge := &db.DAGEdge{
SourceNodeID: sourceID,
TargetNodeID: targetID,
EdgeType: edgeType,
Metadata: e.Metadata,
}
if err := s.dag.CreateEdge(ctx, edge); err != nil {
s.logger.Error().Err(err).Msg("failed to create edge from runner")
}
}
}
s.broker.Publish("dag.updated", mustMarshal(map[string]any{
"item_id": *job.ItemID,
"job_id": jobID,
"runner": runner.Name,
"node_count": len(req.Nodes),
"edge_count": len(req.Edges),
}))
writeJSON(w, http.StatusOK, map[string]any{
"synced": true,
"node_count": len(req.Nodes),
"edge_count": len(req.Edges),
})
}
// generateRunnerToken creates a new runner token. Returns raw token, hash, and prefix.
func generateRunnerToken() (raw, hash, prefix string) {
rawBytes := make([]byte, 32)
if _, err := rand.Read(rawBytes); err != nil {
panic(fmt.Sprintf("generating random bytes: %v", err))
}
raw = "silo_runner_" + hex.EncodeToString(rawBytes)
h := sha256.Sum256([]byte(raw))
hash = hex.EncodeToString(h[:])
prefix = raw[:20] // "silo_runner_" + first 8 hex chars
return
}
// loadAndUpsertJobDefs loads YAML definitions from a directory and upserts them into the database.
func loadAndUpsertJobDefs(ctx context.Context, dir string, repo *db.JobRepository) (map[string]*jobdef.Definition, error) {
defs, err := jobdef.LoadAll(dir)
if err != nil {
return nil, fmt.Errorf("loading job definitions: %w", err)
}
for _, def := range defs {
defJSON, _ := json.Marshal(def)
var defMap map[string]any
json.Unmarshal(defJSON, &defMap)
rec := &db.JobDefinitionRecord{
Name: def.Name,
Version: def.Version,
TriggerType: def.Trigger.Type,
ScopeType: def.Scope.Type,
ComputeType: def.Compute.Type,
RunnerTags: def.Runner.Tags,
TimeoutSeconds: def.Timeout,
MaxRetries: def.MaxRetries,
Priority: def.Priority,
Definition: defMap,
Enabled: true,
}
if err := repo.UpsertDefinition(ctx, rec); err != nil {
return nil, fmt.Errorf("upserting definition %s: %w", def.Name, err)
}
}
return defs, nil
}

View File

@@ -0,0 +1,100 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
)
func newSchemaRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Get("/api/schemas", s.HandleListSchemas)
r.Get("/api/schemas/{name}", s.HandleGetSchema)
r.Get("/api/schemas/{name}/form", s.HandleGetFormDescriptor)
return r
}
func TestHandleListSchemas(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newSchemaRouter(s)
req := httptest.NewRequest("GET", "/api/schemas", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var schemas []map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &schemas); err != nil {
t.Fatalf("decoding response: %v", err)
}
if len(schemas) == 0 {
t.Error("expected at least 1 schema")
}
}
func TestHandleGetSchema(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newSchemaRouter(s)
req := httptest.NewRequest("GET", "/api/schemas/kindred-rd", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var schema map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &schema); err != nil {
t.Fatalf("decoding response: %v", err)
}
if schema["name"] != "kindred-rd" {
t.Errorf("name: got %v, want %q", schema["name"], "kindred-rd")
}
}
func TestHandleGetSchemaNotFound(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newSchemaRouter(s)
req := httptest.NewRequest("GET", "/api/schemas/nonexistent", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("status: got %d, want %d", w.Code, http.StatusNotFound)
}
}
func TestHandleGetFormDescriptor(t *testing.T) {
s := newTestServerWithSchemas(t)
router := newSchemaRouter(s)
req := httptest.NewRequest("GET", "/api/schemas/kindred-rd/form", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var form map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &form); err != nil {
t.Fatalf("decoding response: %v", err)
}
// Form descriptor should have fields
if _, ok := form["fields"]; !ok {
// Some schemas may use "categories" or "segments" instead
if _, ok := form["categories"]; !ok {
if _, ok := form["segments"]; !ok {
t.Error("form descriptor missing fields/categories/segments key")
}
}
}
}

View File

@@ -26,13 +26,13 @@ type ServerState struct {
mu sync.RWMutex
readOnly bool
storageOK bool
storage *storage.Storage
storage storage.FileStore
broker *Broker
done chan struct{}
}
// NewServerState creates a new server state tracker.
func NewServerState(logger zerolog.Logger, store *storage.Storage, broker *Broker) *ServerState {
func NewServerState(logger zerolog.Logger, store storage.FileStore, broker *Broker) *ServerState {
return &ServerState{
logger: logger.With().Str("component", "server-state").Logger(),
storageOK: store != nil, // assume healthy if configured
@@ -86,7 +86,7 @@ func (ss *ServerState) ToggleReadOnly() {
ss.SetReadOnly(!current)
}
// StartStorageHealthCheck launches a periodic check of MinIO reachability.
// StartStorageHealthCheck launches a periodic check of storage reachability.
// Updates storageOK and broadcasts server.state on transitions.
func (ss *ServerState) StartStorageHealthCheck() {
if ss.storage == nil {

View File

@@ -76,7 +76,7 @@ func TestServerStateToggleReadOnly(t *testing.T) {
func TestServerStateBroadcastsOnTransition(t *testing.T) {
b := NewBroker(zerolog.Nop())
c := b.Subscribe()
c := b.Subscribe("", "")
defer b.Unsubscribe(c)
ss := NewServerState(zerolog.Nop(), nil, b)

View File

@@ -0,0 +1,293 @@
package api
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/modules"
)
var validContextLevels = map[string]bool{
"sketch": true,
"partdesign": true,
"assembly": true,
}
type editSessionResponse struct {
ID string `json:"id"`
ItemID string `json:"item_id"`
PartNumber string `json:"part_number,omitempty"`
UserID string `json:"user_id"`
WorkstationID string `json:"workstation_id"`
ContextLevel string `json:"context_level"`
ObjectID *string `json:"object_id"`
DependCone []string `json:"dependency_cone"`
AcquiredAt string `json:"acquired_at"`
LastHeartbeat string `json:"last_heartbeat"`
}
func sessionToResponse(s *db.EditSession, partNumber string) editSessionResponse {
cone := s.DependencyCone
if cone == nil {
cone = []string{}
}
return editSessionResponse{
ID: s.ID,
ItemID: s.ItemID,
PartNumber: partNumber,
UserID: s.UserID,
WorkstationID: s.WorkstationID,
ContextLevel: s.ContextLevel,
ObjectID: s.ObjectID,
DependCone: cone,
AcquiredAt: s.AcquiredAt.UTC().Format("2006-01-02T15:04:05Z"),
LastHeartbeat: s.LastHeartbeat.UTC().Format("2006-01-02T15:04:05Z"),
}
}
// HandleAcquireEditSession acquires an edit session on an item.
// POST /api/items/{partNumber}/edit-sessions
func (s *Server) HandleAcquireEditSession(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
user := auth.UserFromContext(ctx)
if user == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
var req struct {
WorkstationID string `json:"workstation_id"`
ContextLevel string `json:"context_level"`
ObjectID *string `json:"object_id"`
DependencyCone []string `json:"dependency_cone"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_json", err.Error())
return
}
if req.WorkstationID == "" {
writeError(w, http.StatusBadRequest, "validation_error", "workstation_id is required")
return
}
if !validContextLevels[req.ContextLevel] {
writeError(w, http.StatusBadRequest, "validation_error", "context_level must be sketch, partdesign, or assembly")
return
}
// If no dependency cone provided and DAG module is enabled, attempt to compute it.
depCone := req.DependencyCone
if len(depCone) == 0 && req.ObjectID != nil && s.modules.IsEnabled(modules.DAG) {
node, nodeErr := s.dag.GetNodeByKey(ctx, item.ID, item.CurrentRevision, *req.ObjectID)
if nodeErr == nil && node != nil {
coneNodes, coneErr := s.dag.GetForwardCone(ctx, node.ID)
if coneErr == nil {
depCone = make([]string, len(coneNodes))
for i, n := range coneNodes {
depCone[i] = n.NodeKey
}
}
}
}
session := &db.EditSession{
ItemID: item.ID,
UserID: user.ID,
WorkstationID: req.WorkstationID,
ContextLevel: req.ContextLevel,
ObjectID: req.ObjectID,
DependencyCone: depCone,
}
if err := s.editSessions.Acquire(ctx, session); err != nil {
// Check for unique constraint violation (hard interference).
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) && pgErr.Code == "23505" {
s.writeConflictResponse(w, r, item.ID, req.ContextLevel, req.ObjectID)
return
}
s.logger.Error().Err(err).Msg("failed to acquire edit session")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to acquire edit session")
return
}
s.broker.PublishToItem(item.ID, "edit.session_acquired", mustMarshal(map[string]any{
"session_id": session.ID,
"item_id": item.ID,
"part_number": partNumber,
"user": user.Username,
"workstation": req.WorkstationID,
"context_level": session.ContextLevel,
"object_id": session.ObjectID,
}))
writeJSON(w, http.StatusOK, sessionToResponse(session, partNumber))
}
// writeConflictResponse builds a 409 response with holder info.
func (s *Server) writeConflictResponse(w http.ResponseWriter, r *http.Request, itemID, contextLevel string, objectID *string) {
ctx := r.Context()
conflict, err := s.editSessions.GetConflict(ctx, itemID, contextLevel, objectID)
if err != nil || conflict == nil {
writeError(w, http.StatusConflict, "hard_interference", "Another user is editing this object")
return
}
// Look up holder's username and workstation name.
holderUser := "unknown"
if u, err := s.auth.GetUserByID(ctx, conflict.UserID); err == nil && u != nil {
holderUser = u.Username
}
holderWS := conflict.WorkstationID
if ws, err := s.workstations.GetByID(ctx, conflict.WorkstationID); err == nil && ws != nil {
holderWS = ws.Name
}
objDesc := contextLevel
if objectID != nil {
objDesc = *objectID
}
writeJSON(w, http.StatusConflict, map[string]any{
"error": "hard_interference",
"holder": map[string]any{
"user": holderUser,
"workstation": holderWS,
"context_level": conflict.ContextLevel,
"object_id": conflict.ObjectID,
"acquired_at": conflict.AcquiredAt.UTC().Format("2006-01-02T15:04:05Z"),
},
"message": fmt.Sprintf("%s is currently editing %s", holderUser, objDesc),
})
}
// HandleReleaseEditSession releases an edit session.
// DELETE /api/items/{partNumber}/edit-sessions/{sessionID}
func (s *Server) HandleReleaseEditSession(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
sessionID := chi.URLParam(r, "sessionID")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
user := auth.UserFromContext(ctx)
if user == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
session, err := s.editSessions.GetByID(ctx, sessionID)
if err != nil {
s.logger.Error().Err(err).Str("session_id", sessionID).Msg("failed to get edit session")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get edit session")
return
}
if session == nil {
writeError(w, http.StatusNotFound, "not_found", "Edit session not found")
return
}
if session.UserID != user.ID && user.Role != auth.RoleAdmin {
writeError(w, http.StatusForbidden, "forbidden", "You can only release your own edit sessions")
return
}
if err := s.editSessions.Release(ctx, sessionID); err != nil {
s.logger.Error().Err(err).Str("session_id", sessionID).Msg("failed to release edit session")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to release edit session")
return
}
s.broker.PublishToItem(item.ID, "edit.session_released", mustMarshal(map[string]any{
"session_id": session.ID,
"item_id": item.ID,
"part_number": partNumber,
"user": user.Username,
"context_level": session.ContextLevel,
"object_id": session.ObjectID,
}))
w.WriteHeader(http.StatusNoContent)
}
// HandleListItemEditSessions lists active edit sessions for an item.
// GET /api/items/{partNumber}/edit-sessions
func (s *Server) HandleListItemEditSessions(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
sessions, err := s.editSessions.ListForItem(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list edit sessions")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list edit sessions")
return
}
out := make([]editSessionResponse, len(sessions))
for i, sess := range sessions {
out[i] = sessionToResponse(sess, partNumber)
}
writeJSON(w, http.StatusOK, out)
}
// HandleListUserEditSessions lists active edit sessions for the current user.
// GET /api/edit-sessions
func (s *Server) HandleListUserEditSessions(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := auth.UserFromContext(ctx)
if user == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
sessions, err := s.editSessions.ListForUser(ctx, user.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list edit sessions")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list edit sessions")
return
}
out := make([]editSessionResponse, len(sessions))
for i, sess := range sessions {
out[i] = sessionToResponse(sess, "")
}
writeJSON(w, http.StatusOK, out)
}

View File

@@ -0,0 +1,314 @@
package api
import (
"context"
"encoding/json"
"net/http"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
)
// HandleGetAllSettings returns the full config grouped by module with secrets redacted.
func (s *Server) HandleGetAllSettings(w http.ResponseWriter, r *http.Request) {
resp := map[string]any{
"core": s.buildCoreSettings(),
"schemas": s.buildSchemasSettings(),
"storage": s.buildStorageSettings(r.Context()),
"database": s.buildDatabaseSettings(r.Context()),
"auth": s.buildAuthSettings(),
"projects": map[string]any{"enabled": s.modules.IsEnabled("projects")},
"audit": map[string]any{"enabled": s.modules.IsEnabled("audit")},
"odoo": s.buildOdooSettings(),
"freecad": s.buildFreecadSettings(),
"jobs": s.buildJobsSettings(),
"dag": map[string]any{"enabled": s.modules.IsEnabled("dag")},
}
writeJSON(w, http.StatusOK, resp)
}
// HandleGetModuleSettings returns settings for a single module.
func (s *Server) HandleGetModuleSettings(w http.ResponseWriter, r *http.Request) {
module := chi.URLParam(r, "module")
var settings any
switch module {
case "core":
settings = s.buildCoreSettings()
case "schemas":
settings = s.buildSchemasSettings()
case "storage":
settings = s.buildStorageSettings(r.Context())
case "database":
settings = s.buildDatabaseSettings(r.Context())
case "auth":
settings = s.buildAuthSettings()
case "projects":
settings = map[string]any{"enabled": s.modules.IsEnabled("projects")}
case "audit":
settings = map[string]any{"enabled": s.modules.IsEnabled("audit")}
case "odoo":
settings = s.buildOdooSettings()
case "freecad":
settings = s.buildFreecadSettings()
case "jobs":
settings = s.buildJobsSettings()
case "dag":
settings = map[string]any{"enabled": s.modules.IsEnabled("dag")}
default:
writeError(w, http.StatusNotFound, "not_found", "Unknown module: "+module)
return
}
writeJSON(w, http.StatusOK, settings)
}
// HandleUpdateModuleSettings handles module toggle and config overrides.
func (s *Server) HandleUpdateModuleSettings(w http.ResponseWriter, r *http.Request) {
module := chi.URLParam(r, "module")
// Validate module exists
if s.modules.Get(module) == nil {
writeError(w, http.StatusNotFound, "not_found", "Unknown module: "+module)
return
}
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "invalid_json", err.Error())
return
}
user := auth.UserFromContext(r.Context())
username := "system"
if user != nil {
username = user.Username
}
var updated []string
restartRequired := false
// Handle module toggle
if enabledVal, ok := body["enabled"]; ok {
enabled, ok := enabledVal.(bool)
if !ok {
writeError(w, http.StatusBadRequest, "invalid_value", "'enabled' must be a boolean")
return
}
if err := s.modules.SetEnabled(module, enabled); err != nil {
writeError(w, http.StatusBadRequest, "toggle_failed", err.Error())
return
}
if err := s.settings.SetModuleState(r.Context(), module, enabled, username); err != nil {
s.logger.Error().Err(err).Str("module", module).Msg("failed to persist module state")
writeError(w, http.StatusInternalServerError, "persist_failed", "Failed to save module state")
return
}
updated = append(updated, module+".enabled")
}
// Handle config overrides (future use — persisted but not hot-reloaded)
for key, value := range body {
if key == "enabled" {
continue
}
fullKey := module + "." + key
if err := s.settings.SetOverride(r.Context(), fullKey, value, username); err != nil {
s.logger.Error().Err(err).Str("key", fullKey).Msg("failed to persist setting override")
writeError(w, http.StatusInternalServerError, "persist_failed", "Failed to save setting: "+key)
return
}
updated = append(updated, fullKey)
// These namespaces require a restart to take effect
if strings.HasPrefix(fullKey, "database.") ||
strings.HasPrefix(fullKey, "storage.") ||
strings.HasPrefix(fullKey, "server.") ||
strings.HasPrefix(fullKey, "schemas.") {
restartRequired = true
}
}
writeJSON(w, http.StatusOK, map[string]any{
"updated": updated,
"restart_required": restartRequired,
})
// Publish SSE event
s.broker.Publish("settings.changed", mustMarshal(map[string]any{
"module": module,
"changed_keys": updated,
"updated_by": username,
}))
}
// HandleTestModuleConnectivity tests external connectivity for a module.
func (s *Server) HandleTestModuleConnectivity(w http.ResponseWriter, r *http.Request) {
module := chi.URLParam(r, "module")
start := time.Now()
var success bool
var message string
switch module {
case "database":
if err := s.db.Pool().Ping(r.Context()); err != nil {
success = false
message = "Database ping failed: " + err.Error()
} else {
success = true
message = "Database connection OK"
}
case "storage":
if s.storage == nil {
success = false
message = "Storage not configured"
} else if err := s.storage.Ping(r.Context()); err != nil {
success = false
message = "Storage ping failed: " + err.Error()
} else {
success = true
message = "Storage connection OK"
}
case "auth", "odoo":
success = false
message = "Connectivity test not implemented for " + module
default:
writeError(w, http.StatusBadRequest, "not_testable", "No connectivity test available for module: "+module)
return
}
latency := time.Since(start).Milliseconds()
writeJSON(w, http.StatusOK, map[string]any{
"success": success,
"message": message,
"latency_ms": latency,
})
}
// --- build helpers (read config, redact secrets) ---
func redact(s string) string {
if s == "" {
return ""
}
return "****"
}
func (s *Server) buildCoreSettings() map[string]any {
return map[string]any{
"enabled": true,
"host": s.cfg.Server.Host,
"port": s.cfg.Server.Port,
"base_url": s.cfg.Server.BaseURL,
"readonly": s.cfg.Server.ReadOnly,
}
}
func (s *Server) buildSchemasSettings() map[string]any {
return map[string]any{
"enabled": true,
"directory": s.cfg.Schemas.Directory,
"default": s.cfg.Schemas.Default,
"count": len(s.schemas),
}
}
func (s *Server) buildStorageSettings(ctx context.Context) map[string]any {
result := map[string]any{
"enabled": true,
"backend": "filesystem",
"root_dir": s.cfg.Storage.Filesystem.RootDir,
}
if s.storage != nil {
if err := s.storage.Ping(ctx); err != nil {
result["status"] = "unavailable"
} else {
result["status"] = "ok"
}
} else {
result["status"] = "not_configured"
}
return result
}
func (s *Server) buildDatabaseSettings(ctx context.Context) map[string]any {
result := map[string]any{
"enabled": true,
"host": s.cfg.Database.Host,
"port": s.cfg.Database.Port,
"name": s.cfg.Database.Name,
"user": s.cfg.Database.User,
"password": redact(s.cfg.Database.Password),
"sslmode": s.cfg.Database.SSLMode,
"max_connections": s.cfg.Database.MaxConnections,
}
if err := s.db.Pool().Ping(ctx); err != nil {
result["status"] = "unavailable"
} else {
result["status"] = "ok"
}
return result
}
func (s *Server) buildAuthSettings() map[string]any {
return map[string]any{
"enabled": s.modules.IsEnabled("auth"),
"session_secret": redact(s.cfg.Auth.SessionSecret),
"local": map[string]any{
"enabled": s.cfg.Auth.Local.Enabled,
"default_admin_username": s.cfg.Auth.Local.DefaultAdminUsername,
"default_admin_password": redact(s.cfg.Auth.Local.DefaultAdminPassword),
},
"ldap": map[string]any{
"enabled": s.cfg.Auth.LDAP.Enabled,
"url": s.cfg.Auth.LDAP.URL,
"base_dn": s.cfg.Auth.LDAP.BaseDN,
"bind_dn": s.cfg.Auth.LDAP.BindDN,
"bind_password": redact(s.cfg.Auth.LDAP.BindPassword),
},
"oidc": map[string]any{
"enabled": s.cfg.Auth.OIDC.Enabled,
"issuer_url": s.cfg.Auth.OIDC.IssuerURL,
"client_id": s.cfg.Auth.OIDC.ClientID,
"client_secret": redact(s.cfg.Auth.OIDC.ClientSecret),
"redirect_url": s.cfg.Auth.OIDC.RedirectURL,
},
}
}
func (s *Server) buildOdooSettings() map[string]any {
return map[string]any{
"enabled": s.modules.IsEnabled("odoo"),
"url": s.cfg.Odoo.URL,
"database": s.cfg.Odoo.Database,
"username": s.cfg.Odoo.Username,
"api_key": redact(s.cfg.Odoo.APIKey),
}
}
func (s *Server) buildFreecadSettings() map[string]any {
return map[string]any{
"enabled": s.modules.IsEnabled("freecad"),
"uri_scheme": s.cfg.FreeCAD.URIScheme,
"executable": s.cfg.FreeCAD.Executable,
}
}
func (s *Server) buildJobsSettings() map[string]any {
return map[string]any{
"enabled": s.modules.IsEnabled("jobs"),
"directory": s.cfg.Jobs.Directory,
"runner_timeout": s.cfg.Jobs.RunnerTimeout,
"job_timeout_check": s.cfg.Jobs.JobTimeoutCheck,
"default_priority": s.cfg.Jobs.DefaultPriority,
"definitions_count": len(s.jobDefs),
}
}

View File

@@ -0,0 +1,286 @@
package api
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/config"
"github.com/kindredsystems/silo/internal/db"
"github.com/kindredsystems/silo/internal/modules"
"github.com/kindredsystems/silo/internal/schema"
"github.com/kindredsystems/silo/internal/testutil"
"github.com/rs/zerolog"
)
func newSettingsTestServer(t *testing.T) *Server {
t.Helper()
pool := testutil.MustConnectTestPool(t)
database := db.NewFromPool(pool)
broker := NewBroker(zerolog.Nop())
state := NewServerState(zerolog.Nop(), nil, broker)
cfg := &config.Config{
Server: config.ServerConfig{Host: "0.0.0.0", Port: 8080},
Database: config.DatabaseConfig{
Host: "localhost", Port: 5432, Name: "silo_test",
User: "silo", Password: "secret", SSLMode: "disable",
MaxConnections: 10,
},
Storage: config.StorageConfig{
Backend: "filesystem",
Filesystem: config.FilesystemConfig{RootDir: "/tmp/silo-test"},
},
Schemas: config.SchemasConfig{Directory: "/etc/silo/schemas", Default: "kindred-rd"},
Auth: config.AuthConfig{
SessionSecret: "supersecret",
Local: config.LocalAuth{Enabled: true, DefaultAdminUsername: "admin", DefaultAdminPassword: "changeme"},
LDAP: config.LDAPAuth{Enabled: false, BindPassword: "ldapsecret"},
OIDC: config.OIDCAuth{Enabled: false, ClientSecret: "oidcsecret"},
},
FreeCAD: config.FreeCADConfig{URIScheme: "silo"},
Odoo: config.OdooConfig{URL: "https://odoo.example.com", APIKey: "odoo-api-key"},
Jobs: config.JobsConfig{Directory: "/etc/silo/jobdefs", RunnerTimeout: 90, JobTimeoutCheck: 30, DefaultPriority: 100},
}
return NewServer(
zerolog.Nop(),
database,
map[string]*schema.Schema{"test": {Name: "test"}},
cfg.Schemas.Directory,
nil, // storage
nil, // authService
nil, // sessionManager
nil, // oidcBackend
nil, // authConfig
broker,
state,
nil, // jobDefs
"", // jobDefsDir
modules.NewRegistry(), // modules
cfg,
nil, // workflows
)
}
func newSettingsRouter(s *Server) http.Handler {
r := chi.NewRouter()
r.Route("/api/admin/settings", func(r chi.Router) {
r.Get("/", s.HandleGetAllSettings)
r.Get("/{module}", s.HandleGetModuleSettings)
r.Put("/{module}", s.HandleUpdateModuleSettings)
r.Post("/{module}/test", s.HandleTestModuleConnectivity)
})
return r
}
func adminSettingsRequest(r *http.Request) *http.Request {
u := &auth.User{
ID: "admin-id",
Username: "testadmin",
Role: auth.RoleAdmin,
}
return r.WithContext(auth.ContextWithUser(r.Context(), u))
}
func viewerSettingsRequest(r *http.Request) *http.Request {
u := &auth.User{
ID: "viewer-id",
Username: "testviewer",
Role: auth.RoleViewer,
}
return r.WithContext(auth.ContextWithUser(r.Context(), u))
}
func TestGetAllSettings(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
req := adminSettingsRequest(httptest.NewRequest("GET", "/api/admin/settings", nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding: %v", err)
}
// Verify all module keys present
expectedModules := []string{"core", "schemas", "storage", "database", "auth", "projects", "audit", "odoo", "freecad", "jobs", "dag"}
for _, mod := range expectedModules {
if _, ok := resp[mod]; !ok {
t.Errorf("missing module key: %s", mod)
}
}
// Verify secrets are redacted
dbSettings, _ := resp["database"].(map[string]any)
if dbSettings["password"] != "****" {
t.Errorf("database password not redacted: got %v", dbSettings["password"])
}
authSettings, _ := resp["auth"].(map[string]any)
if authSettings["session_secret"] != "****" {
t.Errorf("session_secret not redacted: got %v", authSettings["session_secret"])
}
ldap, _ := authSettings["ldap"].(map[string]any)
if ldap["bind_password"] != "****" {
t.Errorf("ldap bind_password not redacted: got %v", ldap["bind_password"])
}
oidc, _ := authSettings["oidc"].(map[string]any)
if oidc["client_secret"] != "****" {
t.Errorf("oidc client_secret not redacted: got %v", oidc["client_secret"])
}
odoo, _ := resp["odoo"].(map[string]any)
if odoo["api_key"] != "****" {
t.Errorf("odoo api_key not redacted: got %v", odoo["api_key"])
}
}
func TestGetModuleSettings(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
req := adminSettingsRequest(httptest.NewRequest("GET", "/api/admin/settings/jobs", nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding: %v", err)
}
if resp["directory"] != "/etc/silo/jobdefs" {
t.Errorf("jobs directory: got %v, want /etc/silo/jobdefs", resp["directory"])
}
if resp["runner_timeout"] != float64(90) {
t.Errorf("runner_timeout: got %v, want 90", resp["runner_timeout"])
}
}
func TestGetModuleSettings_Unknown(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
req := adminSettingsRequest(httptest.NewRequest("GET", "/api/admin/settings/nonexistent", nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusNotFound {
t.Errorf("status: got %d, want %d", w.Code, http.StatusNotFound)
}
}
func TestToggleModule(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
// Projects is enabled by default; disable it
body := `{"enabled": false}`
req := adminSettingsRequest(httptest.NewRequest("PUT", "/api/admin/settings/projects", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding: %v", err)
}
updated, _ := resp["updated"].([]any)
if len(updated) != 1 || updated[0] != "projects.enabled" {
t.Errorf("updated: got %v, want [projects.enabled]", updated)
}
// Verify registry state
if s.modules.IsEnabled("projects") {
t.Error("projects should be disabled after toggle")
}
}
func TestToggleModule_DependencyError(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
// DAG depends on Jobs. Jobs is disabled by default.
// Enabling DAG without Jobs should fail.
body := `{"enabled": true}`
req := adminSettingsRequest(httptest.NewRequest("PUT", "/api/admin/settings/dag", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusBadRequest, w.Body.String())
}
}
func TestToggleRequiredModule(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
body := `{"enabled": false}`
req := adminSettingsRequest(httptest.NewRequest("PUT", "/api/admin/settings/core", strings.NewReader(body)))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusBadRequest, w.Body.String())
}
}
func TestTestConnectivity_Database(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
req := adminSettingsRequest(httptest.NewRequest("POST", "/api/admin/settings/database/test", nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("status: got %d, want %d; body: %s", w.Code, http.StatusOK, w.Body.String())
}
var resp map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decoding: %v", err)
}
if resp["success"] != true {
t.Errorf("expected success=true, got %v; message: %v", resp["success"], resp["message"])
}
if resp["latency_ms"] == nil {
t.Error("expected latency_ms in response")
}
}
func TestTestConnectivity_NotTestable(t *testing.T) {
s := newSettingsTestServer(t)
router := newSettingsRouter(s)
req := adminSettingsRequest(httptest.NewRequest("POST", "/api/admin/settings/core/test", nil))
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusBadRequest {
t.Errorf("status: got %d, want %d; body: %s", w.Code, http.StatusBadRequest, w.Body.String())
}
}

View File

@@ -0,0 +1,551 @@
package api
import (
"context"
"encoding/json"
"net/http"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
)
// SubmitSolveRequest is the JSON body for POST /api/solver/jobs.
type SubmitSolveRequest struct {
Solver string `json:"solver"`
Operation string `json:"operation"`
Context json.RawMessage `json:"context"`
Priority *int `json:"priority,omitempty"`
ItemPartNumber string `json:"item_part_number,omitempty"`
RevisionNumber *int `json:"revision_number,omitempty"`
}
// SolverJobResponse is the JSON response for solver job creation.
type SolverJobResponse struct {
JobID string `json:"job_id"`
Status string `json:"status"`
CreatedAt string `json:"created_at"`
}
// SolverResultResponse is the JSON response for cached solver results.
type SolverResultResponse struct {
ID string `json:"id"`
RevisionNumber int `json:"revision_number"`
JobID *string `json:"job_id,omitempty"`
Operation string `json:"operation"`
SolverName string `json:"solver_name"`
Status string `json:"status"`
DOF *int `json:"dof,omitempty"`
Diagnostics json.RawMessage `json:"diagnostics"`
Placements json.RawMessage `json:"placements"`
NumFrames int `json:"num_frames"`
SolveTimeMS *float64 `json:"solve_time_ms,omitempty"`
CreatedAt string `json:"created_at"`
}
// operationToDefinition maps solve operations to job definition names.
var operationToDefinition = map[string]string{
"solve": "assembly-solve",
"diagnose": "assembly-validate",
"kinematic": "assembly-kinematic",
}
// HandleSubmitSolverJob creates a solver job via the existing job queue.
// POST /api/solver/jobs
func (s *Server) HandleSubmitSolverJob(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
// Enforce max context size at the HTTP boundary.
maxBytes := int64(s.cfg.Solver.MaxContextSizeMB) * 1024 * 1024
r.Body = http.MaxBytesReader(w, r.Body, maxBytes)
var req SubmitSolveRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
if err.Error() == "http: request body too large" {
writeError(w, http.StatusRequestEntityTooLarge, "context_too_large",
"SolveContext exceeds maximum size")
return
}
writeError(w, http.StatusBadRequest, "invalid_body", "Invalid JSON body")
return
}
// Validate operation.
if req.Operation == "" {
req.Operation = "solve"
}
defName, ok := operationToDefinition[req.Operation]
if !ok {
writeError(w, http.StatusBadRequest, "invalid_operation",
"Operation must be 'solve', 'diagnose', or 'kinematic'")
return
}
// Context is required.
if len(req.Context) == 0 {
writeError(w, http.StatusBadRequest, "missing_context", "SolveContext is required")
return
}
// Look up job definition.
def, err := s.jobs.GetDefinition(ctx, defName)
if err != nil {
s.logger.Error().Err(err).Str("definition", defName).Msg("failed to look up solver job definition")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to look up job definition")
return
}
if def == nil {
writeError(w, http.StatusNotFound, "definition_not_found",
"Solver job definition '"+defName+"' not found; ensure job definition YAML is loaded")
return
}
// Resolve item_part_number → item_id (optional).
var itemID *string
if req.ItemPartNumber != "" {
item, err := s.items.GetByPartNumber(ctx, req.ItemPartNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item for solver job")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to resolve item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "item_not_found",
"Item '"+req.ItemPartNumber+"' not found")
return
}
itemID = &item.ID
}
// Pack solver-specific data into scope_metadata.
scopeMeta := map[string]any{
"solver": req.Solver,
"operation": req.Operation,
"context": req.Context,
}
if req.RevisionNumber != nil {
scopeMeta["revision_number"] = *req.RevisionNumber
}
if req.ItemPartNumber != "" {
scopeMeta["item_part_number"] = req.ItemPartNumber
}
priority := def.Priority
if req.Priority != nil {
priority = *req.Priority
}
username := ""
if user := auth.UserFromContext(ctx); user != nil {
username = user.Username
}
job := &db.Job{
JobDefinitionID: &def.ID,
DefinitionName: def.Name,
Priority: priority,
ItemID: itemID,
ScopeMetadata: scopeMeta,
RunnerTags: def.RunnerTags,
TimeoutSeconds: def.TimeoutSeconds,
MaxRetries: def.MaxRetries,
CreatedBy: &username,
}
// Use solver default timeout if the definition has none.
if job.TimeoutSeconds == 0 {
job.TimeoutSeconds = s.cfg.Solver.DefaultTimeout
}
if err := s.jobs.CreateJob(ctx, job); err != nil {
s.logger.Error().Err(err).Msg("failed to create solver job")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to create solver job")
return
}
s.broker.Publish("job.created", mustMarshal(map[string]any{
"job_id": job.ID,
"definition_name": job.DefinitionName,
"trigger": "manual",
"item_id": job.ItemID,
}))
writeJSON(w, http.StatusCreated, SolverJobResponse{
JobID: job.ID,
Status: job.Status,
CreatedAt: job.CreatedAt.UTC().Format("2006-01-02T15:04:05Z"),
})
}
// HandleGetSolverJob returns a single solver job.
// GET /api/solver/jobs/{jobID}
func (s *Server) HandleGetSolverJob(w http.ResponseWriter, r *http.Request) {
jobID := chi.URLParam(r, "jobID")
job, err := s.jobs.GetJob(r.Context(), jobID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get solver job")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get job")
return
}
if job == nil {
writeError(w, http.StatusNotFound, "not_found", "Job not found")
return
}
writeJSON(w, http.StatusOK, job)
}
// HandleListSolverJobs lists solver jobs with optional filters.
// GET /api/solver/jobs
func (s *Server) HandleListSolverJobs(w http.ResponseWriter, r *http.Request) {
status := r.URL.Query().Get("status")
itemPartNumber := r.URL.Query().Get("item")
operation := r.URL.Query().Get("operation")
limit := 20
if v := r.URL.Query().Get("limit"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 && n <= 100 {
limit = n
}
}
offset := 0
if v := r.URL.Query().Get("offset"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n >= 0 {
offset = n
}
}
// Resolve item part number to ID if provided.
var itemID string
if itemPartNumber != "" {
item, err := s.items.GetByPartNumber(r.Context(), itemPartNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to resolve item for solver job list")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to resolve item")
return
}
if item == nil {
writeJSON(w, http.StatusOK, map[string]any{
"jobs": []*db.Job{},
"total": 0,
"limit": limit,
"offset": offset,
})
return
}
itemID = item.ID
}
jobs, err := s.jobs.ListSolverJobs(r.Context(), status, itemID, operation, limit, offset)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list solver jobs")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list solver jobs")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"jobs": jobs,
"limit": limit,
"offset": offset,
})
}
// HandleCancelSolverJob cancels a solver job.
// POST /api/solver/jobs/{jobID}/cancel
func (s *Server) HandleCancelSolverJob(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
jobID := chi.URLParam(r, "jobID")
user := auth.UserFromContext(ctx)
cancelledBy := "system"
if user != nil {
cancelledBy = user.Username
}
if err := s.jobs.CancelJob(ctx, jobID, cancelledBy); err != nil {
writeError(w, http.StatusBadRequest, "cancel_failed", err.Error())
return
}
s.broker.Publish("job.cancelled", mustMarshal(map[string]any{
"job_id": jobID,
"cancelled_by": cancelledBy,
}))
writeJSON(w, http.StatusOK, map[string]string{
"job_id": jobID,
"status": "cancelled",
})
}
// HandleGetSolverRegistry returns available solvers from online runners.
// GET /api/solver/solvers
func (s *Server) HandleGetSolverRegistry(w http.ResponseWriter, r *http.Request) {
runners, err := s.jobs.ListRunners(r.Context())
if err != nil {
s.logger.Error().Err(err).Msg("failed to list runners for solver registry")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list runners")
return
}
type solverInfo struct {
Name string `json:"name"`
DisplayName string `json:"display_name,omitempty"`
Deterministic bool `json:"deterministic,omitempty"`
SupportedJoints []string `json:"supported_joints,omitempty"`
RunnerCount int `json:"runner_count"`
}
solverMap := make(map[string]*solverInfo)
for _, runner := range runners {
if runner.Status != "online" {
continue
}
// Check runner has the solver tag.
hasSolverTag := false
for _, tag := range runner.Tags {
if tag == "solver" {
hasSolverTag = true
break
}
}
if !hasSolverTag {
continue
}
// Extract solver capabilities from runner metadata.
if runner.Metadata == nil {
continue
}
solvers, ok := runner.Metadata["solvers"]
if !ok {
continue
}
// solvers can be []any (array of solver objects or strings).
solverList, ok := solvers.([]any)
if !ok {
continue
}
for _, entry := range solverList {
switch v := entry.(type) {
case string:
// Simple string entry: just the solver name.
if _, exists := solverMap[v]; !exists {
solverMap[v] = &solverInfo{Name: v}
}
solverMap[v].RunnerCount++
case map[string]any:
// Rich entry with name, display_name, supported_joints, etc.
name, _ := v["name"].(string)
if name == "" {
continue
}
if _, exists := solverMap[name]; !exists {
info := &solverInfo{Name: name}
if dn, ok := v["display_name"].(string); ok {
info.DisplayName = dn
}
if det, ok := v["deterministic"].(bool); ok {
info.Deterministic = det
}
if joints, ok := v["supported_joints"].([]any); ok {
for _, j := range joints {
if js, ok := j.(string); ok {
info.SupportedJoints = append(info.SupportedJoints, js)
}
}
}
solverMap[name] = info
}
solverMap[name].RunnerCount++
}
}
}
solverList := make([]*solverInfo, 0, len(solverMap))
for _, info := range solverMap {
solverList = append(solverList, info)
}
writeJSON(w, http.StatusOK, map[string]any{
"solvers": solverList,
"default_solver": s.cfg.Solver.DefaultSolver,
})
}
// HandleGetSolverResults returns cached solver results for an item.
// GET /api/items/{partNumber}/solver/results
func (s *Server) HandleGetSolverResults(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
partNumber := chi.URLParam(r, "partNumber")
item, err := s.items.GetByPartNumber(ctx, partNumber)
if err != nil {
s.logger.Error().Err(err).Msg("failed to get item for solver results")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
return
}
if item == nil {
writeError(w, http.StatusNotFound, "not_found", "Item not found")
return
}
results, err := s.solverResults.GetByItem(ctx, item.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list solver results")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list solver results")
return
}
resp := make([]SolverResultResponse, len(results))
for i, r := range results {
diag := json.RawMessage(r.Diagnostics)
if diag == nil {
diag = json.RawMessage("[]")
}
place := json.RawMessage(r.Placements)
if place == nil {
place = json.RawMessage("[]")
}
resp[i] = SolverResultResponse{
ID: r.ID,
RevisionNumber: r.RevisionNumber,
JobID: r.JobID,
Operation: r.Operation,
SolverName: r.SolverName,
Status: r.Status,
DOF: r.DOF,
Diagnostics: diag,
Placements: place,
NumFrames: r.NumFrames,
SolveTimeMS: r.SolveTimeMS,
CreatedAt: r.CreatedAt.UTC().Format("2006-01-02T15:04:05Z"),
}
}
writeJSON(w, http.StatusOK, resp)
}
// maybeCacheSolverResult is called asynchronously after a job completes.
// It checks if the job is a solver job and upserts the result into solver_results.
func (s *Server) maybeCacheSolverResult(ctx context.Context, jobID string) {
job, err := s.jobs.GetJob(ctx, jobID)
if err != nil || job == nil {
s.logger.Warn().Err(err).Str("job_id", jobID).Msg("solver result cache: failed to get job")
return
}
if !strings.HasPrefix(job.DefinitionName, "assembly-") {
return
}
if !s.modules.IsEnabled("solver") {
return
}
if job.ItemID == nil {
return
}
// Extract fields from scope_metadata.
operation, _ := job.ScopeMetadata["operation"].(string)
if operation == "" {
operation = "solve"
}
solverName, _ := job.ScopeMetadata["solver"].(string)
var revisionNumber int
if rn, ok := job.ScopeMetadata["revision_number"].(float64); ok {
revisionNumber = int(rn)
}
// Extract fields from result.
if job.Result == nil {
return
}
status, _ := job.Result["status"].(string)
if status == "" {
// Try nested result object.
if inner, ok := job.Result["result"].(map[string]any); ok {
status, _ = inner["status"].(string)
}
}
if status == "" {
status = "Unknown"
}
// Solver name from result takes precedence.
if sn, ok := job.Result["solver_name"].(string); ok && sn != "" {
solverName = sn
}
if solverName == "" {
solverName = "unknown"
}
var dof *int
if d, ok := job.Result["dof"].(float64); ok {
v := int(d)
dof = &v
} else if inner, ok := job.Result["result"].(map[string]any); ok {
if d, ok := inner["dof"].(float64); ok {
v := int(d)
dof = &v
}
}
var solveTimeMS *float64
if t, ok := job.Result["solve_time_ms"].(float64); ok {
solveTimeMS = &t
}
// Marshal diagnostics and placements as raw JSONB.
var diagnostics, placements []byte
if d, ok := job.Result["diagnostics"]; ok {
diagnostics, _ = json.Marshal(d)
} else if inner, ok := job.Result["result"].(map[string]any); ok {
if d, ok := inner["diagnostics"]; ok {
diagnostics, _ = json.Marshal(d)
}
}
if p, ok := job.Result["placements"]; ok {
placements, _ = json.Marshal(p)
} else if inner, ok := job.Result["result"].(map[string]any); ok {
if p, ok := inner["placements"]; ok {
placements, _ = json.Marshal(p)
}
}
numFrames := 0
if nf, ok := job.Result["num_frames"].(float64); ok {
numFrames = int(nf)
} else if inner, ok := job.Result["result"].(map[string]any); ok {
if nf, ok := inner["num_frames"].(float64); ok {
numFrames = int(nf)
}
}
result := &db.SolverResult{
ItemID: *job.ItemID,
RevisionNumber: revisionNumber,
JobID: &job.ID,
Operation: operation,
SolverName: solverName,
Status: status,
DOF: dof,
Diagnostics: diagnostics,
Placements: placements,
NumFrames: numFrames,
SolveTimeMS: solveTimeMS,
}
if err := s.solverResults.Upsert(ctx, result); err != nil {
s.logger.Warn().Err(err).Str("job_id", jobID).Msg("solver result cache: failed to upsert")
} else {
s.logger.Info().Str("job_id", jobID).Str("operation", operation).Msg("cached solver result")
}
}

View File

@@ -5,6 +5,8 @@ import (
"net/http"
"strconv"
"time"
"github.com/kindredsystems/silo/internal/auth"
)
// HandleEvents serves the SSE event stream.
@@ -16,9 +18,12 @@ func (s *Server) HandleEvents(w http.ResponseWriter, r *http.Request) {
return
}
// Disable the write deadline for this long-lived connection.
// The server's WriteTimeout (15s) would otherwise kill it.
// Disable read and write deadlines for this long-lived connection.
// The server's ReadTimeout/WriteTimeout (15s) would otherwise kill it.
rc := http.NewResponseController(w)
if err := rc.SetReadDeadline(time.Time{}); err != nil {
s.logger.Warn().Err(err).Msg("failed to disable read deadline for SSE")
}
if err := rc.SetWriteDeadline(time.Time{}); err != nil {
s.logger.Warn().Err(err).Msg("failed to disable write deadline for SSE")
}
@@ -28,9 +33,19 @@ func (s *Server) HandleEvents(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Connection", "keep-alive")
w.Header().Set("X-Accel-Buffering", "no") // nginx: disable proxy buffering
client := s.broker.Subscribe()
userID := ""
if user := auth.UserFromContext(r.Context()); user != nil {
userID = user.ID
}
wsID := r.URL.Query().Get("workstation_id")
client := s.broker.Subscribe(userID, wsID)
defer s.broker.Unsubscribe(client)
if wsID != "" {
s.workstations.Touch(r.Context(), wsID)
}
// Replay missed events if Last-Event-ID is present.
if lastIDStr := r.Header.Get("Last-Event-ID"); lastIDStr != "" {
if lastID, err := strconv.ParseUint(lastIDStr, 10, 64); err == nil {

View File

@@ -0,0 +1,138 @@
package api
import (
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/kindredsystems/silo/internal/auth"
"github.com/kindredsystems/silo/internal/db"
)
// HandleRegisterWorkstation registers or re-registers a workstation for the current user.
func (s *Server) HandleRegisterWorkstation(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := auth.UserFromContext(ctx)
if user == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
var req struct {
Name string `json:"name"`
Hostname string `json:"hostname"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid_json", err.Error())
return
}
if req.Name == "" {
writeError(w, http.StatusBadRequest, "validation_error", "name is required")
return
}
ws := &db.Workstation{
Name: req.Name,
UserID: user.ID,
Hostname: req.Hostname,
}
if err := s.workstations.Upsert(ctx, ws); err != nil {
s.logger.Error().Err(err).Str("name", req.Name).Msg("failed to register workstation")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to register workstation")
return
}
s.broker.Publish("workstation.registered", mustMarshal(map[string]any{
"id": ws.ID,
"name": ws.Name,
"user_id": ws.UserID,
"hostname": ws.Hostname,
}))
writeJSON(w, http.StatusOK, map[string]any{
"id": ws.ID,
"name": ws.Name,
"hostname": ws.Hostname,
"last_seen": ws.LastSeen.UTC().Format("2006-01-02T15:04:05Z"),
"created_at": ws.CreatedAt.UTC().Format("2006-01-02T15:04:05Z"),
})
}
// HandleListWorkstations returns all workstations for the current user.
func (s *Server) HandleListWorkstations(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := auth.UserFromContext(ctx)
if user == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
workstations, err := s.workstations.ListByUser(ctx, user.ID)
if err != nil {
s.logger.Error().Err(err).Msg("failed to list workstations")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list workstations")
return
}
type wsResponse struct {
ID string `json:"id"`
Name string `json:"name"`
Hostname string `json:"hostname"`
LastSeen string `json:"last_seen"`
CreatedAt string `json:"created_at"`
}
out := make([]wsResponse, len(workstations))
for i, ws := range workstations {
out[i] = wsResponse{
ID: ws.ID,
Name: ws.Name,
Hostname: ws.Hostname,
LastSeen: ws.LastSeen.UTC().Format("2006-01-02T15:04:05Z"),
CreatedAt: ws.CreatedAt.UTC().Format("2006-01-02T15:04:05Z"),
}
}
writeJSON(w, http.StatusOK, out)
}
// HandleDeleteWorkstation removes a workstation owned by the current user (or any, for admins).
func (s *Server) HandleDeleteWorkstation(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
user := auth.UserFromContext(ctx)
if user == nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "Authentication required")
return
}
id := chi.URLParam(r, "id")
ws, err := s.workstations.GetByID(ctx, id)
if err != nil {
s.logger.Error().Err(err).Str("id", id).Msg("failed to get workstation")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get workstation")
return
}
if ws == nil {
writeError(w, http.StatusNotFound, "not_found", "Workstation not found")
return
}
if ws.UserID != user.ID && user.Role != auth.RoleAdmin {
writeError(w, http.StatusForbidden, "forbidden", "You can only delete your own workstations")
return
}
if err := s.workstations.Delete(ctx, id); err != nil {
s.logger.Error().Err(err).Str("id", id).Msg("failed to delete workstation")
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to delete workstation")
return
}
s.broker.Publish("workstation.removed", mustMarshal(map[string]any{
"id": ws.ID,
"name": ws.Name,
"user_id": ws.UserID,
}))
w.WriteHeader(http.StatusNoContent)
}

24
internal/auth/runner.go Normal file
View File

@@ -0,0 +1,24 @@
package auth
import "context"
const runnerContextKey contextKey = iota + 1
// RunnerIdentity represents an authenticated runner in the request context.
type RunnerIdentity struct {
ID string
Name string
Tags []string
}
// RunnerFromContext extracts the authenticated runner from the request context.
// Returns nil if no runner is present.
func RunnerFromContext(ctx context.Context) *RunnerIdentity {
r, _ := ctx.Value(runnerContextKey).(*RunnerIdentity)
return r
}
// ContextWithRunner returns a new context carrying the given runner identity.
func ContextWithRunner(ctx context.Context, r *RunnerIdentity) context.Context {
return context.WithValue(ctx, runnerContextKey, r)
}

View File

@@ -10,13 +10,37 @@ import (
// Config holds all application configuration.
type Config struct {
Server ServerConfig `yaml:"server"`
Database DatabaseConfig `yaml:"database"`
Storage StorageConfig `yaml:"storage"`
Schemas SchemasConfig `yaml:"schemas"`
FreeCAD FreeCADConfig `yaml:"freecad"`
Odoo OdooConfig `yaml:"odoo"`
Auth AuthConfig `yaml:"auth"`
Server ServerConfig `yaml:"server"`
Database DatabaseConfig `yaml:"database"`
Storage StorageConfig `yaml:"storage"`
Schemas SchemasConfig `yaml:"schemas"`
FreeCAD FreeCADConfig `yaml:"freecad"`
Odoo OdooConfig `yaml:"odoo"`
Auth AuthConfig `yaml:"auth"`
Jobs JobsConfig `yaml:"jobs"`
Workflows WorkflowsConfig `yaml:"workflows"`
Solver SolverConfig `yaml:"solver"`
Modules ModulesConfig `yaml:"modules"`
}
// ModulesConfig holds explicit enable/disable toggles for optional modules.
// A nil pointer means "use the module's default state".
type ModulesConfig struct {
Auth *ModuleToggle `yaml:"auth"`
Projects *ModuleToggle `yaml:"projects"`
Audit *ModuleToggle `yaml:"audit"`
Odoo *ModuleToggle `yaml:"odoo"`
FreeCAD *ModuleToggle `yaml:"freecad"`
Jobs *ModuleToggle `yaml:"jobs"`
DAG *ModuleToggle `yaml:"dag"`
Solver *ModuleToggle `yaml:"solver"`
Sessions *ModuleToggle `yaml:"sessions"`
}
// ModuleToggle holds an optional enabled flag. The pointer allows
// distinguishing "not set" (nil) from "explicitly false".
type ModuleToggle struct {
Enabled *bool `yaml:"enabled"`
}
// AuthConfig holds authentication and authorization settings.
@@ -89,14 +113,15 @@ type DatabaseConfig struct {
MaxConnections int `yaml:"max_connections"`
}
// StorageConfig holds MinIO connection settings.
// StorageConfig holds file storage settings.
type StorageConfig struct {
Endpoint string `yaml:"endpoint"`
AccessKey string `yaml:"access_key"`
SecretKey string `yaml:"secret_key"`
Bucket string `yaml:"bucket"`
UseSSL bool `yaml:"use_ssl"`
Region string `yaml:"region"`
Backend string `yaml:"backend"` // "filesystem"
Filesystem FilesystemConfig `yaml:"filesystem"`
}
// FilesystemConfig holds local filesystem storage settings.
type FilesystemConfig struct {
RootDir string `yaml:"root_dir"`
}
// SchemasConfig holds schema loading settings.
@@ -111,6 +136,27 @@ type FreeCADConfig struct {
Executable string `yaml:"executable"`
}
// JobsConfig holds worker/runner system settings.
type JobsConfig struct {
Directory string `yaml:"directory"` // default /etc/silo/jobdefs
RunnerTimeout int `yaml:"runner_timeout"` // seconds, default 90
JobTimeoutCheck int `yaml:"job_timeout_check"` // seconds, default 30
DefaultPriority int `yaml:"default_priority"` // default 100
}
// WorkflowsConfig holds approval workflow definition settings.
type WorkflowsConfig struct {
Directory string `yaml:"directory"` // default /etc/silo/workflows
}
// SolverConfig holds assembly solver service settings.
type SolverConfig struct {
DefaultSolver string `yaml:"default_solver"`
MaxContextSizeMB int `yaml:"max_context_size_mb"`
DefaultTimeout int `yaml:"default_timeout"`
AutoDiagnoseOnCommit bool `yaml:"auto_diagnose_on_commit"`
}
// OdooConfig holds Odoo ERP integration settings.
type OdooConfig struct {
Enabled bool `yaml:"enabled"`
@@ -148,15 +194,33 @@ func Load(path string) (*Config, error) {
if cfg.Database.MaxConnections == 0 {
cfg.Database.MaxConnections = 10
}
if cfg.Storage.Region == "" {
cfg.Storage.Region = "us-east-1"
}
if cfg.Schemas.Directory == "" {
cfg.Schemas.Directory = "/etc/silo/schemas"
}
if cfg.FreeCAD.URIScheme == "" {
cfg.FreeCAD.URIScheme = "silo"
}
if cfg.Jobs.Directory == "" {
cfg.Jobs.Directory = "/etc/silo/jobdefs"
}
if cfg.Jobs.RunnerTimeout == 0 {
cfg.Jobs.RunnerTimeout = 90
}
if cfg.Jobs.JobTimeoutCheck == 0 {
cfg.Jobs.JobTimeoutCheck = 30
}
if cfg.Jobs.DefaultPriority == 0 {
cfg.Jobs.DefaultPriority = 100
}
if cfg.Workflows.Directory == "" {
cfg.Workflows.Directory = "/etc/silo/workflows"
}
if cfg.Solver.MaxContextSizeMB == 0 {
cfg.Solver.MaxContextSizeMB = 10
}
if cfg.Solver.DefaultTimeout == 0 {
cfg.Solver.DefaultTimeout = 300
}
// Override with environment variables
if v := os.Getenv("SILO_DB_HOST"); v != "" {
@@ -171,14 +235,11 @@ func Load(path string) (*Config, error) {
if v := os.Getenv("SILO_DB_PASSWORD"); v != "" {
cfg.Database.Password = v
}
if v := os.Getenv("SILO_MINIO_ENDPOINT"); v != "" {
cfg.Storage.Endpoint = v
if v := os.Getenv("SILO_STORAGE_ROOT_DIR"); v != "" {
cfg.Storage.Filesystem.RootDir = v
}
if v := os.Getenv("SILO_MINIO_ACCESS_KEY"); v != "" {
cfg.Storage.AccessKey = v
}
if v := os.Getenv("SILO_MINIO_SECRET_KEY"); v != "" {
cfg.Storage.SecretKey = v
if v := os.Getenv("SILO_SOLVER_DEFAULT"); v != "" {
cfg.Solver.DefaultSolver = v
}
// Auth defaults

View File

@@ -31,7 +31,7 @@ func (r *ItemRepository) ListItemsWithProperties(ctx context.Context, opts Audit
query = `
SELECT DISTINCT i.id, i.part_number, i.schema_id, i.item_type, i.description,
i.created_at, i.updated_at, i.archived_at, i.current_revision,
i.sourcing_type, i.sourcing_link, i.long_description, i.standard_cost,
i.sourcing_type, i.long_description,
COALESCE(r.properties, '{}'::jsonb) as properties
FROM items i
LEFT JOIN revisions r ON r.item_id = i.id AND r.revision_number = i.current_revision
@@ -45,7 +45,7 @@ func (r *ItemRepository) ListItemsWithProperties(ctx context.Context, opts Audit
query = `
SELECT i.id, i.part_number, i.schema_id, i.item_type, i.description,
i.created_at, i.updated_at, i.archived_at, i.current_revision,
i.sourcing_type, i.sourcing_link, i.long_description, i.standard_cost,
i.sourcing_type, i.long_description,
COALESCE(r.properties, '{}'::jsonb) as properties
FROM items i
LEFT JOIN revisions r ON r.item_id = i.id AND r.revision_number = i.current_revision
@@ -85,7 +85,7 @@ func (r *ItemRepository) ListItemsWithProperties(ctx context.Context, opts Audit
err := rows.Scan(
&iwp.ID, &iwp.PartNumber, &iwp.SchemaID, &iwp.ItemType, &iwp.Description,
&iwp.CreatedAt, &iwp.UpdatedAt, &iwp.ArchivedAt, &iwp.CurrentRevision,
&iwp.SourcingType, &iwp.SourcingLink, &iwp.LongDescription, &iwp.StandardCost,
&iwp.SourcingType, &iwp.LongDescription,
&propsJSON,
)
if err != nil {
@@ -134,6 +134,43 @@ func (r *ItemRepository) BatchCheckBOM(ctx context.Context, itemIDs []string) (m
return result, nil
}
// FileStats holds aggregated file attachment statistics for an item.
type FileStats struct {
Count int
TotalSize int64
}
// BatchGetFileStats returns a map of item ID to file attachment statistics
// for the given item IDs. Items not in the map have no files.
func (r *ItemRepository) BatchGetFileStats(ctx context.Context, itemIDs []string) (map[string]FileStats, error) {
if len(itemIDs) == 0 {
return map[string]FileStats{}, nil
}
rows, err := r.db.pool.Query(ctx, `
SELECT item_id, COUNT(*), COALESCE(SUM(size), 0)
FROM item_files
WHERE item_id = ANY($1)
GROUP BY item_id
`, itemIDs)
if err != nil {
return nil, fmt.Errorf("batch getting file stats: %w", err)
}
defer rows.Close()
result := make(map[string]FileStats)
for rows.Next() {
var itemID string
var fs FileStats
if err := rows.Scan(&itemID, &fs.Count, &fs.TotalSize); err != nil {
return nil, fmt.Errorf("scanning file stats: %w", err)
}
result[itemID] = fs
}
return result, nil
}
// BatchGetProjectCodes returns a map of item ID to project code list for
// the given item IDs.
func (r *ItemRepository) BatchGetProjectCodes(ctx context.Context, itemIDs []string) (map[string][]string, error) {

520
internal/db/dag.go Normal file
View File

@@ -0,0 +1,520 @@
package db
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// DAGNode represents a feature-level node in the dependency graph.
type DAGNode struct {
ID string
ItemID string
RevisionNumber int
NodeKey string
NodeType string
PropertiesHash *string
ValidationState string
ValidationMsg *string
Metadata map[string]any
CreatedAt time.Time
UpdatedAt time.Time
}
// DAGEdge represents a dependency between two nodes.
type DAGEdge struct {
ID string
SourceNodeID string
TargetNodeID string
EdgeType string
Metadata map[string]any
}
// DAGCrossEdge represents a dependency between nodes in different items.
type DAGCrossEdge struct {
ID string
SourceNodeID string
TargetNodeID string
RelationshipID *string
EdgeType string
Metadata map[string]any
}
// DAGRepository provides dependency graph database operations.
type DAGRepository struct {
db *DB
}
// NewDAGRepository creates a new DAG repository.
func NewDAGRepository(db *DB) *DAGRepository {
return &DAGRepository{db: db}
}
// GetNodes returns all DAG nodes for an item at a specific revision.
func (r *DAGRepository) GetNodes(ctx context.Context, itemID string, revisionNumber int) ([]*DAGNode, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, item_id, revision_number, node_key, node_type,
properties_hash, validation_state, validation_msg,
metadata, created_at, updated_at
FROM dag_nodes
WHERE item_id = $1 AND revision_number = $2
ORDER BY node_key
`, itemID, revisionNumber)
if err != nil {
return nil, fmt.Errorf("querying DAG nodes: %w", err)
}
defer rows.Close()
return scanDAGNodes(rows)
}
// GetNodeByKey returns a single DAG node by item, revision, and key.
func (r *DAGRepository) GetNodeByKey(ctx context.Context, itemID string, revisionNumber int, nodeKey string) (*DAGNode, error) {
n := &DAGNode{}
var metadataJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, item_id, revision_number, node_key, node_type,
properties_hash, validation_state, validation_msg,
metadata, created_at, updated_at
FROM dag_nodes
WHERE item_id = $1 AND revision_number = $2 AND node_key = $3
`, itemID, revisionNumber, nodeKey).Scan(
&n.ID, &n.ItemID, &n.RevisionNumber, &n.NodeKey, &n.NodeType,
&n.PropertiesHash, &n.ValidationState, &n.ValidationMsg,
&metadataJSON, &n.CreatedAt, &n.UpdatedAt,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("querying DAG node: %w", err)
}
if metadataJSON != nil {
if err := json.Unmarshal(metadataJSON, &n.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling node metadata: %w", err)
}
}
return n, nil
}
// GetNodeByID returns a single DAG node by its ID.
func (r *DAGRepository) GetNodeByID(ctx context.Context, nodeID string) (*DAGNode, error) {
n := &DAGNode{}
var metadataJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, item_id, revision_number, node_key, node_type,
properties_hash, validation_state, validation_msg,
metadata, created_at, updated_at
FROM dag_nodes
WHERE id = $1
`, nodeID).Scan(
&n.ID, &n.ItemID, &n.RevisionNumber, &n.NodeKey, &n.NodeType,
&n.PropertiesHash, &n.ValidationState, &n.ValidationMsg,
&metadataJSON, &n.CreatedAt, &n.UpdatedAt,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("querying DAG node by ID: %w", err)
}
if metadataJSON != nil {
if err := json.Unmarshal(metadataJSON, &n.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling node metadata: %w", err)
}
}
return n, nil
}
// UpsertNode inserts or updates a single DAG node.
func (r *DAGRepository) UpsertNode(ctx context.Context, n *DAGNode) error {
metadataJSON, err := json.Marshal(n.Metadata)
if err != nil {
return fmt.Errorf("marshaling metadata: %w", err)
}
err = r.db.pool.QueryRow(ctx, `
INSERT INTO dag_nodes (item_id, revision_number, node_key, node_type,
properties_hash, validation_state, validation_msg, metadata)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT (item_id, revision_number, node_key)
DO UPDATE SET
node_type = EXCLUDED.node_type,
properties_hash = EXCLUDED.properties_hash,
validation_state = EXCLUDED.validation_state,
validation_msg = EXCLUDED.validation_msg,
metadata = EXCLUDED.metadata,
updated_at = now()
RETURNING id, created_at, updated_at
`, n.ItemID, n.RevisionNumber, n.NodeKey, n.NodeType,
n.PropertiesHash, n.ValidationState, n.ValidationMsg, metadataJSON,
).Scan(&n.ID, &n.CreatedAt, &n.UpdatedAt)
if err != nil {
return fmt.Errorf("upserting DAG node: %w", err)
}
return nil
}
// GetEdges returns all edges for nodes belonging to an item at a specific revision.
func (r *DAGRepository) GetEdges(ctx context.Context, itemID string, revisionNumber int) ([]*DAGEdge, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT e.id, e.source_node_id, e.target_node_id, e.edge_type, e.metadata
FROM dag_edges e
JOIN dag_nodes src ON src.id = e.source_node_id
WHERE src.item_id = $1 AND src.revision_number = $2
ORDER BY e.source_node_id, e.target_node_id
`, itemID, revisionNumber)
if err != nil {
return nil, fmt.Errorf("querying DAG edges: %w", err)
}
defer rows.Close()
var edges []*DAGEdge
for rows.Next() {
e := &DAGEdge{}
var metadataJSON []byte
if err := rows.Scan(&e.ID, &e.SourceNodeID, &e.TargetNodeID, &e.EdgeType, &metadataJSON); err != nil {
return nil, fmt.Errorf("scanning DAG edge: %w", err)
}
if metadataJSON != nil {
if err := json.Unmarshal(metadataJSON, &e.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling edge metadata: %w", err)
}
}
edges = append(edges, e)
}
return edges, rows.Err()
}
// CreateEdge inserts a new edge between two nodes.
func (r *DAGRepository) CreateEdge(ctx context.Context, e *DAGEdge) error {
if e.EdgeType == "" {
e.EdgeType = "depends_on"
}
metadataJSON, err := json.Marshal(e.Metadata)
if err != nil {
return fmt.Errorf("marshaling edge metadata: %w", err)
}
err = r.db.pool.QueryRow(ctx, `
INSERT INTO dag_edges (source_node_id, target_node_id, edge_type, metadata)
VALUES ($1, $2, $3, $4)
ON CONFLICT (source_node_id, target_node_id, edge_type) DO NOTHING
RETURNING id
`, e.SourceNodeID, e.TargetNodeID, e.EdgeType, metadataJSON).Scan(&e.ID)
if err == pgx.ErrNoRows {
// Edge already exists, not an error
return nil
}
if err != nil {
return fmt.Errorf("creating DAG edge: %w", err)
}
return nil
}
// DeleteEdgesForItem removes all edges for nodes belonging to an item/revision.
func (r *DAGRepository) DeleteEdgesForItem(ctx context.Context, itemID string, revisionNumber int) error {
_, err := r.db.pool.Exec(ctx, `
DELETE FROM dag_edges
WHERE source_node_id IN (
SELECT id FROM dag_nodes WHERE item_id = $1 AND revision_number = $2
)
`, itemID, revisionNumber)
if err != nil {
return fmt.Errorf("deleting edges for item: %w", err)
}
return nil
}
// GetForwardCone returns all downstream dependent nodes reachable from the
// given node via edges. This is the key query for interference detection.
func (r *DAGRepository) GetForwardCone(ctx context.Context, nodeID string) ([]*DAGNode, error) {
rows, err := r.db.pool.Query(ctx, `
WITH RECURSIVE forward_cone AS (
SELECT target_node_id AS node_id
FROM dag_edges
WHERE source_node_id = $1
UNION
SELECT e.target_node_id
FROM dag_edges e
JOIN forward_cone fc ON fc.node_id = e.source_node_id
)
SELECT n.id, n.item_id, n.revision_number, n.node_key, n.node_type,
n.properties_hash, n.validation_state, n.validation_msg,
n.metadata, n.created_at, n.updated_at
FROM dag_nodes n
JOIN forward_cone fc ON n.id = fc.node_id
ORDER BY n.node_key
`, nodeID)
if err != nil {
return nil, fmt.Errorf("querying forward cone: %w", err)
}
defer rows.Close()
return scanDAGNodes(rows)
}
// GetBackwardCone returns all upstream dependency nodes that the given
// node depends on.
func (r *DAGRepository) GetBackwardCone(ctx context.Context, nodeID string) ([]*DAGNode, error) {
rows, err := r.db.pool.Query(ctx, `
WITH RECURSIVE backward_cone AS (
SELECT source_node_id AS node_id
FROM dag_edges
WHERE target_node_id = $1
UNION
SELECT e.source_node_id
FROM dag_edges e
JOIN backward_cone bc ON bc.node_id = e.target_node_id
)
SELECT n.id, n.item_id, n.revision_number, n.node_key, n.node_type,
n.properties_hash, n.validation_state, n.validation_msg,
n.metadata, n.created_at, n.updated_at
FROM dag_nodes n
JOIN backward_cone bc ON n.id = bc.node_id
ORDER BY n.node_key
`, nodeID)
if err != nil {
return nil, fmt.Errorf("querying backward cone: %w", err)
}
defer rows.Close()
return scanDAGNodes(rows)
}
// GetDirtySubgraph returns all non-clean nodes for an item.
func (r *DAGRepository) GetDirtySubgraph(ctx context.Context, itemID string) ([]*DAGNode, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, item_id, revision_number, node_key, node_type,
properties_hash, validation_state, validation_msg,
metadata, created_at, updated_at
FROM dag_nodes
WHERE item_id = $1 AND validation_state != 'clean'
ORDER BY node_key
`, itemID)
if err != nil {
return nil, fmt.Errorf("querying dirty subgraph: %w", err)
}
defer rows.Close()
return scanDAGNodes(rows)
}
// MarkDirty marks a node and all its downstream dependents as dirty.
func (r *DAGRepository) MarkDirty(ctx context.Context, nodeID string) (int64, error) {
result, err := r.db.pool.Exec(ctx, `
WITH RECURSIVE forward_cone AS (
SELECT $1::uuid AS node_id
UNION
SELECT e.target_node_id
FROM dag_edges e
JOIN forward_cone fc ON fc.node_id = e.source_node_id
)
UPDATE dag_nodes SET validation_state = 'dirty', updated_at = now()
WHERE id IN (SELECT node_id FROM forward_cone)
AND validation_state = 'clean'
`, nodeID)
if err != nil {
return 0, fmt.Errorf("marking dirty: %w", err)
}
return result.RowsAffected(), nil
}
// MarkValidating sets a node's state to 'validating'.
func (r *DAGRepository) MarkValidating(ctx context.Context, nodeID string) error {
_, err := r.db.pool.Exec(ctx, `
UPDATE dag_nodes SET validation_state = 'validating', updated_at = now()
WHERE id = $1
`, nodeID)
if err != nil {
return fmt.Errorf("marking validating: %w", err)
}
return nil
}
// MarkClean sets a node's state to 'clean' and updates its properties hash.
func (r *DAGRepository) MarkClean(ctx context.Context, nodeID string, propertiesHash string) error {
_, err := r.db.pool.Exec(ctx, `
UPDATE dag_nodes
SET validation_state = 'clean',
properties_hash = $2,
validation_msg = NULL,
updated_at = now()
WHERE id = $1
`, nodeID, propertiesHash)
if err != nil {
return fmt.Errorf("marking clean: %w", err)
}
return nil
}
// MarkFailed sets a node's state to 'failed' with an error message.
func (r *DAGRepository) MarkFailed(ctx context.Context, nodeID string, message string) error {
_, err := r.db.pool.Exec(ctx, `
UPDATE dag_nodes
SET validation_state = 'failed',
validation_msg = $2,
updated_at = now()
WHERE id = $1
`, nodeID, message)
if err != nil {
return fmt.Errorf("marking failed: %w", err)
}
return nil
}
// HasCycle checks whether adding an edge from sourceID to targetID would
// create a cycle. It walks upward from sourceID to see if targetID is
// already an ancestor.
func (r *DAGRepository) HasCycle(ctx context.Context, sourceID, targetID string) (bool, error) {
if sourceID == targetID {
return true, nil
}
var hasCycle bool
err := r.db.pool.QueryRow(ctx, `
WITH RECURSIVE ancestors AS (
SELECT source_node_id AS node_id
FROM dag_edges
WHERE target_node_id = $1
UNION
SELECT e.source_node_id
FROM dag_edges e
JOIN ancestors a ON a.node_id = e.target_node_id
)
SELECT EXISTS (
SELECT 1 FROM ancestors WHERE node_id = $2
)
`, sourceID, targetID).Scan(&hasCycle)
if err != nil {
return false, fmt.Errorf("checking for cycle: %w", err)
}
return hasCycle, nil
}
// SyncFeatureTree replaces the entire feature DAG for an item/revision
// within a single transaction. It upserts nodes, replaces edges, and
// marks changed nodes dirty.
func (r *DAGRepository) SyncFeatureTree(ctx context.Context, itemID string, revisionNumber int, nodes []DAGNode, edges []DAGEdge) error {
tx, err := r.db.pool.Begin(ctx)
if err != nil {
return fmt.Errorf("beginning transaction: %w", err)
}
defer tx.Rollback(ctx)
// Upsert all nodes
for i := range nodes {
n := &nodes[i]
n.ItemID = itemID
n.RevisionNumber = revisionNumber
if n.ValidationState == "" {
n.ValidationState = "clean"
}
metadataJSON, err := json.Marshal(n.Metadata)
if err != nil {
return fmt.Errorf("marshaling node metadata: %w", err)
}
err = tx.QueryRow(ctx, `
INSERT INTO dag_nodes (item_id, revision_number, node_key, node_type,
properties_hash, validation_state, validation_msg, metadata)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT (item_id, revision_number, node_key)
DO UPDATE SET
node_type = EXCLUDED.node_type,
properties_hash = EXCLUDED.properties_hash,
metadata = EXCLUDED.metadata,
updated_at = now()
RETURNING id, created_at, updated_at
`, n.ItemID, n.RevisionNumber, n.NodeKey, n.NodeType,
n.PropertiesHash, n.ValidationState, n.ValidationMsg, metadataJSON,
).Scan(&n.ID, &n.CreatedAt, &n.UpdatedAt)
if err != nil {
return fmt.Errorf("upserting node %s: %w", n.NodeKey, err)
}
}
// Build key→ID map for edge resolution
keyToID := make(map[string]string, len(nodes))
for _, n := range nodes {
keyToID[n.NodeKey] = n.ID
}
// Delete existing edges for this item/revision
_, err = tx.Exec(ctx, `
DELETE FROM dag_edges
WHERE source_node_id IN (
SELECT id FROM dag_nodes WHERE item_id = $1 AND revision_number = $2
)
`, itemID, revisionNumber)
if err != nil {
return fmt.Errorf("deleting old edges: %w", err)
}
// Insert new edges
for i := range edges {
e := &edges[i]
if e.EdgeType == "" {
e.EdgeType = "depends_on"
}
// Resolve source/target from node keys if IDs are not set
sourceID := e.SourceNodeID
targetID := e.TargetNodeID
if sourceID == "" {
return fmt.Errorf("edge %d: source_node_id is required", i)
}
if targetID == "" {
return fmt.Errorf("edge %d: target_node_id is required", i)
}
metadataJSON, err := json.Marshal(e.Metadata)
if err != nil {
return fmt.Errorf("marshaling edge metadata: %w", err)
}
err = tx.QueryRow(ctx, `
INSERT INTO dag_edges (source_node_id, target_node_id, edge_type, metadata)
VALUES ($1, $2, $3, $4)
RETURNING id
`, sourceID, targetID, e.EdgeType, metadataJSON).Scan(&e.ID)
if err != nil {
return fmt.Errorf("creating edge: %w", err)
}
}
return tx.Commit(ctx)
}
// DeleteNodesForItem removes all DAG nodes (and cascades to edges) for an item/revision.
func (r *DAGRepository) DeleteNodesForItem(ctx context.Context, itemID string, revisionNumber int) error {
_, err := r.db.pool.Exec(ctx, `
DELETE FROM dag_nodes WHERE item_id = $1 AND revision_number = $2
`, itemID, revisionNumber)
if err != nil {
return fmt.Errorf("deleting nodes for item: %w", err)
}
return nil
}
func scanDAGNodes(rows pgx.Rows) ([]*DAGNode, error) {
var nodes []*DAGNode
for rows.Next() {
n := &DAGNode{}
var metadataJSON []byte
err := rows.Scan(
&n.ID, &n.ItemID, &n.RevisionNumber, &n.NodeKey, &n.NodeType,
&n.PropertiesHash, &n.ValidationState, &n.ValidationMsg,
&metadataJSON, &n.CreatedAt, &n.UpdatedAt,
)
if err != nil {
return nil, fmt.Errorf("scanning DAG node: %w", err)
}
if metadataJSON != nil {
if err := json.Unmarshal(metadataJSON, &n.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling node metadata: %w", err)
}
}
nodes = append(nodes, n)
}
return nodes, rows.Err()
}

View File

@@ -0,0 +1,222 @@
package db
import (
"context"
"time"
"github.com/jackc/pgx/v5"
)
// EditSession represents an active editing context.
type EditSession struct {
ID string
ItemID string
UserID string
WorkstationID string
ContextLevel string
ObjectID *string
DependencyCone []string
AcquiredAt time.Time
LastHeartbeat time.Time
}
// EditSessionRepository provides edit session database operations.
type EditSessionRepository struct {
db *DB
}
// NewEditSessionRepository creates a new edit session repository.
func NewEditSessionRepository(db *DB) *EditSessionRepository {
return &EditSessionRepository{db: db}
}
// Acquire inserts a new edit session. Returns a unique constraint error
// if another session already holds the same (item_id, context_level, object_id).
func (r *EditSessionRepository) Acquire(ctx context.Context, s *EditSession) error {
return r.db.pool.QueryRow(ctx, `
INSERT INTO edit_sessions (item_id, user_id, workstation_id, context_level, object_id, dependency_cone)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, acquired_at, last_heartbeat
`, s.ItemID, s.UserID, s.WorkstationID, s.ContextLevel, s.ObjectID, s.DependencyCone).
Scan(&s.ID, &s.AcquiredAt, &s.LastHeartbeat)
}
// Release deletes an edit session by ID.
func (r *EditSessionRepository) Release(ctx context.Context, id string) error {
_, err := r.db.pool.Exec(ctx, `DELETE FROM edit_sessions WHERE id = $1`, id)
return err
}
// ReleaseForWorkstation deletes all sessions for a workstation, returning
// the released sessions so callers can publish SSE notifications.
func (r *EditSessionRepository) ReleaseForWorkstation(ctx context.Context, workstationID string) ([]EditSession, error) {
rows, err := r.db.pool.Query(ctx, `
DELETE FROM edit_sessions
WHERE workstation_id = $1
RETURNING id, item_id, user_id, workstation_id, context_level, object_id, dependency_cone, acquired_at, last_heartbeat
`, workstationID)
if err != nil {
return nil, err
}
defer rows.Close()
var sessions []EditSession
for rows.Next() {
var s EditSession
if err := rows.Scan(&s.ID, &s.ItemID, &s.UserID, &s.WorkstationID,
&s.ContextLevel, &s.ObjectID, &s.DependencyCone,
&s.AcquiredAt, &s.LastHeartbeat); err != nil {
return nil, err
}
sessions = append(sessions, s)
}
return sessions, rows.Err()
}
// GetByID returns an edit session by its ID.
func (r *EditSessionRepository) GetByID(ctx context.Context, id string) (*EditSession, error) {
s := &EditSession{}
err := r.db.pool.QueryRow(ctx, `
SELECT id, item_id, user_id, workstation_id, context_level, object_id,
dependency_cone, acquired_at, last_heartbeat
FROM edit_sessions
WHERE id = $1
`, id).Scan(&s.ID, &s.ItemID, &s.UserID, &s.WorkstationID,
&s.ContextLevel, &s.ObjectID, &s.DependencyCone,
&s.AcquiredAt, &s.LastHeartbeat)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return s, nil
}
// ListForItem returns all active edit sessions for an item.
func (r *EditSessionRepository) ListForItem(ctx context.Context, itemID string) ([]*EditSession, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, item_id, user_id, workstation_id, context_level, object_id,
dependency_cone, acquired_at, last_heartbeat
FROM edit_sessions
WHERE item_id = $1
ORDER BY acquired_at
`, itemID)
if err != nil {
return nil, err
}
defer rows.Close()
var sessions []*EditSession
for rows.Next() {
s := &EditSession{}
if err := rows.Scan(&s.ID, &s.ItemID, &s.UserID, &s.WorkstationID,
&s.ContextLevel, &s.ObjectID, &s.DependencyCone,
&s.AcquiredAt, &s.LastHeartbeat); err != nil {
return nil, err
}
sessions = append(sessions, s)
}
return sessions, rows.Err()
}
// ListForUser returns all active edit sessions for a user.
func (r *EditSessionRepository) ListForUser(ctx context.Context, userID string) ([]*EditSession, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, item_id, user_id, workstation_id, context_level, object_id,
dependency_cone, acquired_at, last_heartbeat
FROM edit_sessions
WHERE user_id = $1
ORDER BY acquired_at
`, userID)
if err != nil {
return nil, err
}
defer rows.Close()
var sessions []*EditSession
for rows.Next() {
s := &EditSession{}
if err := rows.Scan(&s.ID, &s.ItemID, &s.UserID, &s.WorkstationID,
&s.ContextLevel, &s.ObjectID, &s.DependencyCone,
&s.AcquiredAt, &s.LastHeartbeat); err != nil {
return nil, err
}
sessions = append(sessions, s)
}
return sessions, rows.Err()
}
// TouchHeartbeat updates last_heartbeat for all sessions of a workstation.
func (r *EditSessionRepository) TouchHeartbeat(ctx context.Context, workstationID string) error {
_, err := r.db.pool.Exec(ctx, `
UPDATE edit_sessions SET last_heartbeat = now() WHERE workstation_id = $1
`, workstationID)
return err
}
// ExpireStale deletes sessions whose last_heartbeat is older than the given
// timeout, returning the expired sessions for SSE notification.
func (r *EditSessionRepository) ExpireStale(ctx context.Context, timeout time.Duration) ([]EditSession, error) {
rows, err := r.db.pool.Query(ctx, `
DELETE FROM edit_sessions
WHERE last_heartbeat < now() - $1::interval
RETURNING id, item_id, user_id, workstation_id, context_level, object_id, dependency_cone, acquired_at, last_heartbeat
`, timeout.String())
if err != nil {
return nil, err
}
defer rows.Close()
var sessions []EditSession
for rows.Next() {
var s EditSession
if err := rows.Scan(&s.ID, &s.ItemID, &s.UserID, &s.WorkstationID,
&s.ContextLevel, &s.ObjectID, &s.DependencyCone,
&s.AcquiredAt, &s.LastHeartbeat); err != nil {
return nil, err
}
sessions = append(sessions, s)
}
return sessions, rows.Err()
}
// GetConflict returns the existing session holding a given (item, context_level, object_id)
// slot, for building 409 conflict responses.
func (r *EditSessionRepository) GetConflict(ctx context.Context, itemID, contextLevel string, objectID *string) (*EditSession, error) {
s := &EditSession{}
var query string
var args []any
if objectID != nil {
query = `
SELECT id, item_id, user_id, workstation_id, context_level, object_id,
dependency_cone, acquired_at, last_heartbeat
FROM edit_sessions
WHERE item_id = $1 AND context_level = $2 AND object_id = $3
`
args = []any{itemID, contextLevel, *objectID}
} else {
query = `
SELECT id, item_id, user_id, workstation_id, context_level, object_id,
dependency_cone, acquired_at, last_heartbeat
FROM edit_sessions
WHERE item_id = $1 AND context_level = $2 AND object_id IS NULL
`
args = []any{itemID, contextLevel}
}
err := r.db.pool.QueryRow(ctx, query, args...).Scan(
&s.ID, &s.ItemID, &s.UserID, &s.WorkstationID,
&s.ContextLevel, &s.ObjectID, &s.DependencyCone,
&s.AcquiredAt, &s.LastHeartbeat)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return s, nil
}

View File

@@ -0,0 +1,212 @@
package db
import (
"context"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// ItemApproval represents a row in the item_approvals table.
type ItemApproval struct {
ID string
ItemID string
WorkflowName string
ECONumber *string
State string // draft | pending | approved | rejected
UpdatedAt time.Time
UpdatedBy *string
Signatures []ApprovalSignature // populated by WithSignatures methods
}
// ApprovalSignature represents a row in the approval_signatures table.
type ApprovalSignature struct {
ID string
ApprovalID string
Username string
Role string
Status string // pending | approved | rejected
SignedAt *time.Time
Comment *string
}
// ItemApprovalRepository provides item_approvals database operations.
type ItemApprovalRepository struct {
db *DB
}
// NewItemApprovalRepository creates a new item approval repository.
func NewItemApprovalRepository(db *DB) *ItemApprovalRepository {
return &ItemApprovalRepository{db: db}
}
// Create inserts a new approval row. The ID is populated on return.
func (r *ItemApprovalRepository) Create(ctx context.Context, a *ItemApproval) error {
return r.db.pool.QueryRow(ctx, `
INSERT INTO item_approvals (item_id, workflow_name, eco_number, state, updated_by)
VALUES ($1, $2, $3, $4, $5)
RETURNING id, updated_at
`, a.ItemID, a.WorkflowName, a.ECONumber, a.State, a.UpdatedBy).Scan(&a.ID, &a.UpdatedAt)
}
// AddSignature inserts a new signature row. The ID is populated on return.
func (r *ItemApprovalRepository) AddSignature(ctx context.Context, s *ApprovalSignature) error {
return r.db.pool.QueryRow(ctx, `
INSERT INTO approval_signatures (approval_id, username, role, status)
VALUES ($1, $2, $3, $4)
RETURNING id
`, s.ApprovalID, s.Username, s.Role, s.Status).Scan(&s.ID)
}
// GetWithSignatures returns a single approval with its signatures.
func (r *ItemApprovalRepository) GetWithSignatures(ctx context.Context, approvalID string) (*ItemApproval, error) {
a := &ItemApproval{}
err := r.db.pool.QueryRow(ctx, `
SELECT id, item_id, workflow_name, eco_number, state, updated_at, updated_by
FROM item_approvals
WHERE id = $1
`, approvalID).Scan(&a.ID, &a.ItemID, &a.WorkflowName, &a.ECONumber, &a.State, &a.UpdatedAt, &a.UpdatedBy)
if err != nil {
if err == pgx.ErrNoRows {
return nil, nil
}
return nil, fmt.Errorf("getting approval: %w", err)
}
sigs, err := r.signaturesForApproval(ctx, approvalID)
if err != nil {
return nil, err
}
a.Signatures = sigs
return a, nil
}
// ListByItemWithSignatures returns all approvals for an item, each with signatures.
func (r *ItemApprovalRepository) ListByItemWithSignatures(ctx context.Context, itemID string) ([]*ItemApproval, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, item_id, workflow_name, eco_number, state, updated_at, updated_by
FROM item_approvals
WHERE item_id = $1
ORDER BY updated_at DESC
`, itemID)
if err != nil {
return nil, fmt.Errorf("listing approvals: %w", err)
}
defer rows.Close()
var approvals []*ItemApproval
var approvalIDs []string
for rows.Next() {
a := &ItemApproval{}
if err := rows.Scan(&a.ID, &a.ItemID, &a.WorkflowName, &a.ECONumber, &a.State, &a.UpdatedAt, &a.UpdatedBy); err != nil {
return nil, fmt.Errorf("scanning approval: %w", err)
}
approvals = append(approvals, a)
approvalIDs = append(approvalIDs, a.ID)
}
if len(approvalIDs) == 0 {
return approvals, nil
}
// Batch-fetch all signatures
sigRows, err := r.db.pool.Query(ctx, `
SELECT id, approval_id, username, role, status, signed_at, comment
FROM approval_signatures
WHERE approval_id = ANY($1)
ORDER BY username
`, approvalIDs)
if err != nil {
return nil, fmt.Errorf("listing signatures: %w", err)
}
defer sigRows.Close()
sigMap := make(map[string][]ApprovalSignature)
for sigRows.Next() {
var s ApprovalSignature
if err := sigRows.Scan(&s.ID, &s.ApprovalID, &s.Username, &s.Role, &s.Status, &s.SignedAt, &s.Comment); err != nil {
return nil, fmt.Errorf("scanning signature: %w", err)
}
sigMap[s.ApprovalID] = append(sigMap[s.ApprovalID], s)
}
for _, a := range approvals {
a.Signatures = sigMap[a.ID]
if a.Signatures == nil {
a.Signatures = []ApprovalSignature{}
}
}
return approvals, nil
}
// UpdateState updates the approval state and updated_by.
func (r *ItemApprovalRepository) UpdateState(ctx context.Context, approvalID, state, updatedBy string) error {
_, err := r.db.pool.Exec(ctx, `
UPDATE item_approvals
SET state = $2, updated_by = $3, updated_at = now()
WHERE id = $1
`, approvalID, state, updatedBy)
if err != nil {
return fmt.Errorf("updating approval state: %w", err)
}
return nil
}
// GetSignatureForUser returns the signature for a specific user on an approval.
func (r *ItemApprovalRepository) GetSignatureForUser(ctx context.Context, approvalID, username string) (*ApprovalSignature, error) {
s := &ApprovalSignature{}
err := r.db.pool.QueryRow(ctx, `
SELECT id, approval_id, username, role, status, signed_at, comment
FROM approval_signatures
WHERE approval_id = $1 AND username = $2
`, approvalID, username).Scan(&s.ID, &s.ApprovalID, &s.Username, &s.Role, &s.Status, &s.SignedAt, &s.Comment)
if err != nil {
if err == pgx.ErrNoRows {
return nil, nil
}
return nil, fmt.Errorf("getting signature: %w", err)
}
return s, nil
}
// UpdateSignature updates a signature's status, comment, and signed_at timestamp.
func (r *ItemApprovalRepository) UpdateSignature(ctx context.Context, sigID, status string, comment *string) error {
_, err := r.db.pool.Exec(ctx, `
UPDATE approval_signatures
SET status = $2, comment = $3, signed_at = now()
WHERE id = $1
`, sigID, status, comment)
if err != nil {
return fmt.Errorf("updating signature: %w", err)
}
return nil
}
// signaturesForApproval returns all signatures for a single approval.
func (r *ItemApprovalRepository) signaturesForApproval(ctx context.Context, approvalID string) ([]ApprovalSignature, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, approval_id, username, role, status, signed_at, comment
FROM approval_signatures
WHERE approval_id = $1
ORDER BY username
`, approvalID)
if err != nil {
return nil, fmt.Errorf("listing signatures: %w", err)
}
defer rows.Close()
var sigs []ApprovalSignature
for rows.Next() {
var s ApprovalSignature
if err := rows.Scan(&s.ID, &s.ApprovalID, &s.Username, &s.Role, &s.Status, &s.SignedAt, &s.Comment); err != nil {
return nil, fmt.Errorf("scanning signature: %w", err)
}
sigs = append(sigs, s)
}
if sigs == nil {
sigs = []ApprovalSignature{}
}
return sigs, nil
}

View File

@@ -0,0 +1,127 @@
package db
import (
"context"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// ItemDependency represents a row in the item_dependencies table.
type ItemDependency struct {
ID string
ParentItemID string
ChildUUID string
ChildPartNumber *string
ChildRevision *int
Quantity *float64
Label *string
Relationship string
RevisionNumber int
CreatedAt time.Time
}
// ResolvedDependency extends ItemDependency with resolution info from a LEFT JOIN.
type ResolvedDependency struct {
ItemDependency
ResolvedPartNumber *string
ResolvedRevision *int
Resolved bool
}
// ItemDependencyRepository provides item_dependencies database operations.
type ItemDependencyRepository struct {
db *DB
}
// NewItemDependencyRepository creates a new item dependency repository.
func NewItemDependencyRepository(db *DB) *ItemDependencyRepository {
return &ItemDependencyRepository{db: db}
}
// ReplaceForRevision atomically replaces all dependencies for an item's revision.
// Deletes existing rows for the parent item and inserts the new set.
func (r *ItemDependencyRepository) ReplaceForRevision(ctx context.Context, parentItemID string, revisionNumber int, deps []*ItemDependency) error {
return r.db.Tx(ctx, func(tx pgx.Tx) error {
_, err := tx.Exec(ctx, `DELETE FROM item_dependencies WHERE parent_item_id = $1`, parentItemID)
if err != nil {
return fmt.Errorf("deleting old dependencies: %w", err)
}
for _, d := range deps {
_, err := tx.Exec(ctx, `
INSERT INTO item_dependencies
(parent_item_id, child_uuid, child_part_number, child_revision,
quantity, label, relationship, revision_number)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
`, parentItemID, d.ChildUUID, d.ChildPartNumber, d.ChildRevision,
d.Quantity, d.Label, d.Relationship, revisionNumber)
if err != nil {
return fmt.Errorf("inserting dependency: %w", err)
}
}
return nil
})
}
// ListByItem returns all dependencies for an item.
func (r *ItemDependencyRepository) ListByItem(ctx context.Context, parentItemID string) ([]*ItemDependency, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, parent_item_id, child_uuid, child_part_number, child_revision,
quantity, label, relationship, revision_number, created_at
FROM item_dependencies
WHERE parent_item_id = $1
ORDER BY label NULLS LAST
`, parentItemID)
if err != nil {
return nil, fmt.Errorf("listing dependencies: %w", err)
}
defer rows.Close()
var deps []*ItemDependency
for rows.Next() {
d := &ItemDependency{}
if err := rows.Scan(
&d.ID, &d.ParentItemID, &d.ChildUUID, &d.ChildPartNumber, &d.ChildRevision,
&d.Quantity, &d.Label, &d.Relationship, &d.RevisionNumber, &d.CreatedAt,
); err != nil {
return nil, fmt.Errorf("scanning dependency: %w", err)
}
deps = append(deps, d)
}
return deps, nil
}
// Resolve returns dependencies with child UUIDs resolved against the items table.
// Unresolvable UUIDs (external or deleted items) have Resolved=false.
func (r *ItemDependencyRepository) Resolve(ctx context.Context, parentItemID string) ([]*ResolvedDependency, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT d.id, d.parent_item_id, d.child_uuid, d.child_part_number, d.child_revision,
d.quantity, d.label, d.relationship, d.revision_number, d.created_at,
i.part_number, i.current_revision
FROM item_dependencies d
LEFT JOIN items i ON i.id = d.child_uuid AND i.archived_at IS NULL
WHERE d.parent_item_id = $1
ORDER BY d.label NULLS LAST
`, parentItemID)
if err != nil {
return nil, fmt.Errorf("resolving dependencies: %w", err)
}
defer rows.Close()
var deps []*ResolvedDependency
for rows.Next() {
d := &ResolvedDependency{}
if err := rows.Scan(
&d.ID, &d.ParentItemID, &d.ChildUUID, &d.ChildPartNumber, &d.ChildRevision,
&d.Quantity, &d.Label, &d.Relationship, &d.RevisionNumber, &d.CreatedAt,
&d.ResolvedPartNumber, &d.ResolvedRevision,
); err != nil {
return nil, fmt.Errorf("scanning resolved dependency: %w", err)
}
d.Resolved = d.ResolvedPartNumber != nil
deps = append(deps, d)
}
return deps, nil
}

View File

@@ -8,13 +8,14 @@ import (
// ItemFile represents a file attachment on an item.
type ItemFile struct {
ID string
ItemID string
Filename string
ContentType string
Size int64
ObjectKey string
CreatedAt time.Time
ID string
ItemID string
Filename string
ContentType string
Size int64
ObjectKey string
StorageBackend string
CreatedAt time.Time
}
// ItemFileRepository provides item_files database operations.
@@ -29,11 +30,14 @@ func NewItemFileRepository(db *DB) *ItemFileRepository {
// Create inserts a new item file record.
func (r *ItemFileRepository) Create(ctx context.Context, f *ItemFile) error {
if f.StorageBackend == "" {
f.StorageBackend = "filesystem"
}
err := r.db.pool.QueryRow(ctx,
`INSERT INTO item_files (item_id, filename, content_type, size, object_key)
VALUES ($1, $2, $3, $4, $5)
`INSERT INTO item_files (item_id, filename, content_type, size, object_key, storage_backend)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, created_at`,
f.ItemID, f.Filename, f.ContentType, f.Size, f.ObjectKey,
f.ItemID, f.Filename, f.ContentType, f.Size, f.ObjectKey, f.StorageBackend,
).Scan(&f.ID, &f.CreatedAt)
if err != nil {
return fmt.Errorf("creating item file: %w", err)
@@ -44,7 +48,8 @@ func (r *ItemFileRepository) Create(ctx context.Context, f *ItemFile) error {
// ListByItem returns all file attachments for an item.
func (r *ItemFileRepository) ListByItem(ctx context.Context, itemID string) ([]*ItemFile, error) {
rows, err := r.db.pool.Query(ctx,
`SELECT id, item_id, filename, content_type, size, object_key, created_at
`SELECT id, item_id, filename, content_type, size, object_key,
COALESCE(storage_backend, 'filesystem'), created_at
FROM item_files WHERE item_id = $1 ORDER BY created_at`,
itemID,
)
@@ -56,7 +61,7 @@ func (r *ItemFileRepository) ListByItem(ctx context.Context, itemID string) ([]*
var files []*ItemFile
for rows.Next() {
f := &ItemFile{}
if err := rows.Scan(&f.ID, &f.ItemID, &f.Filename, &f.ContentType, &f.Size, &f.ObjectKey, &f.CreatedAt); err != nil {
if err := rows.Scan(&f.ID, &f.ItemID, &f.Filename, &f.ContentType, &f.Size, &f.ObjectKey, &f.StorageBackend, &f.CreatedAt); err != nil {
return nil, fmt.Errorf("scanning item file: %w", err)
}
files = append(files, f)
@@ -68,10 +73,11 @@ func (r *ItemFileRepository) ListByItem(ctx context.Context, itemID string) ([]*
func (r *ItemFileRepository) Get(ctx context.Context, id string) (*ItemFile, error) {
f := &ItemFile{}
err := r.db.pool.QueryRow(ctx,
`SELECT id, item_id, filename, content_type, size, object_key, created_at
`SELECT id, item_id, filename, content_type, size, object_key,
COALESCE(storage_backend, 'filesystem'), created_at
FROM item_files WHERE id = $1`,
id,
).Scan(&f.ID, &f.ItemID, &f.Filename, &f.ContentType, &f.Size, &f.ObjectKey, &f.CreatedAt)
).Scan(&f.ID, &f.ItemID, &f.Filename, &f.ContentType, &f.Size, &f.ObjectKey, &f.StorageBackend, &f.CreatedAt)
if err != nil {
return nil, fmt.Errorf("getting item file: %w", err)
}

View File

@@ -0,0 +1,121 @@
package db
import (
"context"
"testing"
)
func TestItemFileCreate(t *testing.T) {
database := mustConnectTestDB(t)
itemRepo := NewItemRepository(database)
fileRepo := NewItemFileRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "FILE-001", ItemType: "part", Description: "file test"}
if err := itemRepo.Create(ctx, item, nil); err != nil {
t.Fatalf("Create item: %v", err)
}
f := &ItemFile{
ItemID: item.ID,
Filename: "drawing.pdf",
ContentType: "application/pdf",
Size: 12345,
ObjectKey: "items/FILE-001/files/abc/drawing.pdf",
}
if err := fileRepo.Create(ctx, f); err != nil {
t.Fatalf("Create file: %v", err)
}
if f.ID == "" {
t.Error("expected file ID to be set")
}
if f.CreatedAt.IsZero() {
t.Error("expected created_at to be set")
}
}
func TestItemFileListByItem(t *testing.T) {
database := mustConnectTestDB(t)
itemRepo := NewItemRepository(database)
fileRepo := NewItemFileRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "FLIST-001", ItemType: "part", Description: "file list test"}
itemRepo.Create(ctx, item, nil)
for i, name := range []string{"a.pdf", "b.step"} {
fileRepo.Create(ctx, &ItemFile{
ItemID: item.ID,
Filename: name,
ContentType: "application/octet-stream",
Size: int64(i * 1000),
ObjectKey: "items/FLIST-001/files/" + name,
})
}
files, err := fileRepo.ListByItem(ctx, item.ID)
if err != nil {
t.Fatalf("ListByItem: %v", err)
}
if len(files) != 2 {
t.Errorf("expected 2 files, got %d", len(files))
}
}
func TestItemFileGet(t *testing.T) {
database := mustConnectTestDB(t)
itemRepo := NewItemRepository(database)
fileRepo := NewItemFileRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "FGET-001", ItemType: "part", Description: "file get test"}
itemRepo.Create(ctx, item, nil)
f := &ItemFile{
ItemID: item.ID,
Filename: "model.FCStd",
ContentType: "application/x-freecad",
Size: 99999,
ObjectKey: "items/FGET-001/files/xyz/model.FCStd",
}
fileRepo.Create(ctx, f)
got, err := fileRepo.Get(ctx, f.ID)
if err != nil {
t.Fatalf("Get: %v", err)
}
if got.Filename != "model.FCStd" {
t.Errorf("filename: got %q, want %q", got.Filename, "model.FCStd")
}
if got.Size != 99999 {
t.Errorf("size: got %d, want %d", got.Size, 99999)
}
}
func TestItemFileDelete(t *testing.T) {
database := mustConnectTestDB(t)
itemRepo := NewItemRepository(database)
fileRepo := NewItemFileRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "FDEL-001", ItemType: "part", Description: "file delete test"}
itemRepo.Create(ctx, item, nil)
f := &ItemFile{
ItemID: item.ID,
Filename: "temp.bin",
ContentType: "application/octet-stream",
Size: 100,
ObjectKey: "items/FDEL-001/files/tmp/temp.bin",
}
fileRepo.Create(ctx, f)
if err := fileRepo.Delete(ctx, f.ID); err != nil {
t.Fatalf("Delete: %v", err)
}
_, err := fileRepo.Get(ctx, f.ID)
if err == nil {
t.Error("expected error after delete, got nil")
}
}

View File

@@ -0,0 +1,93 @@
package db
import (
"context"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// ItemMacro represents a row in the item_macros table.
type ItemMacro struct {
ID string
ItemID string
Filename string
Trigger string
Content string
RevisionNumber int
CreatedAt time.Time
}
// ItemMacroRepository provides item_macros database operations.
type ItemMacroRepository struct {
db *DB
}
// NewItemMacroRepository creates a new item macro repository.
func NewItemMacroRepository(db *DB) *ItemMacroRepository {
return &ItemMacroRepository{db: db}
}
// ReplaceForItem atomically replaces all macros for an item.
// Deletes existing rows and inserts the new set.
func (r *ItemMacroRepository) ReplaceForItem(ctx context.Context, itemID string, revisionNumber int, macros []*ItemMacro) error {
return r.db.Tx(ctx, func(tx pgx.Tx) error {
_, err := tx.Exec(ctx, `DELETE FROM item_macros WHERE item_id = $1`, itemID)
if err != nil {
return fmt.Errorf("deleting old macros: %w", err)
}
for _, m := range macros {
_, err := tx.Exec(ctx, `
INSERT INTO item_macros (item_id, filename, trigger, content, revision_number)
VALUES ($1, $2, $3, $4, $5)
`, itemID, m.Filename, m.Trigger, m.Content, revisionNumber)
if err != nil {
return fmt.Errorf("inserting macro %s: %w", m.Filename, err)
}
}
return nil
})
}
// ListByItem returns all macros for an item (without content), ordered by filename.
func (r *ItemMacroRepository) ListByItem(ctx context.Context, itemID string) ([]*ItemMacro, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, item_id, filename, trigger, revision_number, created_at
FROM item_macros
WHERE item_id = $1
ORDER BY filename
`, itemID)
if err != nil {
return nil, fmt.Errorf("listing macros: %w", err)
}
defer rows.Close()
var macros []*ItemMacro
for rows.Next() {
m := &ItemMacro{}
if err := rows.Scan(&m.ID, &m.ItemID, &m.Filename, &m.Trigger, &m.RevisionNumber, &m.CreatedAt); err != nil {
return nil, fmt.Errorf("scanning macro: %w", err)
}
macros = append(macros, m)
}
return macros, nil
}
// GetByFilename returns a single macro by item ID and filename, including content.
func (r *ItemMacroRepository) GetByFilename(ctx context.Context, itemID string, filename string) (*ItemMacro, error) {
m := &ItemMacro{}
err := r.db.pool.QueryRow(ctx, `
SELECT id, item_id, filename, trigger, content, revision_number, created_at
FROM item_macros
WHERE item_id = $1 AND filename = $2
`, itemID, filename).Scan(&m.ID, &m.ItemID, &m.Filename, &m.Trigger, &m.Content, &m.RevisionNumber, &m.CreatedAt)
if err != nil {
if err == pgx.ErrNoRows {
return nil, nil
}
return nil, fmt.Errorf("getting macro: %w", err)
}
return m, nil
}

View File

@@ -0,0 +1,161 @@
package db
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// ItemMetadata represents a row in the item_metadata table.
type ItemMetadata struct {
ItemID string
SchemaName *string
Tags []string
LifecycleState string
Fields map[string]any
KCVersion *string
ManifestUUID *string
SiloInstance *string
RevisionHash *string
UpdatedAt time.Time
UpdatedBy *string
}
// ItemMetadataRepository provides item_metadata database operations.
type ItemMetadataRepository struct {
db *DB
}
// NewItemMetadataRepository creates a new item metadata repository.
func NewItemMetadataRepository(db *DB) *ItemMetadataRepository {
return &ItemMetadataRepository{db: db}
}
// Get returns metadata for an item, or nil if none exists.
func (r *ItemMetadataRepository) Get(ctx context.Context, itemID string) (*ItemMetadata, error) {
m := &ItemMetadata{}
var fieldsJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT item_id, schema_name, tags, lifecycle_state, fields,
kc_version, manifest_uuid, silo_instance, revision_hash,
updated_at, updated_by
FROM item_metadata
WHERE item_id = $1
`, itemID).Scan(
&m.ItemID, &m.SchemaName, &m.Tags, &m.LifecycleState, &fieldsJSON,
&m.KCVersion, &m.ManifestUUID, &m.SiloInstance, &m.RevisionHash,
&m.UpdatedAt, &m.UpdatedBy,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("getting item metadata: %w", err)
}
if fieldsJSON != nil {
if err := json.Unmarshal(fieldsJSON, &m.Fields); err != nil {
return nil, fmt.Errorf("unmarshaling fields: %w", err)
}
}
if m.Fields == nil {
m.Fields = make(map[string]any)
}
if m.Tags == nil {
m.Tags = []string{}
}
return m, nil
}
// Upsert inserts or updates the metadata row for an item.
// Used by the commit extraction pipeline.
func (r *ItemMetadataRepository) Upsert(ctx context.Context, m *ItemMetadata) error {
fieldsJSON, err := json.Marshal(m.Fields)
if err != nil {
return fmt.Errorf("marshaling fields: %w", err)
}
_, err = r.db.pool.Exec(ctx, `
INSERT INTO item_metadata
(item_id, schema_name, tags, lifecycle_state, fields,
kc_version, manifest_uuid, silo_instance, revision_hash,
updated_at, updated_by)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, now(), $10)
ON CONFLICT (item_id) DO UPDATE SET
schema_name = EXCLUDED.schema_name,
tags = EXCLUDED.tags,
lifecycle_state = EXCLUDED.lifecycle_state,
fields = EXCLUDED.fields,
kc_version = EXCLUDED.kc_version,
manifest_uuid = EXCLUDED.manifest_uuid,
silo_instance = EXCLUDED.silo_instance,
revision_hash = EXCLUDED.revision_hash,
updated_at = now(),
updated_by = EXCLUDED.updated_by
`, m.ItemID, m.SchemaName, m.Tags, m.LifecycleState, fieldsJSON,
m.KCVersion, m.ManifestUUID, m.SiloInstance, m.RevisionHash,
m.UpdatedBy)
if err != nil {
return fmt.Errorf("upserting item metadata: %w", err)
}
return nil
}
// UpdateFields merges the given fields into the existing JSONB fields column.
func (r *ItemMetadataRepository) UpdateFields(ctx context.Context, itemID string, fields map[string]any, updatedBy string) error {
fieldsJSON, err := json.Marshal(fields)
if err != nil {
return fmt.Errorf("marshaling fields: %w", err)
}
tag, err := r.db.pool.Exec(ctx, `
UPDATE item_metadata
SET fields = fields || $2::jsonb,
updated_at = now(),
updated_by = $3
WHERE item_id = $1
`, itemID, fieldsJSON, updatedBy)
if err != nil {
return fmt.Errorf("updating metadata fields: %w", err)
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("item metadata not found")
}
return nil
}
// UpdateLifecycle sets the lifecycle_state column.
func (r *ItemMetadataRepository) UpdateLifecycle(ctx context.Context, itemID, state, updatedBy string) error {
tag, err := r.db.pool.Exec(ctx, `
UPDATE item_metadata
SET lifecycle_state = $2,
updated_at = now(),
updated_by = $3
WHERE item_id = $1
`, itemID, state, updatedBy)
if err != nil {
return fmt.Errorf("updating lifecycle state: %w", err)
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("item metadata not found")
}
return nil
}
// SetTags replaces the tags array.
func (r *ItemMetadataRepository) SetTags(ctx context.Context, itemID string, tags []string, updatedBy string) error {
tag, err := r.db.pool.Exec(ctx, `
UPDATE item_metadata
SET tags = $2,
updated_at = now(),
updated_by = $3
WHERE item_id = $1
`, itemID, tags, updatedBy)
if err != nil {
return fmt.Errorf("updating tags: %w", err)
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("item metadata not found")
}
return nil
}

View File

@@ -24,29 +24,28 @@ type Item struct {
CADFilePath *string
CreatedBy *string
UpdatedBy *string
SourcingType string // "manufactured" or "purchased"
SourcingLink *string // URL to supplier/datasheet
LongDescription *string // extended description
StandardCost *float64 // baseline unit cost
ThumbnailKey *string // MinIO key for item thumbnail
SourcingType string // "manufactured" or "purchased"
LongDescription *string // extended description
ThumbnailKey *string // storage key for item thumbnail
}
// Revision represents a revision record.
type Revision struct {
ID string
ItemID string
RevisionNumber int
Properties map[string]any
FileKey *string
FileVersion *string
FileChecksum *string
FileSize *int64
ThumbnailKey *string
CreatedAt time.Time
CreatedBy *string
Comment *string
Status string // draft, review, released, obsolete
Labels []string // arbitrary tags
ID string
ItemID string
RevisionNumber int
Properties map[string]any
FileKey *string
FileVersion *string
FileChecksum *string
FileSize *int64
FileStorageBackend string
ThumbnailKey *string
CreatedAt time.Time
CreatedBy *string
Comment *string
Status string // draft, review, released, obsolete
Labels []string // arbitrary tags
}
// RevisionStatus constants
@@ -96,11 +95,11 @@ func (r *ItemRepository) Create(ctx context.Context, item *Item, properties map[
}
err := tx.QueryRow(ctx, `
INSERT INTO items (part_number, schema_id, item_type, description, created_by,
sourcing_type, sourcing_link, long_description, standard_cost)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
sourcing_type, long_description)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING id, created_at, updated_at, current_revision
`, item.PartNumber, item.SchemaID, item.ItemType, item.Description, item.CreatedBy,
sourcingType, item.SourcingLink, item.LongDescription, item.StandardCost,
sourcingType, item.LongDescription,
).Scan(
&item.ID, &item.CreatedAt, &item.UpdatedAt, &item.CurrentRevision,
)
@@ -133,7 +132,7 @@ func (r *ItemRepository) GetByPartNumber(ctx context.Context, partNumber string)
SELECT id, part_number, schema_id, item_type, description,
created_at, updated_at, archived_at, current_revision,
cad_synced_at, cad_file_path,
sourcing_type, sourcing_link, long_description, standard_cost,
sourcing_type, long_description,
thumbnail_key
FROM items
WHERE part_number = $1 AND archived_at IS NULL
@@ -141,7 +140,7 @@ func (r *ItemRepository) GetByPartNumber(ctx context.Context, partNumber string)
&item.ID, &item.PartNumber, &item.SchemaID, &item.ItemType, &item.Description,
&item.CreatedAt, &item.UpdatedAt, &item.ArchivedAt, &item.CurrentRevision,
&item.CADSyncedAt, &item.CADFilePath,
&item.SourcingType, &item.SourcingLink, &item.LongDescription, &item.StandardCost,
&item.SourcingType, &item.LongDescription,
&item.ThumbnailKey,
)
if err == pgx.ErrNoRows {
@@ -160,7 +159,7 @@ func (r *ItemRepository) GetByID(ctx context.Context, id string) (*Item, error)
SELECT id, part_number, schema_id, item_type, description,
created_at, updated_at, archived_at, current_revision,
cad_synced_at, cad_file_path,
sourcing_type, sourcing_link, long_description, standard_cost,
sourcing_type, long_description,
thumbnail_key
FROM items
WHERE id = $1
@@ -168,7 +167,7 @@ func (r *ItemRepository) GetByID(ctx context.Context, id string) (*Item, error)
&item.ID, &item.PartNumber, &item.SchemaID, &item.ItemType, &item.Description,
&item.CreatedAt, &item.UpdatedAt, &item.ArchivedAt, &item.CurrentRevision,
&item.CADSyncedAt, &item.CADFilePath,
&item.SourcingType, &item.SourcingLink, &item.LongDescription, &item.StandardCost,
&item.SourcingType, &item.LongDescription,
&item.ThumbnailKey,
)
if err == pgx.ErrNoRows {
@@ -192,7 +191,7 @@ func (r *ItemRepository) List(ctx context.Context, opts ListOptions) ([]*Item, e
query = `
SELECT DISTINCT i.id, i.part_number, i.schema_id, i.item_type, i.description,
i.created_at, i.updated_at, i.archived_at, i.current_revision,
i.sourcing_type, i.sourcing_link, i.long_description, i.standard_cost,
i.sourcing_type, i.long_description,
i.thumbnail_key
FROM items i
JOIN item_projects ip ON ip.item_id = i.id
@@ -205,7 +204,7 @@ func (r *ItemRepository) List(ctx context.Context, opts ListOptions) ([]*Item, e
query = `
SELECT id, part_number, schema_id, item_type, description,
created_at, updated_at, archived_at, current_revision,
sourcing_type, sourcing_link, long_description, standard_cost,
sourcing_type, long_description,
thumbnail_key
FROM items
WHERE archived_at IS NULL
@@ -257,7 +256,7 @@ func (r *ItemRepository) List(ctx context.Context, opts ListOptions) ([]*Item, e
err := rows.Scan(
&item.ID, &item.PartNumber, &item.SchemaID, &item.ItemType, &item.Description,
&item.CreatedAt, &item.UpdatedAt, &item.ArchivedAt, &item.CurrentRevision,
&item.SourcingType, &item.SourcingLink, &item.LongDescription, &item.StandardCost,
&item.SourcingType, &item.LongDescription,
&item.ThumbnailKey,
)
if err != nil {
@@ -308,16 +307,20 @@ func (r *ItemRepository) CreateRevision(ctx context.Context, rev *Revision) erro
return fmt.Errorf("marshaling properties: %w", err)
}
if rev.FileStorageBackend == "" {
rev.FileStorageBackend = "filesystem"
}
err = r.db.pool.QueryRow(ctx, `
INSERT INTO revisions (
item_id, revision_number, properties, file_key, file_version,
file_checksum, file_size, thumbnail_key, created_by, comment
file_checksum, file_size, file_storage_backend, thumbnail_key, created_by, comment
)
SELECT $1, current_revision + 1, $2, $3, $4, $5, $6, $7, $8, $9
SELECT $1, current_revision + 1, $2, $3, $4, $5, $6, $7, $8, $9, $10
FROM items WHERE id = $1
RETURNING id, revision_number, created_at
`, rev.ItemID, propsJSON, rev.FileKey, rev.FileVersion,
rev.FileChecksum, rev.FileSize, rev.ThumbnailKey, rev.CreatedBy, rev.Comment,
rev.FileChecksum, rev.FileSize, rev.FileStorageBackend, rev.ThumbnailKey, rev.CreatedBy, rev.Comment,
).Scan(&rev.ID, &rev.RevisionNumber, &rev.CreatedAt)
if err != nil {
return fmt.Errorf("inserting revision: %w", err)
@@ -344,7 +347,8 @@ func (r *ItemRepository) GetRevisions(ctx context.Context, itemID string) ([]*Re
if hasStatusColumn {
rows, err = r.db.pool.Query(ctx, `
SELECT id, item_id, revision_number, properties, file_key, file_version,
file_checksum, file_size, thumbnail_key, created_at, created_by, comment,
file_checksum, file_size, COALESCE(file_storage_backend, 'filesystem'),
thumbnail_key, created_at, created_by, comment,
COALESCE(status, 'draft') as status, COALESCE(labels, ARRAY[]::TEXT[]) as labels
FROM revisions
WHERE item_id = $1
@@ -371,7 +375,8 @@ func (r *ItemRepository) GetRevisions(ctx context.Context, itemID string) ([]*Re
if hasStatusColumn {
err = rows.Scan(
&rev.ID, &rev.ItemID, &rev.RevisionNumber, &propsJSON, &rev.FileKey, &rev.FileVersion,
&rev.FileChecksum, &rev.FileSize, &rev.ThumbnailKey, &rev.CreatedAt, &rev.CreatedBy, &rev.Comment,
&rev.FileChecksum, &rev.FileSize, &rev.FileStorageBackend,
&rev.ThumbnailKey, &rev.CreatedAt, &rev.CreatedBy, &rev.Comment,
&rev.Status, &rev.Labels,
)
} else {
@@ -381,6 +386,7 @@ func (r *ItemRepository) GetRevisions(ctx context.Context, itemID string) ([]*Re
)
rev.Status = "draft"
rev.Labels = []string{}
rev.FileStorageBackend = "filesystem"
}
if err != nil {
return nil, fmt.Errorf("scanning revision: %w", err)
@@ -414,13 +420,15 @@ func (r *ItemRepository) GetRevision(ctx context.Context, itemID string, revisio
if hasStatusColumn {
err = r.db.pool.QueryRow(ctx, `
SELECT id, item_id, revision_number, properties, file_key, file_version,
file_checksum, file_size, thumbnail_key, created_at, created_by, comment,
file_checksum, file_size, COALESCE(file_storage_backend, 'filesystem'),
thumbnail_key, created_at, created_by, comment,
COALESCE(status, 'draft') as status, COALESCE(labels, ARRAY[]::TEXT[]) as labels
FROM revisions
WHERE item_id = $1 AND revision_number = $2
`, itemID, revisionNumber).Scan(
&rev.ID, &rev.ItemID, &rev.RevisionNumber, &propsJSON, &rev.FileKey, &rev.FileVersion,
&rev.FileChecksum, &rev.FileSize, &rev.ThumbnailKey, &rev.CreatedAt, &rev.CreatedBy, &rev.Comment,
&rev.FileChecksum, &rev.FileSize, &rev.FileStorageBackend,
&rev.ThumbnailKey, &rev.CreatedAt, &rev.CreatedBy, &rev.Comment,
&rev.Status, &rev.Labels,
)
} else {
@@ -435,6 +443,7 @@ func (r *ItemRepository) GetRevision(ctx context.Context, itemID string, revisio
)
rev.Status = "draft"
rev.Labels = []string{}
rev.FileStorageBackend = "filesystem"
}
if err == pgx.ErrNoRows {
@@ -608,15 +617,16 @@ func (r *ItemRepository) CreateRevisionFromExisting(ctx context.Context, itemID
// Create new revision with copied properties (and optionally file reference)
newRev := &Revision{
ItemID: itemID,
Properties: source.Properties,
FileKey: source.FileKey,
FileVersion: source.FileVersion,
FileChecksum: source.FileChecksum,
FileSize: source.FileSize,
ThumbnailKey: source.ThumbnailKey,
CreatedBy: createdBy,
Comment: &comment,
ItemID: itemID,
Properties: source.Properties,
FileKey: source.FileKey,
FileVersion: source.FileVersion,
FileChecksum: source.FileChecksum,
FileSize: source.FileSize,
FileStorageBackend: source.FileStorageBackend,
ThumbnailKey: source.ThumbnailKey,
CreatedBy: createdBy,
Comment: &comment,
}
// Insert the new revision
@@ -628,13 +638,13 @@ func (r *ItemRepository) CreateRevisionFromExisting(ctx context.Context, itemID
err = r.db.pool.QueryRow(ctx, `
INSERT INTO revisions (
item_id, revision_number, properties, file_key, file_version,
file_checksum, file_size, thumbnail_key, created_by, comment, status
file_checksum, file_size, file_storage_backend, thumbnail_key, created_by, comment, status
)
SELECT $1, current_revision + 1, $2, $3, $4, $5, $6, $7, $8, $9, 'draft'
SELECT $1, current_revision + 1, $2, $3, $4, $5, $6, $7, $8, $9, $10, 'draft'
FROM items WHERE id = $1
RETURNING id, revision_number, created_at
`, newRev.ItemID, propsJSON, newRev.FileKey, newRev.FileVersion,
newRev.FileChecksum, newRev.FileSize, newRev.ThumbnailKey, newRev.CreatedBy, newRev.Comment,
newRev.FileChecksum, newRev.FileSize, newRev.FileStorageBackend, newRev.ThumbnailKey, newRev.CreatedBy, newRev.Comment,
).Scan(&newRev.ID, &newRev.RevisionNumber, &newRev.CreatedAt)
if err != nil {
return nil, fmt.Errorf("inserting revision: %w", err)
@@ -659,9 +669,7 @@ type UpdateItemFields struct {
Description string
UpdatedBy *string
SourcingType *string
SourcingLink *string
LongDescription *string
StandardCost *float64
}
// Update modifies an item's fields. The UUID remains stable.
@@ -670,16 +678,12 @@ func (r *ItemRepository) Update(ctx context.Context, id string, fields UpdateIte
UPDATE items
SET part_number = $2, item_type = $3, description = $4, updated_by = $5,
sourcing_type = COALESCE($6, sourcing_type),
sourcing_link = CASE WHEN $7::boolean THEN $8 ELSE sourcing_link END,
long_description = CASE WHEN $9::boolean THEN $10 ELSE long_description END,
standard_cost = CASE WHEN $11::boolean THEN $12 ELSE standard_cost END,
long_description = CASE WHEN $7::boolean THEN $8 ELSE long_description END,
updated_at = now()
WHERE id = $1 AND archived_at IS NULL
`, id, fields.PartNumber, fields.ItemType, fields.Description, fields.UpdatedBy,
fields.SourcingType,
fields.SourcingLink != nil, fields.SourcingLink,
fields.LongDescription != nil, fields.LongDescription,
fields.StandardCost != nil, fields.StandardCost,
)
if err != nil {
return fmt.Errorf("updating item: %w", err)

View File

@@ -0,0 +1,281 @@
package db
import (
"context"
"fmt"
"strings"
"testing"
)
func TestItemCreateDuplicatePartNumber(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "DUP-001", ItemType: "part", Description: "first"}
if err := repo.Create(ctx, item, nil); err != nil {
t.Fatalf("Create: %v", err)
}
dup := &Item{PartNumber: "DUP-001", ItemType: "part", Description: "duplicate"}
err := repo.Create(ctx, dup, nil)
if err == nil {
t.Fatal("expected error for duplicate part number, got nil")
}
if !strings.Contains(err.Error(), "23505") && !strings.Contains(err.Error(), "duplicate") {
t.Errorf("expected duplicate key error, got: %v", err)
}
}
func TestItemDelete(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "HDEL-001", ItemType: "part", Description: "hard delete"}
if err := repo.Create(ctx, item, nil); err != nil {
t.Fatalf("Create: %v", err)
}
if err := repo.Delete(ctx, item.ID); err != nil {
t.Fatalf("Delete: %v", err)
}
got, err := repo.GetByID(ctx, item.ID)
if err != nil {
t.Fatalf("GetByID after delete: %v", err)
}
if got != nil {
t.Error("expected nil after hard delete")
}
}
func TestItemListPagination(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
for i := 0; i < 5; i++ {
item := &Item{
PartNumber: fmt.Sprintf("PAGE-%04d", i),
ItemType: "part",
Description: fmt.Sprintf("page item %d", i),
}
if err := repo.Create(ctx, item, nil); err != nil {
t.Fatalf("Create #%d: %v", i, err)
}
}
// Fetch page of 2 with offset 2
items, err := repo.List(ctx, ListOptions{Limit: 2, Offset: 2})
if err != nil {
t.Fatalf("List: %v", err)
}
if len(items) != 2 {
t.Errorf("expected 2 items, got %d", len(items))
}
}
func TestItemListSearch(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
repo.Create(ctx, &Item{PartNumber: "SRCH-001", ItemType: "part", Description: "alpha widget"}, nil)
repo.Create(ctx, &Item{PartNumber: "SRCH-002", ItemType: "part", Description: "beta gadget"}, nil)
repo.Create(ctx, &Item{PartNumber: "SRCH-003", ItemType: "part", Description: "alpha gizmo"}, nil)
items, err := repo.List(ctx, ListOptions{Search: "alpha"})
if err != nil {
t.Fatalf("List: %v", err)
}
if len(items) != 2 {
t.Errorf("expected 2 items matching 'alpha', got %d", len(items))
}
}
func TestRevisionStatusUpdate(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "STAT-001", ItemType: "part", Description: "status test"}
if err := repo.Create(ctx, item, map[string]any{"v": 1}); err != nil {
t.Fatalf("Create: %v", err)
}
status := "released"
if err := repo.UpdateRevisionStatus(ctx, item.ID, 1, &status, nil); err != nil {
t.Fatalf("UpdateRevisionStatus: %v", err)
}
rev, err := repo.GetRevision(ctx, item.ID, 1)
if err != nil {
t.Fatalf("GetRevision: %v", err)
}
if rev.Status != "released" {
t.Errorf("status: got %q, want %q", rev.Status, "released")
}
}
func TestRevisionLabelsUpdate(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "LBL-001", ItemType: "part", Description: "label test"}
if err := repo.Create(ctx, item, nil); err != nil {
t.Fatalf("Create: %v", err)
}
labels := []string{"prototype", "urgent"}
if err := repo.UpdateRevisionStatus(ctx, item.ID, 1, nil, labels); err != nil {
t.Fatalf("UpdateRevisionStatus: %v", err)
}
rev, err := repo.GetRevision(ctx, item.ID, 1)
if err != nil {
t.Fatalf("GetRevision: %v", err)
}
if len(rev.Labels) != 2 {
t.Errorf("labels count: got %d, want 2", len(rev.Labels))
}
}
func TestRevisionCompare(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "CMP-001", ItemType: "part", Description: "compare test"}
if err := repo.Create(ctx, item, map[string]any{"color": "red", "weight": 10}); err != nil {
t.Fatalf("Create: %v", err)
}
// Rev 2: change color, remove weight, add size
repo.CreateRevision(ctx, &Revision{
ItemID: item.ID,
Properties: map[string]any{"color": "blue", "size": "large"},
})
diff, err := repo.CompareRevisions(ctx, item.ID, 1, 2)
if err != nil {
t.Fatalf("CompareRevisions: %v", err)
}
if len(diff.Added) == 0 {
t.Error("expected added fields (size)")
}
if len(diff.Removed) == 0 {
t.Error("expected removed fields (weight)")
}
if len(diff.Changed) == 0 {
t.Error("expected changed fields (color)")
}
}
func TestRevisionRollback(t *testing.T) {
database := mustConnectTestDB(t)
repo := NewItemRepository(database)
ctx := context.Background()
item := &Item{PartNumber: "RBK-001", ItemType: "part", Description: "rollback test"}
if err := repo.Create(ctx, item, map[string]any{"version": "original"}); err != nil {
t.Fatalf("Create: %v", err)
}
// Rev 2: change property
repo.CreateRevision(ctx, &Revision{
ItemID: item.ID,
Properties: map[string]any{"version": "modified"},
})
// Rollback to rev 1 — should create rev 3
comment := "rollback to rev 1"
rev3, err := repo.CreateRevisionFromExisting(ctx, item.ID, 1, comment, nil)
if err != nil {
t.Fatalf("CreateRevisionFromExisting: %v", err)
}
if rev3.RevisionNumber != 3 {
t.Errorf("revision number: got %d, want 3", rev3.RevisionNumber)
}
// Rev 3 should have rev 1's properties
got, err := repo.GetRevision(ctx, item.ID, 3)
if err != nil {
t.Fatalf("GetRevision: %v", err)
}
if got.Properties["version"] != "original" {
t.Errorf("rolled back version: got %v, want %q", got.Properties["version"], "original")
}
}
func TestProjectItemAssociationsByCode(t *testing.T) {
database := mustConnectTestDB(t)
projRepo := NewProjectRepository(database)
itemRepo := NewItemRepository(database)
ctx := context.Background()
proj := &Project{Code: "BYTAG", Name: "Tag Project"}
projRepo.Create(ctx, proj)
item := &Item{PartNumber: "TAG-001", ItemType: "part", Description: "taggable"}
itemRepo.Create(ctx, item, nil)
// Tag by code
if err := projRepo.AddItemToProjectByCode(ctx, item.ID, "BYTAG"); err != nil {
t.Fatalf("AddItemToProjectByCode: %v", err)
}
projects, err := projRepo.GetProjectsForItem(ctx, item.ID)
if err != nil {
t.Fatalf("GetProjectsForItem: %v", err)
}
if len(projects) != 1 {
t.Fatalf("expected 1 project, got %d", len(projects))
}
if projects[0].Code != "BYTAG" {
t.Errorf("project code: got %q, want %q", projects[0].Code, "BYTAG")
}
// Untag by code
if err := projRepo.RemoveItemFromProjectByCode(ctx, item.ID, "BYTAG"); err != nil {
t.Fatalf("RemoveItemFromProjectByCode: %v", err)
}
projects, _ = projRepo.GetProjectsForItem(ctx, item.ID)
if len(projects) != 0 {
t.Errorf("expected 0 projects after removal, got %d", len(projects))
}
}
func TestListByProject(t *testing.T) {
database := mustConnectTestDB(t)
projRepo := NewProjectRepository(database)
itemRepo := NewItemRepository(database)
ctx := context.Background()
proj := &Project{Code: "FILT", Name: "Filter Project"}
projRepo.Create(ctx, proj)
// Create 3 items, tag only 2
for i := 0; i < 3; i++ {
item := &Item{
PartNumber: fmt.Sprintf("FILT-%04d", i),
ItemType: "part",
Description: fmt.Sprintf("filter item %d", i),
}
itemRepo.Create(ctx, item, nil)
if i < 2 {
projRepo.AddItemToProjectByCode(ctx, item.ID, "FILT")
}
}
items, err := itemRepo.List(ctx, ListOptions{Project: "FILT"})
if err != nil {
t.Fatalf("List with project filter: %v", err)
}
if len(items) != 2 {
t.Errorf("expected 2 items in project FILT, got %d", len(items))
}
}

View File

@@ -134,12 +134,10 @@ func TestItemUpdate(t *testing.T) {
t.Fatalf("Create: %v", err)
}
cost := 42.50
err := repo.Update(ctx, item.ID, UpdateItemFields{
PartNumber: "UPD-001",
ItemType: "part",
Description: "updated",
StandardCost: &cost,
PartNumber: "UPD-001",
ItemType: "part",
Description: "updated",
})
if err != nil {
t.Fatalf("Update: %v", err)
@@ -149,9 +147,6 @@ func TestItemUpdate(t *testing.T) {
if got.Description != "updated" {
t.Errorf("description: got %q, want %q", got.Description, "updated")
}
if got.StandardCost == nil || *got.StandardCost != 42.50 {
t.Errorf("standard_cost: got %v, want 42.50", got.StandardCost)
}
}
func TestItemArchiveUnarchive(t *testing.T) {

808
internal/db/jobs.go Normal file
View File

@@ -0,0 +1,808 @@
package db
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// Runner represents a registered compute worker.
type Runner struct {
ID string
Name string
TokenHash string
TokenPrefix string
Tags []string
Status string
LastHeartbeat *time.Time
LastJobID *string
Metadata map[string]any
CreatedAt time.Time
UpdatedAt time.Time
}
// JobDefinitionRecord is a job definition stored in the database.
type JobDefinitionRecord struct {
ID string
Name string
Version int
TriggerType string
ScopeType string
ComputeType string
RunnerTags []string
TimeoutSeconds int
MaxRetries int
Priority int
Definition map[string]any
Enabled bool
CreatedAt time.Time
UpdatedAt time.Time
}
// Job represents a single compute job instance.
type Job struct {
ID string
JobDefinitionID *string
DefinitionName string
Status string
Priority int
ItemID *string
ProjectID *string
ScopeMetadata map[string]any
RunnerID *string
RunnerTags []string
CreatedAt time.Time
ClaimedAt *time.Time
StartedAt *time.Time
CompletedAt *time.Time
TimeoutSeconds int
ExpiresAt *time.Time
Progress int
ProgressMessage *string
Result map[string]any
ErrorMessage *string
RetryCount int
MaxRetries int
CreatedBy *string
CancelledBy *string
}
// JobLogEntry is a single log line for a job.
type JobLogEntry struct {
ID string
JobID string
Timestamp time.Time
Level string
Message string
Metadata map[string]any
}
// JobRepository provides job and runner database operations.
type JobRepository struct {
db *DB
}
// NewJobRepository creates a new job repository.
func NewJobRepository(db *DB) *JobRepository {
return &JobRepository{db: db}
}
// ---------------------------------------------------------------------------
// Job Definitions
// ---------------------------------------------------------------------------
// UpsertDefinition inserts or updates a job definition record.
func (r *JobRepository) UpsertDefinition(ctx context.Context, d *JobDefinitionRecord) error {
defJSON, err := json.Marshal(d.Definition)
if err != nil {
return fmt.Errorf("marshaling definition: %w", err)
}
err = r.db.pool.QueryRow(ctx, `
INSERT INTO job_definitions (name, version, trigger_type, scope_type, compute_type,
runner_tags, timeout_seconds, max_retries, priority,
definition, enabled)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
ON CONFLICT (name) DO UPDATE SET
version = EXCLUDED.version,
trigger_type = EXCLUDED.trigger_type,
scope_type = EXCLUDED.scope_type,
compute_type = EXCLUDED.compute_type,
runner_tags = EXCLUDED.runner_tags,
timeout_seconds = EXCLUDED.timeout_seconds,
max_retries = EXCLUDED.max_retries,
priority = EXCLUDED.priority,
definition = EXCLUDED.definition,
enabled = EXCLUDED.enabled,
updated_at = now()
RETURNING id, created_at, updated_at
`, d.Name, d.Version, d.TriggerType, d.ScopeType, d.ComputeType,
d.RunnerTags, d.TimeoutSeconds, d.MaxRetries, d.Priority,
defJSON, d.Enabled,
).Scan(&d.ID, &d.CreatedAt, &d.UpdatedAt)
if err != nil {
return fmt.Errorf("upserting job definition: %w", err)
}
return nil
}
// GetDefinition returns a job definition by name.
func (r *JobRepository) GetDefinition(ctx context.Context, name string) (*JobDefinitionRecord, error) {
d := &JobDefinitionRecord{}
var defJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, name, version, trigger_type, scope_type, compute_type,
runner_tags, timeout_seconds, max_retries, priority,
definition, enabled, created_at, updated_at
FROM job_definitions WHERE name = $1
`, name).Scan(
&d.ID, &d.Name, &d.Version, &d.TriggerType, &d.ScopeType, &d.ComputeType,
&d.RunnerTags, &d.TimeoutSeconds, &d.MaxRetries, &d.Priority,
&defJSON, &d.Enabled, &d.CreatedAt, &d.UpdatedAt,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("querying job definition: %w", err)
}
if defJSON != nil {
if err := json.Unmarshal(defJSON, &d.Definition); err != nil {
return nil, fmt.Errorf("unmarshaling definition: %w", err)
}
}
return d, nil
}
// ListDefinitions returns all job definitions.
func (r *JobRepository) ListDefinitions(ctx context.Context) ([]*JobDefinitionRecord, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, name, version, trigger_type, scope_type, compute_type,
runner_tags, timeout_seconds, max_retries, priority,
definition, enabled, created_at, updated_at
FROM job_definitions ORDER BY name
`)
if err != nil {
return nil, fmt.Errorf("querying job definitions: %w", err)
}
defer rows.Close()
return scanJobDefinitions(rows)
}
// GetDefinitionsByTrigger returns all enabled definitions matching a trigger type.
func (r *JobRepository) GetDefinitionsByTrigger(ctx context.Context, triggerType string) ([]*JobDefinitionRecord, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, name, version, trigger_type, scope_type, compute_type,
runner_tags, timeout_seconds, max_retries, priority,
definition, enabled, created_at, updated_at
FROM job_definitions
WHERE trigger_type = $1 AND enabled = true
ORDER BY priority ASC, name
`, triggerType)
if err != nil {
return nil, fmt.Errorf("querying definitions by trigger: %w", err)
}
defer rows.Close()
return scanJobDefinitions(rows)
}
// GetDefinitionByID returns a job definition by ID.
func (r *JobRepository) GetDefinitionByID(ctx context.Context, id string) (*JobDefinitionRecord, error) {
d := &JobDefinitionRecord{}
var defJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, name, version, trigger_type, scope_type, compute_type,
runner_tags, timeout_seconds, max_retries, priority,
definition, enabled, created_at, updated_at
FROM job_definitions WHERE id = $1
`, id).Scan(
&d.ID, &d.Name, &d.Version, &d.TriggerType, &d.ScopeType, &d.ComputeType,
&d.RunnerTags, &d.TimeoutSeconds, &d.MaxRetries, &d.Priority,
&defJSON, &d.Enabled, &d.CreatedAt, &d.UpdatedAt,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("querying job definition by ID: %w", err)
}
if defJSON != nil {
if err := json.Unmarshal(defJSON, &d.Definition); err != nil {
return nil, fmt.Errorf("unmarshaling definition: %w", err)
}
}
return d, nil
}
// ---------------------------------------------------------------------------
// Jobs
// ---------------------------------------------------------------------------
// CreateJob inserts a new job.
func (r *JobRepository) CreateJob(ctx context.Context, j *Job) error {
scopeJSON, err := json.Marshal(j.ScopeMetadata)
if err != nil {
return fmt.Errorf("marshaling scope metadata: %w", err)
}
err = r.db.pool.QueryRow(ctx, `
INSERT INTO jobs (job_definition_id, definition_name, status, priority,
item_id, project_id, scope_metadata,
runner_tags, timeout_seconds, max_retries, created_by)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
RETURNING id, created_at
`, j.JobDefinitionID, j.DefinitionName, "pending", j.Priority,
j.ItemID, j.ProjectID, scopeJSON,
j.RunnerTags, j.TimeoutSeconds, j.MaxRetries, j.CreatedBy,
).Scan(&j.ID, &j.CreatedAt)
if err != nil {
return fmt.Errorf("creating job: %w", err)
}
j.Status = "pending"
return nil
}
// GetJob returns a job by ID.
func (r *JobRepository) GetJob(ctx context.Context, jobID string) (*Job, error) {
j := &Job{}
var scopeJSON, resultJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, job_definition_id, definition_name, status, priority,
item_id, project_id, scope_metadata, runner_id, runner_tags,
created_at, claimed_at, started_at, completed_at,
timeout_seconds, expires_at, progress, progress_message,
result, error_message, retry_count, max_retries,
created_by, cancelled_by
FROM jobs WHERE id = $1
`, jobID).Scan(
&j.ID, &j.JobDefinitionID, &j.DefinitionName, &j.Status, &j.Priority,
&j.ItemID, &j.ProjectID, &scopeJSON, &j.RunnerID, &j.RunnerTags,
&j.CreatedAt, &j.ClaimedAt, &j.StartedAt, &j.CompletedAt,
&j.TimeoutSeconds, &j.ExpiresAt, &j.Progress, &j.ProgressMessage,
&resultJSON, &j.ErrorMessage, &j.RetryCount, &j.MaxRetries,
&j.CreatedBy, &j.CancelledBy,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("querying job: %w", err)
}
if scopeJSON != nil {
if err := json.Unmarshal(scopeJSON, &j.ScopeMetadata); err != nil {
return nil, fmt.Errorf("unmarshaling scope metadata: %w", err)
}
}
if resultJSON != nil {
if err := json.Unmarshal(resultJSON, &j.Result); err != nil {
return nil, fmt.Errorf("unmarshaling result: %w", err)
}
}
return j, nil
}
// ListJobs returns jobs matching optional filters.
func (r *JobRepository) ListJobs(ctx context.Context, status, itemID string, limit, offset int) ([]*Job, error) {
query := `
SELECT id, job_definition_id, definition_name, status, priority,
item_id, project_id, scope_metadata, runner_id, runner_tags,
created_at, claimed_at, started_at, completed_at,
timeout_seconds, expires_at, progress, progress_message,
result, error_message, retry_count, max_retries,
created_by, cancelled_by
FROM jobs WHERE 1=1`
args := []any{}
argN := 1
if status != "" {
query += fmt.Sprintf(" AND status = $%d", argN)
args = append(args, status)
argN++
}
if itemID != "" {
query += fmt.Sprintf(" AND item_id = $%d", argN)
args = append(args, itemID)
argN++
}
query += " ORDER BY created_at DESC"
if limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argN)
args = append(args, limit)
argN++
}
if offset > 0 {
query += fmt.Sprintf(" OFFSET $%d", argN)
args = append(args, offset)
}
rows, err := r.db.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("querying jobs: %w", err)
}
defer rows.Close()
return scanJobs(rows)
}
// ListSolverJobs returns solver jobs (definition_name LIKE 'assembly-%') with optional filters.
func (r *JobRepository) ListSolverJobs(ctx context.Context, status, itemID, operation string, limit, offset int) ([]*Job, error) {
query := `
SELECT id, job_definition_id, definition_name, status, priority,
item_id, project_id, scope_metadata, runner_id, runner_tags,
created_at, claimed_at, started_at, completed_at,
timeout_seconds, expires_at, progress, progress_message,
result, error_message, retry_count, max_retries,
created_by, cancelled_by
FROM jobs WHERE definition_name LIKE 'assembly-%'`
args := []any{}
argN := 1
if status != "" {
query += fmt.Sprintf(" AND status = $%d", argN)
args = append(args, status)
argN++
}
if itemID != "" {
query += fmt.Sprintf(" AND item_id = $%d", argN)
args = append(args, itemID)
argN++
}
if operation != "" {
query += fmt.Sprintf(" AND scope_metadata->>'operation' = $%d", argN)
args = append(args, operation)
argN++
}
query += " ORDER BY created_at DESC"
if limit > 0 {
query += fmt.Sprintf(" LIMIT $%d", argN)
args = append(args, limit)
argN++
}
if offset > 0 {
query += fmt.Sprintf(" OFFSET $%d", argN)
args = append(args, offset)
}
rows, err := r.db.pool.Query(ctx, query, args...)
if err != nil {
return nil, fmt.Errorf("querying solver jobs: %w", err)
}
defer rows.Close()
return scanJobs(rows)
}
// ClaimJob atomically claims the next available job matching the runner's tags.
// Uses SELECT FOR UPDATE SKIP LOCKED for exactly-once delivery.
func (r *JobRepository) ClaimJob(ctx context.Context, runnerID string, tags []string) (*Job, error) {
j := &Job{}
var scopeJSON, resultJSON []byte
err := r.db.pool.QueryRow(ctx, `
WITH claimable AS (
SELECT id FROM jobs
WHERE status = 'pending' AND runner_tags <@ $2::text[]
ORDER BY priority ASC, created_at ASC
LIMIT 1
FOR UPDATE SKIP LOCKED
)
UPDATE jobs SET
status = 'claimed',
runner_id = $1,
claimed_at = now(),
expires_at = now() + (timeout_seconds || ' seconds')::interval
FROM claimable
WHERE jobs.id = claimable.id
RETURNING jobs.id, jobs.job_definition_id, jobs.definition_name, jobs.status,
jobs.priority, jobs.item_id, jobs.project_id, jobs.scope_metadata,
jobs.runner_id, jobs.runner_tags, jobs.created_at, jobs.claimed_at,
jobs.started_at, jobs.completed_at, jobs.timeout_seconds, jobs.expires_at,
jobs.progress, jobs.progress_message, jobs.result, jobs.error_message,
jobs.retry_count, jobs.max_retries, jobs.created_by, jobs.cancelled_by
`, runnerID, tags).Scan(
&j.ID, &j.JobDefinitionID, &j.DefinitionName, &j.Status,
&j.Priority, &j.ItemID, &j.ProjectID, &scopeJSON,
&j.RunnerID, &j.RunnerTags, &j.CreatedAt, &j.ClaimedAt,
&j.StartedAt, &j.CompletedAt, &j.TimeoutSeconds, &j.ExpiresAt,
&j.Progress, &j.ProgressMessage, &resultJSON, &j.ErrorMessage,
&j.RetryCount, &j.MaxRetries, &j.CreatedBy, &j.CancelledBy,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("claiming job: %w", err)
}
if scopeJSON != nil {
if err := json.Unmarshal(scopeJSON, &j.ScopeMetadata); err != nil {
return nil, fmt.Errorf("unmarshaling scope metadata: %w", err)
}
}
if resultJSON != nil {
if err := json.Unmarshal(resultJSON, &j.Result); err != nil {
return nil, fmt.Errorf("unmarshaling result: %w", err)
}
}
return j, nil
}
// StartJob transitions a claimed job to running.
func (r *JobRepository) StartJob(ctx context.Context, jobID, runnerID string) error {
result, err := r.db.pool.Exec(ctx, `
UPDATE jobs SET status = 'running', started_at = now()
WHERE id = $1 AND runner_id = $2 AND status = 'claimed'
`, jobID, runnerID)
if err != nil {
return fmt.Errorf("starting job: %w", err)
}
if result.RowsAffected() == 0 {
return fmt.Errorf("job %s not claimable by runner %s or not in claimed state", jobID, runnerID)
}
return nil
}
// UpdateProgress updates a running job's progress.
func (r *JobRepository) UpdateProgress(ctx context.Context, jobID, runnerID string, progress int, message string) error {
var msg *string
if message != "" {
msg = &message
}
result, err := r.db.pool.Exec(ctx, `
UPDATE jobs SET progress = $3, progress_message = $4
WHERE id = $1 AND runner_id = $2 AND status IN ('claimed', 'running')
`, jobID, runnerID, progress, msg)
if err != nil {
return fmt.Errorf("updating progress: %w", err)
}
if result.RowsAffected() == 0 {
return fmt.Errorf("job %s not owned by runner %s or not active", jobID, runnerID)
}
return nil
}
// CompleteJob marks a job as completed with optional result data.
func (r *JobRepository) CompleteJob(ctx context.Context, jobID, runnerID string, resultData map[string]any) error {
var resultJSON []byte
var err error
if resultData != nil {
resultJSON, err = json.Marshal(resultData)
if err != nil {
return fmt.Errorf("marshaling result: %w", err)
}
}
res, err := r.db.pool.Exec(ctx, `
UPDATE jobs SET
status = 'completed',
progress = 100,
result = $3,
completed_at = now()
WHERE id = $1 AND runner_id = $2 AND status IN ('claimed', 'running')
`, jobID, runnerID, resultJSON)
if err != nil {
return fmt.Errorf("completing job: %w", err)
}
if res.RowsAffected() == 0 {
return fmt.Errorf("job %s not owned by runner %s or not active", jobID, runnerID)
}
return nil
}
// FailJob marks a job as failed with an error message.
func (r *JobRepository) FailJob(ctx context.Context, jobID, runnerID string, errMsg string) error {
res, err := r.db.pool.Exec(ctx, `
UPDATE jobs SET
status = 'failed',
error_message = $3,
completed_at = now()
WHERE id = $1 AND runner_id = $2 AND status IN ('claimed', 'running')
`, jobID, runnerID, errMsg)
if err != nil {
return fmt.Errorf("failing job: %w", err)
}
if res.RowsAffected() == 0 {
return fmt.Errorf("job %s not owned by runner %s or not active", jobID, runnerID)
}
return nil
}
// CancelJob cancels a pending or active job.
func (r *JobRepository) CancelJob(ctx context.Context, jobID string, cancelledBy string) error {
res, err := r.db.pool.Exec(ctx, `
UPDATE jobs SET
status = 'cancelled',
cancelled_by = $2,
completed_at = now()
WHERE id = $1 AND status IN ('pending', 'claimed', 'running')
`, jobID, cancelledBy)
if err != nil {
return fmt.Errorf("cancelling job: %w", err)
}
if res.RowsAffected() == 0 {
return fmt.Errorf("job %s not cancellable", jobID)
}
return nil
}
// TimeoutExpiredJobs marks expired claimed/running jobs as failed.
// Returns the number of jobs timed out.
func (r *JobRepository) TimeoutExpiredJobs(ctx context.Context) (int64, error) {
result, err := r.db.pool.Exec(ctx, `
UPDATE jobs SET
status = 'failed',
error_message = 'job timed out',
completed_at = now()
WHERE status IN ('claimed', 'running')
AND expires_at IS NOT NULL
AND expires_at < now()
`)
if err != nil {
return 0, fmt.Errorf("timing out expired jobs: %w", err)
}
return result.RowsAffected(), nil
}
// ---------------------------------------------------------------------------
// Job Log
// ---------------------------------------------------------------------------
// AppendLog adds a log entry to a job.
func (r *JobRepository) AppendLog(ctx context.Context, entry *JobLogEntry) error {
metaJSON, err := json.Marshal(entry.Metadata)
if err != nil {
return fmt.Errorf("marshaling log metadata: %w", err)
}
err = r.db.pool.QueryRow(ctx, `
INSERT INTO job_log (job_id, level, message, metadata)
VALUES ($1, $2, $3, $4)
RETURNING id, timestamp
`, entry.JobID, entry.Level, entry.Message, metaJSON,
).Scan(&entry.ID, &entry.Timestamp)
if err != nil {
return fmt.Errorf("appending job log: %w", err)
}
return nil
}
// GetJobLogs returns all log entries for a job.
func (r *JobRepository) GetJobLogs(ctx context.Context, jobID string) ([]*JobLogEntry, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, job_id, timestamp, level, message, metadata
FROM job_log WHERE job_id = $1 ORDER BY timestamp ASC
`, jobID)
if err != nil {
return nil, fmt.Errorf("querying job logs: %w", err)
}
defer rows.Close()
var entries []*JobLogEntry
for rows.Next() {
e := &JobLogEntry{}
var metaJSON []byte
if err := rows.Scan(&e.ID, &e.JobID, &e.Timestamp, &e.Level, &e.Message, &metaJSON); err != nil {
return nil, fmt.Errorf("scanning job log: %w", err)
}
if metaJSON != nil {
if err := json.Unmarshal(metaJSON, &e.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling log metadata: %w", err)
}
}
entries = append(entries, e)
}
return entries, rows.Err()
}
// ---------------------------------------------------------------------------
// Runners
// ---------------------------------------------------------------------------
// RegisterRunner creates a new runner record.
func (r *JobRepository) RegisterRunner(ctx context.Context, runner *Runner) error {
metaJSON, err := json.Marshal(runner.Metadata)
if err != nil {
return fmt.Errorf("marshaling runner metadata: %w", err)
}
err = r.db.pool.QueryRow(ctx, `
INSERT INTO runners (name, token_hash, token_prefix, tags, status, metadata)
VALUES ($1, $2, $3, $4, 'offline', $5)
RETURNING id, created_at, updated_at
`, runner.Name, runner.TokenHash, runner.TokenPrefix, runner.Tags, metaJSON,
).Scan(&runner.ID, &runner.CreatedAt, &runner.UpdatedAt)
if err != nil {
return fmt.Errorf("registering runner: %w", err)
}
runner.Status = "offline"
return nil
}
// GetRunnerByToken looks up a runner by token hash.
func (r *JobRepository) GetRunnerByToken(ctx context.Context, tokenHash string) (*Runner, error) {
runner := &Runner{}
var metaJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, name, token_hash, token_prefix, tags, status,
last_heartbeat, last_job_id, metadata, created_at, updated_at
FROM runners WHERE token_hash = $1
`, tokenHash).Scan(
&runner.ID, &runner.Name, &runner.TokenHash, &runner.TokenPrefix,
&runner.Tags, &runner.Status, &runner.LastHeartbeat, &runner.LastJobID,
&metaJSON, &runner.CreatedAt, &runner.UpdatedAt,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("querying runner by token: %w", err)
}
if metaJSON != nil {
if err := json.Unmarshal(metaJSON, &runner.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling runner metadata: %w", err)
}
}
return runner, nil
}
// GetRunner returns a runner by ID.
func (r *JobRepository) GetRunner(ctx context.Context, runnerID string) (*Runner, error) {
runner := &Runner{}
var metaJSON []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, name, token_hash, token_prefix, tags, status,
last_heartbeat, last_job_id, metadata, created_at, updated_at
FROM runners WHERE id = $1
`, runnerID).Scan(
&runner.ID, &runner.Name, &runner.TokenHash, &runner.TokenPrefix,
&runner.Tags, &runner.Status, &runner.LastHeartbeat, &runner.LastJobID,
&metaJSON, &runner.CreatedAt, &runner.UpdatedAt,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("querying runner: %w", err)
}
if metaJSON != nil {
if err := json.Unmarshal(metaJSON, &runner.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling runner metadata: %w", err)
}
}
return runner, nil
}
// Heartbeat updates a runner's heartbeat timestamp and sets status to online.
func (r *JobRepository) Heartbeat(ctx context.Context, runnerID string) error {
res, err := r.db.pool.Exec(ctx, `
UPDATE runners SET
status = 'online',
last_heartbeat = now(),
updated_at = now()
WHERE id = $1
`, runnerID)
if err != nil {
return fmt.Errorf("updating heartbeat: %w", err)
}
if res.RowsAffected() == 0 {
return fmt.Errorf("runner %s not found", runnerID)
}
return nil
}
// ListRunners returns all registered runners.
func (r *JobRepository) ListRunners(ctx context.Context) ([]*Runner, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, name, token_hash, token_prefix, tags, status,
last_heartbeat, last_job_id, metadata, created_at, updated_at
FROM runners ORDER BY name
`)
if err != nil {
return nil, fmt.Errorf("querying runners: %w", err)
}
defer rows.Close()
var runners []*Runner
for rows.Next() {
runner := &Runner{}
var metaJSON []byte
if err := rows.Scan(
&runner.ID, &runner.Name, &runner.TokenHash, &runner.TokenPrefix,
&runner.Tags, &runner.Status, &runner.LastHeartbeat, &runner.LastJobID,
&metaJSON, &runner.CreatedAt, &runner.UpdatedAt,
); err != nil {
return nil, fmt.Errorf("scanning runner: %w", err)
}
if metaJSON != nil {
if err := json.Unmarshal(metaJSON, &runner.Metadata); err != nil {
return nil, fmt.Errorf("unmarshaling runner metadata: %w", err)
}
}
runners = append(runners, runner)
}
return runners, rows.Err()
}
// DeleteRunner removes a runner by ID.
func (r *JobRepository) DeleteRunner(ctx context.Context, runnerID string) error {
res, err := r.db.pool.Exec(ctx, `DELETE FROM runners WHERE id = $1`, runnerID)
if err != nil {
return fmt.Errorf("deleting runner: %w", err)
}
if res.RowsAffected() == 0 {
return fmt.Errorf("runner %s not found", runnerID)
}
return nil
}
// ExpireStaleRunners marks runners with no recent heartbeat as offline.
func (r *JobRepository) ExpireStaleRunners(ctx context.Context, timeout time.Duration) (int64, error) {
result, err := r.db.pool.Exec(ctx, `
UPDATE runners SET status = 'offline', updated_at = now()
WHERE status = 'online'
AND last_heartbeat < now() - $1::interval
`, timeout.String())
if err != nil {
return 0, fmt.Errorf("expiring stale runners: %w", err)
}
return result.RowsAffected(), nil
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
func scanJobDefinitions(rows pgx.Rows) ([]*JobDefinitionRecord, error) {
var defs []*JobDefinitionRecord
for rows.Next() {
d := &JobDefinitionRecord{}
var defJSON []byte
if err := rows.Scan(
&d.ID, &d.Name, &d.Version, &d.TriggerType, &d.ScopeType, &d.ComputeType,
&d.RunnerTags, &d.TimeoutSeconds, &d.MaxRetries, &d.Priority,
&defJSON, &d.Enabled, &d.CreatedAt, &d.UpdatedAt,
); err != nil {
return nil, fmt.Errorf("scanning job definition: %w", err)
}
if defJSON != nil {
if err := json.Unmarshal(defJSON, &d.Definition); err != nil {
return nil, fmt.Errorf("unmarshaling definition: %w", err)
}
}
defs = append(defs, d)
}
return defs, rows.Err()
}
func scanJobs(rows pgx.Rows) ([]*Job, error) {
var jobs []*Job
for rows.Next() {
j := &Job{}
var scopeJSON, resultJSON []byte
if err := rows.Scan(
&j.ID, &j.JobDefinitionID, &j.DefinitionName, &j.Status, &j.Priority,
&j.ItemID, &j.ProjectID, &scopeJSON, &j.RunnerID, &j.RunnerTags,
&j.CreatedAt, &j.ClaimedAt, &j.StartedAt, &j.CompletedAt,
&j.TimeoutSeconds, &j.ExpiresAt, &j.Progress, &j.ProgressMessage,
&resultJSON, &j.ErrorMessage, &j.RetryCount, &j.MaxRetries,
&j.CreatedBy, &j.CancelledBy,
); err != nil {
return nil, fmt.Errorf("scanning job: %w", err)
}
if scopeJSON != nil {
if err := json.Unmarshal(scopeJSON, &j.ScopeMetadata); err != nil {
return nil, fmt.Errorf("unmarshaling scope metadata: %w", err)
}
}
if resultJSON != nil {
if err := json.Unmarshal(resultJSON, &j.Result); err != nil {
return nil, fmt.Errorf("unmarshaling result: %w", err)
}
}
jobs = append(jobs, j)
}
return jobs, rows.Err()
}

230
internal/db/locations.go Normal file
View File

@@ -0,0 +1,230 @@
package db
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/jackc/pgx/v5"
)
// Location represents a location in the hierarchy.
type Location struct {
ID string
Path string
Name string
ParentID *string
LocationType string
Depth int
Metadata map[string]any
CreatedAt time.Time
}
// LocationRepository provides location database operations.
type LocationRepository struct {
db *DB
}
// NewLocationRepository creates a new location repository.
func NewLocationRepository(db *DB) *LocationRepository {
return &LocationRepository{db: db}
}
// List returns all locations ordered by path.
func (r *LocationRepository) List(ctx context.Context) ([]*Location, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, path, name, parent_id, location_type, depth, metadata, created_at
FROM locations
ORDER BY path
`)
if err != nil {
return nil, err
}
defer rows.Close()
return scanLocations(rows)
}
// GetByPath returns a location by its path.
func (r *LocationRepository) GetByPath(ctx context.Context, path string) (*Location, error) {
loc := &Location{}
var meta []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, path, name, parent_id, location_type, depth, metadata, created_at
FROM locations
WHERE path = $1
`, path).Scan(&loc.ID, &loc.Path, &loc.Name, &loc.ParentID, &loc.LocationType, &loc.Depth, &meta, &loc.CreatedAt)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
if meta != nil {
json.Unmarshal(meta, &loc.Metadata)
}
return loc, nil
}
// GetByID returns a location by its ID.
func (r *LocationRepository) GetByID(ctx context.Context, id string) (*Location, error) {
loc := &Location{}
var meta []byte
err := r.db.pool.QueryRow(ctx, `
SELECT id, path, name, parent_id, location_type, depth, metadata, created_at
FROM locations
WHERE id = $1
`, id).Scan(&loc.ID, &loc.Path, &loc.Name, &loc.ParentID, &loc.LocationType, &loc.Depth, &meta, &loc.CreatedAt)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
if meta != nil {
json.Unmarshal(meta, &loc.Metadata)
}
return loc, nil
}
// GetChildren returns direct children of a location.
func (r *LocationRepository) GetChildren(ctx context.Context, parentID string) ([]*Location, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, path, name, parent_id, location_type, depth, metadata, created_at
FROM locations
WHERE parent_id = $1
ORDER BY path
`, parentID)
if err != nil {
return nil, err
}
defer rows.Close()
return scanLocations(rows)
}
// GetTree returns a location and all its descendants (by path prefix).
func (r *LocationRepository) GetTree(ctx context.Context, rootPath string) ([]*Location, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, path, name, parent_id, location_type, depth, metadata, created_at
FROM locations
WHERE path = $1 OR path LIKE $2
ORDER BY path
`, rootPath, rootPath+"/%")
if err != nil {
return nil, err
}
defer rows.Close()
return scanLocations(rows)
}
// Create inserts a new location. ParentID and Depth are resolved from the path.
func (r *LocationRepository) Create(ctx context.Context, loc *Location) error {
// Auto-calculate depth from path segments
loc.Depth = strings.Count(loc.Path, "/")
// Resolve parent_id from path if not explicitly set
if loc.ParentID == nil && loc.Depth > 0 {
parentPath := loc.Path[:strings.LastIndex(loc.Path, "/")]
parent, err := r.GetByPath(ctx, parentPath)
if err != nil {
return fmt.Errorf("looking up parent %q: %w", parentPath, err)
}
if parent == nil {
return fmt.Errorf("parent location %q does not exist", parentPath)
}
loc.ParentID = &parent.ID
}
meta, err := json.Marshal(loc.Metadata)
if err != nil {
return fmt.Errorf("marshaling metadata: %w", err)
}
return r.db.pool.QueryRow(ctx, `
INSERT INTO locations (path, name, parent_id, location_type, depth, metadata)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING id, created_at
`, loc.Path, loc.Name, loc.ParentID, loc.LocationType, loc.Depth, meta).Scan(&loc.ID, &loc.CreatedAt)
}
// Update updates a location's name, type, and metadata.
func (r *LocationRepository) Update(ctx context.Context, path string, name, locationType string, metadata map[string]any) error {
meta, err := json.Marshal(metadata)
if err != nil {
return fmt.Errorf("marshaling metadata: %w", err)
}
tag, err := r.db.pool.Exec(ctx, `
UPDATE locations
SET name = $2, location_type = $3, metadata = $4
WHERE path = $1
`, path, name, locationType, meta)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("location %q not found", path)
}
return nil
}
// Delete removes a location. Returns an error if inventory rows reference it.
func (r *LocationRepository) Delete(ctx context.Context, path string) error {
// Check for inventory references
var count int
err := r.db.pool.QueryRow(ctx, `
SELECT COUNT(*) FROM inventory
WHERE location_id = (SELECT id FROM locations WHERE path = $1)
`, path).Scan(&count)
if err != nil {
return err
}
if count > 0 {
return fmt.Errorf("cannot delete location %q: %d inventory record(s) exist", path, count)
}
// Delete children first (cascade by path prefix), deepest first
_, err = r.db.pool.Exec(ctx, `
DELETE FROM locations
WHERE path LIKE $1
`, path+"/%")
if err != nil {
return err
}
tag, err := r.db.pool.Exec(ctx, `DELETE FROM locations WHERE path = $1`, path)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("location %q not found", path)
}
return nil
}
// HasInventory checks if a location (or descendants) have inventory records.
func (r *LocationRepository) HasInventory(ctx context.Context, path string) (bool, error) {
var count int
err := r.db.pool.QueryRow(ctx, `
SELECT COUNT(*) FROM inventory i
JOIN locations l ON l.id = i.location_id
WHERE l.path = $1 OR l.path LIKE $2
`, path, path+"/%").Scan(&count)
return count > 0, err
}
func scanLocations(rows pgx.Rows) ([]*Location, error) {
var locs []*Location
for rows.Next() {
loc := &Location{}
var meta []byte
if err := rows.Scan(&loc.ID, &loc.Path, &loc.Name, &loc.ParentID, &loc.LocationType, &loc.Depth, &meta, &loc.CreatedAt); err != nil {
return nil, err
}
if meta != nil {
json.Unmarshal(meta, &loc.Metadata)
}
locs = append(locs, loc)
}
return locs, rows.Err()
}

View File

@@ -0,0 +1,11 @@
-- 022_workstations.sql — workstation identity for edit sessions
CREATE TABLE workstations (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name TEXT NOT NULL,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
hostname TEXT NOT NULL DEFAULT '',
last_seen TIMESTAMPTZ NOT NULL DEFAULT now(),
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE(user_id, name)
);

View File

@@ -0,0 +1,17 @@
-- 023_edit_sessions.sql — active editing context tracking
CREATE TABLE edit_sessions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
workstation_id UUID NOT NULL REFERENCES workstations(id) ON DELETE CASCADE,
context_level TEXT NOT NULL CHECK (context_level IN ('sketch', 'partdesign', 'assembly')),
object_id TEXT,
dependency_cone TEXT[],
acquired_at TIMESTAMPTZ NOT NULL DEFAULT now(),
last_heartbeat TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX idx_edit_sessions_item ON edit_sessions(item_id);
CREATE INDEX idx_edit_sessions_user ON edit_sessions(user_id);
CREATE UNIQUE INDEX idx_edit_sessions_active ON edit_sessions(item_id, context_level, object_id);

View File

@@ -240,7 +240,7 @@ func (r *ProjectRepository) GetItemsForProject(ctx context.Context, projectID st
SELECT i.id, i.part_number, i.schema_id, i.item_type, i.description,
i.created_at, i.updated_at, i.archived_at, i.current_revision,
i.cad_synced_at, i.cad_file_path,
i.sourcing_type, i.sourcing_link, i.long_description, i.standard_cost,
i.sourcing_type, i.long_description,
i.thumbnail_key
FROM items i
JOIN item_projects ip ON ip.item_id = i.id
@@ -259,7 +259,7 @@ func (r *ProjectRepository) GetItemsForProject(ctx context.Context, projectID st
&item.ID, &item.PartNumber, &item.SchemaID, &item.ItemType, &item.Description,
&item.CreatedAt, &item.UpdatedAt, &item.ArchivedAt, &item.CurrentRevision,
&item.CADSyncedAt, &item.CADFilePath,
&item.SourcingType, &item.SourcingLink, &item.LongDescription, &item.StandardCost,
&item.SourcingType, &item.LongDescription,
&item.ThumbnailKey,
); err != nil {
return nil, err

105
internal/db/settings.go Normal file
View File

@@ -0,0 +1,105 @@
package db
import (
"context"
"encoding/json"
"fmt"
)
// SettingsRepository provides access to module_state and settings_overrides tables.
type SettingsRepository struct {
db *DB
}
// NewSettingsRepository creates a new SettingsRepository.
func NewSettingsRepository(db *DB) *SettingsRepository {
return &SettingsRepository{db: db}
}
// GetModuleStates returns all module enabled/disabled states from the database.
func (r *SettingsRepository) GetModuleStates(ctx context.Context) (map[string]bool, error) {
rows, err := r.db.pool.Query(ctx,
`SELECT module_id, enabled FROM module_state`)
if err != nil {
return nil, fmt.Errorf("querying module states: %w", err)
}
defer rows.Close()
states := make(map[string]bool)
for rows.Next() {
var id string
var enabled bool
if err := rows.Scan(&id, &enabled); err != nil {
return nil, fmt.Errorf("scanning module state: %w", err)
}
states[id] = enabled
}
return states, rows.Err()
}
// SetModuleState persists a module's enabled state. Uses upsert semantics.
func (r *SettingsRepository) SetModuleState(ctx context.Context, moduleID string, enabled bool, updatedBy string) error {
_, err := r.db.pool.Exec(ctx,
`INSERT INTO module_state (module_id, enabled, updated_by, updated_at)
VALUES ($1, $2, $3, now())
ON CONFLICT (module_id) DO UPDATE
SET enabled = EXCLUDED.enabled,
updated_by = EXCLUDED.updated_by,
updated_at = now()`,
moduleID, enabled, updatedBy)
if err != nil {
return fmt.Errorf("setting module state: %w", err)
}
return nil
}
// GetOverrides returns all settings overrides from the database.
func (r *SettingsRepository) GetOverrides(ctx context.Context) (map[string]json.RawMessage, error) {
rows, err := r.db.pool.Query(ctx,
`SELECT key, value FROM settings_overrides`)
if err != nil {
return nil, fmt.Errorf("querying settings overrides: %w", err)
}
defer rows.Close()
overrides := make(map[string]json.RawMessage)
for rows.Next() {
var key string
var value json.RawMessage
if err := rows.Scan(&key, &value); err != nil {
return nil, fmt.Errorf("scanning settings override: %w", err)
}
overrides[key] = value
}
return overrides, rows.Err()
}
// SetOverride persists a settings override. Uses upsert semantics.
func (r *SettingsRepository) SetOverride(ctx context.Context, key string, value any, updatedBy string) error {
jsonVal, err := json.Marshal(value)
if err != nil {
return fmt.Errorf("marshaling override value: %w", err)
}
_, err = r.db.pool.Exec(ctx,
`INSERT INTO settings_overrides (key, value, updated_by, updated_at)
VALUES ($1, $2, $3, now())
ON CONFLICT (key) DO UPDATE
SET value = EXCLUDED.value,
updated_by = EXCLUDED.updated_by,
updated_at = now()`,
key, jsonVal, updatedBy)
if err != nil {
return fmt.Errorf("setting override: %w", err)
}
return nil
}
// DeleteOverride removes a settings override.
func (r *SettingsRepository) DeleteOverride(ctx context.Context, key string) error {
_, err := r.db.pool.Exec(ctx,
`DELETE FROM settings_overrides WHERE key = $1`, key)
if err != nil {
return fmt.Errorf("deleting override: %w", err)
}
return nil
}

View File

@@ -0,0 +1,121 @@
package db
import (
"context"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
// SolverResult represents a row in the solver_results table.
type SolverResult struct {
ID string
ItemID string
RevisionNumber int
JobID *string
Operation string // solve, diagnose, kinematic
SolverName string
Status string // SolveStatus string (Success, Failed, etc.)
DOF *int
Diagnostics []byte // raw JSONB
Placements []byte // raw JSONB
NumFrames int
SolveTimeMS *float64
CreatedAt time.Time
}
// SolverResultRepository provides solver_results database operations.
type SolverResultRepository struct {
db *DB
}
// NewSolverResultRepository creates a new solver result repository.
func NewSolverResultRepository(db *DB) *SolverResultRepository {
return &SolverResultRepository{db: db}
}
// Upsert inserts or updates a solver result. The UNIQUE(item_id, revision_number, operation)
// constraint means each revision has at most one result per operation type.
func (r *SolverResultRepository) Upsert(ctx context.Context, s *SolverResult) error {
err := r.db.pool.QueryRow(ctx, `
INSERT INTO solver_results (item_id, revision_number, job_id, operation,
solver_name, status, dof, diagnostics, placements,
num_frames, solve_time_ms)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11)
ON CONFLICT (item_id, revision_number, operation) DO UPDATE SET
job_id = EXCLUDED.job_id,
solver_name = EXCLUDED.solver_name,
status = EXCLUDED.status,
dof = EXCLUDED.dof,
diagnostics = EXCLUDED.diagnostics,
placements = EXCLUDED.placements,
num_frames = EXCLUDED.num_frames,
solve_time_ms = EXCLUDED.solve_time_ms,
created_at = now()
RETURNING id, created_at
`, s.ItemID, s.RevisionNumber, s.JobID, s.Operation,
s.SolverName, s.Status, s.DOF, s.Diagnostics, s.Placements,
s.NumFrames, s.SolveTimeMS,
).Scan(&s.ID, &s.CreatedAt)
if err != nil {
return fmt.Errorf("upserting solver result: %w", err)
}
return nil
}
// GetByItem returns all solver results for an item, ordered by revision descending.
func (r *SolverResultRepository) GetByItem(ctx context.Context, itemID string) ([]*SolverResult, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, item_id, revision_number, job_id, operation,
solver_name, status, dof, diagnostics, placements,
num_frames, solve_time_ms, created_at
FROM solver_results
WHERE item_id = $1
ORDER BY revision_number DESC, operation
`, itemID)
if err != nil {
return nil, fmt.Errorf("listing solver results: %w", err)
}
defer rows.Close()
return scanSolverResults(rows)
}
// GetByItemRevision returns a single solver result for an item/revision/operation.
func (r *SolverResultRepository) GetByItemRevision(ctx context.Context, itemID string, revision int, operation string) (*SolverResult, error) {
s := &SolverResult{}
err := r.db.pool.QueryRow(ctx, `
SELECT id, item_id, revision_number, job_id, operation,
solver_name, status, dof, diagnostics, placements,
num_frames, solve_time_ms, created_at
FROM solver_results
WHERE item_id = $1 AND revision_number = $2 AND operation = $3
`, itemID, revision, operation).Scan(
&s.ID, &s.ItemID, &s.RevisionNumber, &s.JobID, &s.Operation,
&s.SolverName, &s.Status, &s.DOF, &s.Diagnostics, &s.Placements,
&s.NumFrames, &s.SolveTimeMS, &s.CreatedAt,
)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, fmt.Errorf("getting solver result: %w", err)
}
return s, nil
}
func scanSolverResults(rows pgx.Rows) ([]*SolverResult, error) {
var results []*SolverResult
for rows.Next() {
s := &SolverResult{}
if err := rows.Scan(
&s.ID, &s.ItemID, &s.RevisionNumber, &s.JobID, &s.Operation,
&s.SolverName, &s.Status, &s.DOF, &s.Diagnostics, &s.Placements,
&s.NumFrames, &s.SolveTimeMS, &s.CreatedAt,
); err != nil {
return nil, fmt.Errorf("scanning solver result: %w", err)
}
results = append(results, s)
}
return results, rows.Err()
}

View File

@@ -0,0 +1,95 @@
package db
import (
"context"
"time"
"github.com/jackc/pgx/v5"
)
// Workstation represents a registered client machine.
type Workstation struct {
ID string
Name string
UserID string
Hostname string
LastSeen time.Time
CreatedAt time.Time
}
// WorkstationRepository provides workstation database operations.
type WorkstationRepository struct {
db *DB
}
// NewWorkstationRepository creates a new workstation repository.
func NewWorkstationRepository(db *DB) *WorkstationRepository {
return &WorkstationRepository{db: db}
}
// Upsert registers a workstation, updating hostname and last_seen if it already exists.
func (r *WorkstationRepository) Upsert(ctx context.Context, w *Workstation) error {
return r.db.pool.QueryRow(ctx, `
INSERT INTO workstations (name, user_id, hostname)
VALUES ($1, $2, $3)
ON CONFLICT (user_id, name) DO UPDATE
SET hostname = EXCLUDED.hostname, last_seen = now()
RETURNING id, last_seen, created_at
`, w.Name, w.UserID, w.Hostname).Scan(&w.ID, &w.LastSeen, &w.CreatedAt)
}
// GetByID returns a workstation by its ID.
func (r *WorkstationRepository) GetByID(ctx context.Context, id string) (*Workstation, error) {
w := &Workstation{}
err := r.db.pool.QueryRow(ctx, `
SELECT id, name, user_id, hostname, last_seen, created_at
FROM workstations
WHERE id = $1
`, id).Scan(&w.ID, &w.Name, &w.UserID, &w.Hostname, &w.LastSeen, &w.CreatedAt)
if err == pgx.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return w, nil
}
// ListByUser returns all workstations for a user.
func (r *WorkstationRepository) ListByUser(ctx context.Context, userID string) ([]*Workstation, error) {
rows, err := r.db.pool.Query(ctx, `
SELECT id, name, user_id, hostname, last_seen, created_at
FROM workstations
WHERE user_id = $1
ORDER BY name
`, userID)
if err != nil {
return nil, err
}
defer rows.Close()
var workstations []*Workstation
for rows.Next() {
w := &Workstation{}
if err := rows.Scan(&w.ID, &w.Name, &w.UserID, &w.Hostname, &w.LastSeen, &w.CreatedAt); err != nil {
return nil, err
}
workstations = append(workstations, w)
}
return workstations, rows.Err()
}
// Touch updates a workstation's last_seen timestamp.
func (r *WorkstationRepository) Touch(ctx context.Context, id string) error {
_, err := r.db.pool.Exec(ctx, `
UPDATE workstations SET last_seen = now() WHERE id = $1
`, id)
return err
}
// Delete removes a workstation.
func (r *WorkstationRepository) Delete(ctx context.Context, id string) error {
_, err := r.db.pool.Exec(ctx, `DELETE FROM workstations WHERE id = $1`, id)
return err
}

166
internal/jobdef/jobdef.go Normal file
View File

@@ -0,0 +1,166 @@
// Package jobdef handles YAML job definition parsing and validation.
package jobdef
import (
"fmt"
"os"
"path/filepath"
"strings"
"gopkg.in/yaml.v3"
)
// Definition represents a compute job definition loaded from YAML.
type Definition struct {
Name string `yaml:"name" json:"name"`
Version int `yaml:"version" json:"version"`
Description string `yaml:"description" json:"description"`
Trigger TriggerConfig `yaml:"trigger" json:"trigger"`
Scope ScopeConfig `yaml:"scope" json:"scope"`
Compute ComputeConfig `yaml:"compute" json:"compute"`
Runner RunnerConfig `yaml:"runner" json:"runner"`
Timeout int `yaml:"timeout" json:"timeout"`
MaxRetries int `yaml:"max_retries" json:"max_retries"`
Priority int `yaml:"priority" json:"priority"`
}
// TriggerConfig describes when a job is created.
type TriggerConfig struct {
Type string `yaml:"type" json:"type"`
Filter map[string]string `yaml:"filter,omitempty" json:"filter,omitempty"`
}
// ScopeConfig describes what a job operates on.
type ScopeConfig struct {
Type string `yaml:"type" json:"type"`
}
// ComputeConfig describes the computation to perform.
type ComputeConfig struct {
Type string `yaml:"type" json:"type"`
Command string `yaml:"command" json:"command"`
Args map[string]any `yaml:"args,omitempty" json:"args,omitempty"`
}
// RunnerConfig describes runner requirements.
type RunnerConfig struct {
Tags []string `yaml:"tags" json:"tags"`
}
// DefinitionFile wraps a definition for YAML parsing.
type DefinitionFile struct {
Job Definition `yaml:"job"`
}
var validTriggerTypes = map[string]bool{
"revision_created": true,
"bom_changed": true,
"manual": true,
"schedule": true,
}
var validScopeTypes = map[string]bool{
"item": true,
"assembly": true,
"project": true,
}
var validComputeTypes = map[string]bool{
"validate": true,
"rebuild": true,
"diff": true,
"export": true,
"custom": true,
}
// Load reads a job definition from a YAML file.
func Load(path string) (*Definition, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("reading job definition file: %w", err)
}
var df DefinitionFile
if err := yaml.Unmarshal(data, &df); err != nil {
return nil, fmt.Errorf("parsing job definition YAML: %w", err)
}
def := &df.Job
// Apply defaults
if def.Timeout <= 0 {
def.Timeout = 600
}
if def.MaxRetries <= 0 {
def.MaxRetries = 1
}
if def.Priority <= 0 {
def.Priority = 100
}
if def.Version <= 0 {
def.Version = 1
}
if err := def.Validate(); err != nil {
return nil, fmt.Errorf("validating %s: %w", path, err)
}
return def, nil
}
// LoadAll reads all job definitions from a directory.
func LoadAll(dir string) (map[string]*Definition, error) {
defs := make(map[string]*Definition)
entries, err := os.ReadDir(dir)
if err != nil {
return nil, fmt.Errorf("reading job definitions directory: %w", err)
}
for _, entry := range entries {
if entry.IsDir() {
continue
}
if !strings.HasSuffix(entry.Name(), ".yaml") && !strings.HasSuffix(entry.Name(), ".yml") {
continue
}
path := filepath.Join(dir, entry.Name())
def, err := Load(path)
if err != nil {
return nil, fmt.Errorf("loading %s: %w", entry.Name(), err)
}
defs[def.Name] = def
}
return defs, nil
}
// Validate checks that the definition is well-formed.
func (d *Definition) Validate() error {
if d.Name == "" {
return fmt.Errorf("job definition name is required")
}
if d.Trigger.Type == "" {
return fmt.Errorf("trigger type is required")
}
if !validTriggerTypes[d.Trigger.Type] {
return fmt.Errorf("invalid trigger type %q", d.Trigger.Type)
}
if d.Scope.Type == "" {
return fmt.Errorf("scope type is required")
}
if !validScopeTypes[d.Scope.Type] {
return fmt.Errorf("invalid scope type %q", d.Scope.Type)
}
if d.Compute.Type == "" {
return fmt.Errorf("compute type is required")
}
if !validComputeTypes[d.Compute.Type] {
return fmt.Errorf("invalid compute type %q", d.Compute.Type)
}
if d.Compute.Command == "" {
return fmt.Errorf("compute command is required")
}
return nil
}

View File

@@ -0,0 +1,328 @@
package jobdef
import (
"os"
"path/filepath"
"testing"
)
func TestLoadValid(t *testing.T) {
dir := t.TempDir()
content := `
job:
name: test-job
version: 1
description: "A test job"
trigger:
type: manual
scope:
type: item
compute:
type: validate
command: create-validate
runner:
tags: [create]
timeout: 300
max_retries: 2
priority: 50
`
path := filepath.Join(dir, "test-job.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("writing test file: %v", err)
}
def, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
if def.Name != "test-job" {
t.Errorf("name = %q, want %q", def.Name, "test-job")
}
if def.Version != 1 {
t.Errorf("version = %d, want 1", def.Version)
}
if def.Trigger.Type != "manual" {
t.Errorf("trigger type = %q, want %q", def.Trigger.Type, "manual")
}
if def.Scope.Type != "item" {
t.Errorf("scope type = %q, want %q", def.Scope.Type, "item")
}
if def.Compute.Type != "validate" {
t.Errorf("compute type = %q, want %q", def.Compute.Type, "validate")
}
if def.Compute.Command != "create-validate" {
t.Errorf("compute command = %q, want %q", def.Compute.Command, "create-validate")
}
if len(def.Runner.Tags) != 1 || def.Runner.Tags[0] != "create" {
t.Errorf("runner tags = %v, want [create]", def.Runner.Tags)
}
if def.Timeout != 300 {
t.Errorf("timeout = %d, want 300", def.Timeout)
}
if def.MaxRetries != 2 {
t.Errorf("max_retries = %d, want 2", def.MaxRetries)
}
if def.Priority != 50 {
t.Errorf("priority = %d, want 50", def.Priority)
}
}
func TestLoadDefaults(t *testing.T) {
dir := t.TempDir()
content := `
job:
name: minimal
trigger:
type: manual
scope:
type: item
compute:
type: custom
command: do-something
`
path := filepath.Join(dir, "minimal.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("writing test file: %v", err)
}
def, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
if def.Timeout != 600 {
t.Errorf("default timeout = %d, want 600", def.Timeout)
}
if def.MaxRetries != 1 {
t.Errorf("default max_retries = %d, want 1", def.MaxRetries)
}
if def.Priority != 100 {
t.Errorf("default priority = %d, want 100", def.Priority)
}
if def.Version != 1 {
t.Errorf("default version = %d, want 1", def.Version)
}
}
func TestLoadInvalidTriggerType(t *testing.T) {
dir := t.TempDir()
content := `
job:
name: bad-trigger
trigger:
type: invalid_trigger
scope:
type: item
compute:
type: validate
command: create-validate
`
path := filepath.Join(dir, "bad.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("writing test file: %v", err)
}
_, err := Load(path)
if err == nil {
t.Fatal("expected error for invalid trigger type")
}
}
func TestLoadMissingName(t *testing.T) {
dir := t.TempDir()
content := `
job:
trigger:
type: manual
scope:
type: item
compute:
type: validate
command: create-validate
`
path := filepath.Join(dir, "no-name.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("writing test file: %v", err)
}
_, err := Load(path)
if err == nil {
t.Fatal("expected error for missing name")
}
}
func TestLoadMissingCommand(t *testing.T) {
dir := t.TempDir()
content := `
job:
name: no-command
trigger:
type: manual
scope:
type: item
compute:
type: validate
`
path := filepath.Join(dir, "no-cmd.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("writing test file: %v", err)
}
_, err := Load(path)
if err == nil {
t.Fatal("expected error for missing command")
}
}
func TestLoadAllDirectory(t *testing.T) {
dir := t.TempDir()
job1 := `
job:
name: job-one
trigger:
type: manual
scope:
type: item
compute:
type: validate
command: create-validate
`
job2 := `
job:
name: job-two
trigger:
type: revision_created
scope:
type: assembly
compute:
type: export
command: create-export
`
if err := os.WriteFile(filepath.Join(dir, "one.yaml"), []byte(job1), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "two.yml"), []byte(job2), 0644); err != nil {
t.Fatal(err)
}
// Non-YAML file should be ignored
if err := os.WriteFile(filepath.Join(dir, "readme.txt"), []byte("ignore me"), 0644); err != nil {
t.Fatal(err)
}
defs, err := LoadAll(dir)
if err != nil {
t.Fatalf("LoadAll: %v", err)
}
if len(defs) != 2 {
t.Fatalf("loaded %d definitions, want 2", len(defs))
}
if _, ok := defs["job-one"]; !ok {
t.Error("job-one not found")
}
if _, ok := defs["job-two"]; !ok {
t.Error("job-two not found")
}
}
func TestLoadAllEmptyDirectory(t *testing.T) {
dir := t.TempDir()
defs, err := LoadAll(dir)
if err != nil {
t.Fatalf("LoadAll: %v", err)
}
if len(defs) != 0 {
t.Errorf("loaded %d definitions from empty dir, want 0", len(defs))
}
}
func TestLoadWithFilter(t *testing.T) {
dir := t.TempDir()
content := `
job:
name: filtered-job
trigger:
type: revision_created
filter:
item_type: assembly
scope:
type: assembly
compute:
type: validate
command: create-validate
`
path := filepath.Join(dir, "filtered.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("writing test file: %v", err)
}
def, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
if def.Trigger.Filter["item_type"] != "assembly" {
t.Errorf("filter item_type = %q, want %q", def.Trigger.Filter["item_type"], "assembly")
}
}
func TestLoadWithArgs(t *testing.T) {
dir := t.TempDir()
content := `
job:
name: args-job
trigger:
type: manual
scope:
type: item
compute:
type: export
command: create-export
args:
format: step
include_mesh: true
`
path := filepath.Join(dir, "args.yaml")
if err := os.WriteFile(path, []byte(content), 0644); err != nil {
t.Fatalf("writing test file: %v", err)
}
def, err := Load(path)
if err != nil {
t.Fatalf("Load: %v", err)
}
if def.Compute.Args["format"] != "step" {
t.Errorf("args format = %v, want %q", def.Compute.Args["format"], "step")
}
if def.Compute.Args["include_mesh"] != true {
t.Errorf("args include_mesh = %v, want true", def.Compute.Args["include_mesh"])
}
}
func TestValidateInvalidScopeType(t *testing.T) {
d := &Definition{
Name: "test",
Trigger: TriggerConfig{Type: "manual"},
Scope: ScopeConfig{Type: "galaxy"},
Compute: ComputeConfig{Type: "validate", Command: "create-validate"},
}
if err := d.Validate(); err == nil {
t.Fatal("expected error for invalid scope type")
}
}
func TestValidateInvalidComputeType(t *testing.T) {
d := &Definition{
Name: "test",
Trigger: TriggerConfig{Type: "manual"},
Scope: ScopeConfig{Type: "item"},
Compute: ComputeConfig{Type: "teleport", Command: "beam-up"},
}
if err := d.Validate(); err == nil {
t.Fatal("expected error for invalid compute type")
}
}

200
internal/kc/kc.go Normal file
View File

@@ -0,0 +1,200 @@
// Package kc extracts and parses the silo/ metadata directory from .kc files.
//
// A .kc file is a ZIP archive (superset of .fcstd) that contains a silo/
// directory with JSON metadata entries. This package handles extraction and
// packing — no database or HTTP dependencies.
package kc
import (
"archive/zip"
"bytes"
"encoding/json"
"fmt"
"io"
"strings"
)
// Manifest represents the contents of silo/manifest.json.
type Manifest struct {
UUID string `json:"uuid"`
KCVersion string `json:"kc_version"`
RevisionHash string `json:"revision_hash"`
SiloInstance string `json:"silo_instance"`
}
// Metadata represents the contents of silo/metadata.json.
type Metadata struct {
SchemaName string `json:"schema_name"`
Tags []string `json:"tags"`
LifecycleState string `json:"lifecycle_state"`
Fields map[string]any `json:"fields"`
}
// Dependency represents one entry in silo/dependencies.json.
type Dependency struct {
UUID string `json:"uuid"`
PartNumber string `json:"part_number"`
Revision int `json:"revision"`
Quantity float64 `json:"quantity"`
Label string `json:"label"`
Relationship string `json:"relationship"`
}
// MacroFile represents a script file found under silo/macros/.
type MacroFile struct {
Filename string
Content string
}
// ExtractResult holds the parsed silo/ directory contents from a .kc file.
type ExtractResult struct {
Manifest *Manifest
Metadata *Metadata
Dependencies []Dependency
Macros []MacroFile
}
// HistoryEntry represents one entry in silo/history.json.
type HistoryEntry struct {
RevisionNumber int `json:"revision_number"`
CreatedAt string `json:"created_at"`
CreatedBy *string `json:"created_by,omitempty"`
Comment *string `json:"comment,omitempty"`
Status string `json:"status"`
Labels []string `json:"labels"`
}
// ApprovalEntry represents one entry in silo/approvals.json.
type ApprovalEntry struct {
ID string `json:"id"`
WorkflowName string `json:"workflow"`
ECONumber string `json:"eco_number,omitempty"`
State string `json:"state"`
UpdatedAt string `json:"updated_at"`
UpdatedBy string `json:"updated_by,omitempty"`
Signatures []SignatureEntry `json:"signatures"`
}
// SignatureEntry represents one signer in an approval.
type SignatureEntry struct {
Username string `json:"username"`
Role string `json:"role"`
Status string `json:"status"`
SignedAt string `json:"signed_at,omitempty"`
Comment string `json:"comment,omitempty"`
}
// PackInput holds all the data needed to repack silo/ entries in a .kc file.
// Each field is optional — nil/empty means the entry is omitted from the ZIP.
type PackInput struct {
Manifest *Manifest
Metadata *Metadata
History []HistoryEntry
Dependencies []Dependency
Approvals []ApprovalEntry
}
// Extract opens a ZIP archive from data and parses the silo/ directory.
// Returns nil, nil if no silo/ directory is found (plain .fcstd file).
// Returns nil, error if silo/ entries exist but fail to parse.
func Extract(data []byte) (*ExtractResult, error) {
r, err := zip.NewReader(bytes.NewReader(data), int64(len(data)))
if err != nil {
return nil, fmt.Errorf("kc: open zip: %w", err)
}
var manifestFile, metadataFile, dependenciesFile *zip.File
var macroFiles []*zip.File
hasSiloDir := false
for _, f := range r.File {
if f.Name == "silo/" || strings.HasPrefix(f.Name, "silo/") {
hasSiloDir = true
}
switch f.Name {
case "silo/manifest.json":
manifestFile = f
case "silo/metadata.json":
metadataFile = f
case "silo/dependencies.json":
dependenciesFile = f
default:
if strings.HasPrefix(f.Name, "silo/macros/") && !f.FileInfo().IsDir() {
name := strings.TrimPrefix(f.Name, "silo/macros/")
if name != "" {
macroFiles = append(macroFiles, f)
}
}
}
}
if !hasSiloDir {
return nil, nil // plain .fcstd, no extraction
}
result := &ExtractResult{}
if manifestFile != nil {
m, err := readJSON[Manifest](manifestFile)
if err != nil {
return nil, fmt.Errorf("kc: parse manifest.json: %w", err)
}
result.Manifest = m
}
if metadataFile != nil {
m, err := readJSON[Metadata](metadataFile)
if err != nil {
return nil, fmt.Errorf("kc: parse metadata.json: %w", err)
}
result.Metadata = m
}
if dependenciesFile != nil {
deps, err := readJSON[[]Dependency](dependenciesFile)
if err != nil {
return nil, fmt.Errorf("kc: parse dependencies.json: %w", err)
}
if deps != nil {
result.Dependencies = *deps
}
}
for _, mf := range macroFiles {
rc, err := mf.Open()
if err != nil {
return nil, fmt.Errorf("kc: open macro %s: %w", mf.Name, err)
}
content, err := io.ReadAll(rc)
rc.Close()
if err != nil {
return nil, fmt.Errorf("kc: read macro %s: %w", mf.Name, err)
}
result.Macros = append(result.Macros, MacroFile{
Filename: strings.TrimPrefix(mf.Name, "silo/macros/"),
Content: string(content),
})
}
return result, nil
}
// readJSON opens a zip.File and decodes its contents as JSON into T.
func readJSON[T any](f *zip.File) (*T, error) {
rc, err := f.Open()
if err != nil {
return nil, err
}
defer rc.Close()
data, err := io.ReadAll(rc)
if err != nil {
return nil, err
}
var v T
if err := json.Unmarshal(data, &v); err != nil {
return nil, err
}
return &v, nil
}

Some files were not shown because too many files have changed in this diff Show More