Commit Graph

205 Commits

Author SHA1 Message Date
e260c175bf Merge pull request 'docs: update documentation for .kc file integration (Phases 1-4)' (#153) from docs/kc-integration into main
Reviewed-on: #153
2026-02-19 01:11:23 +00:00
Forbes
bae06da1a1 docs: update documentation for .kc file integration (Phases 1-4)
- SPECIFICATION.md: add 8 KC endpoints to Section 11.1, new Section 11.3
  documenting .kc format, extraction pipeline, packing, lifecycle state
  machine, and all response shapes. Update endpoint count 78 → 86.
- ROADMAP.md: mark .kc Format Spec as Complete in Tier 0 table
- STATUS.md: add KC features to core systems table, update migration
  list through 018, update endpoint count
- MODULES.md: add metadata, dependencies, and macros endpoints to
  core module listing
2026-02-18 19:10:56 -06:00
161c1c1e62 Merge pull request 'feat(api): macro indexing from .kc files and read-only API' (#152) from feat/kc-macros into main
Reviewed-on: #152
2026-02-19 01:06:00 +00:00
df0fc13193 Merge branch 'main' into feat/kc-macros 2026-02-19 01:05:51 +00:00
Forbes
6e6c9c2c75 feat(api): macro indexing from .kc files and read-only API
- Add MacroFile type to internal/kc and extract silo/macros/* files
  from .kc ZIP archives on commit
- Create ItemMacroRepository with ReplaceForItem, ListByItem, and
  GetByFilename methods
- Add GET /{partNumber}/macros (list) and
  GET /{partNumber}/macros/{filename} (source content) endpoints
- Index macros in extractKCMetadata with SSE broadcast
- List endpoint omits content for lightweight responses

Closes #144
2026-02-18 19:03:44 -06:00
98be1fa78c Merge pull request 'feat(api): item dependency extraction, indexing, and resolve endpoints' (#151) from feat/kc-dependencies into main
Reviewed-on: #151
2026-02-19 00:55:55 +00:00
f8b8eda973 Merge branch 'main' into feat/kc-dependencies 2026-02-19 00:55:40 +00:00
Forbes
cffcf56085 feat(api): item dependency extraction, indexing, and resolve endpoints
- Add Dependency type to internal/kc and extract silo/dependencies.json
  from .kc files on commit
- Create ItemDependencyRepository with ReplaceForRevision, ListByItem,
  and Resolve (LEFT JOIN against items table)
- Add GET /{partNumber}/dependencies and
  GET /{partNumber}/dependencies/resolve endpoints
- Index dependencies in extractKCMetadata with SSE broadcast
- Pack real dependency data into .kc files on checkout
- Update PackInput.Dependencies from []any to []Dependency

Closes #143
2026-02-18 18:53:40 -06:00
1a34455ad5 Merge pull request 'feat(kc): checkout packing + ETag caching (Phase 2)' (#150) from feat/kc-checkout-packing into main
Reviewed-on: #150
2026-02-18 23:06:17 +00:00
Forbes
c216d64702 feat(kc): checkout packing + ETag caching (Phase 2)
Implements issue #142 — .kc checkout pipeline that repacks silo/ entries
with current DB state before serving downloads.

When a client downloads a .kc file via GET /api/items/{pn}/file/{rev},
the server now:
1. Reads the file from storage into memory
2. Checks for silo/ directory (plain .fcstd files bypass packing)
3. Repacks silo/ entries with current item_metadata + revision history
4. Streams the repacked ZIP to the client

New files:
- internal/kc/pack.go: Pack() replaces silo/ entries in ZIP, preserving
  all non-silo entries (FreeCAD files, thumbnails) with original
  compression and timestamps. HasSiloDir() for lightweight detection.
- internal/api/pack_handlers.go: packKCFile server helper, computeETag,
  canSkipRepack lazy optimization.

ETag caching:
- ETag computed from revision_number + metadata.updated_at
- If-None-Match support returns 304 Not Modified before reading storage
- Cache-Control: private, must-revalidate

Lazy packing optimization:
- Skips repack if revision_hash matches and metadata unchanged since upload

Phase 2 packs: manifest.json, metadata.json, history.json,
dependencies.json (empty []). Approvals, macros, jobs deferred to
Phase 3-5.

Closes #142
2026-02-18 17:01:26 -06:00
28f133411e Merge pull request 'feat(kc): commit extraction pipeline + metadata API (Phase 1)' (#149) from feat/kc-extraction-pipeline into main
Reviewed-on: #149
2026-02-18 22:39:59 +00:00
6528df0461 Merge branch 'main' into feat/kc-extraction-pipeline 2026-02-18 22:39:49 +00:00
Forbes
dd010331c0 feat(kc): commit extraction pipeline + metadata API (Phase 1)
Implements issue #141 — .kc server-side metadata integration Phase 1.

When a .kc file is uploaded, the server extracts silo/manifest.json and
silo/metadata.json from the ZIP archive and indexes them into the
item_metadata table. Plain .fcstd files continue to work unchanged.
Extraction is best-effort: failures are logged but do not block the upload.

New packages:
- internal/kc: ZIP extraction library (Extract, Manifest, Metadata types)
- internal/db: ItemMetadataRepository (Get, Upsert, UpdateFields,
  UpdateLifecycle, SetTags)

New API endpoints under /api/items/{partNumber}:
- GET    /metadata           — read indexed metadata (viewer)
- PUT    /metadata           — merge fields into JSONB (editor)
- PATCH  /metadata/lifecycle — transition lifecycle state (editor)
- PATCH  /metadata/tags      — add/remove tags (editor)

SSE events: metadata.updated, metadata.lifecycle, metadata.tags

Lifecycle transitions (Phase 1): draft→review→released→obsolete,
review→draft (reject).

Closes #141
2026-02-18 16:37:39 -06:00
628cd1d252 Merge pull request 'feat(db): .kc metadata database migration' (#148) from feat/kc-metadata-migration into main
Reviewed-on: #148
2026-02-18 21:05:15 +00:00
Forbes
8d777e83bb feat(db): .kc metadata database migration (#140)
Add migration 018_kc_metadata.sql with all tables needed for .kc
server-side metadata indexing:

- item_metadata: indexed manifest + metadata fields from silo/
  directory (tags, lifecycle_state, fields JSONB, manifest info)
- item_dependencies: CAD-extracted assembly dependencies
  (complements existing relationships table)
- item_approvals + approval_signatures: ECO workflow state
- item_macros: registered macros from silo/macros/

Also adds docs/KC_SERVER.md specification document.

Closes #140
2026-02-18 15:04:03 -06:00
d96ba8d394 Merge pull request 'docs: replace MinIO with filesystem storage throughout' (#139) from update-silo-fs-docs into main
Reviewed-on: #139
2026-02-18 20:46:51 +00:00
Forbes
56c76940ed docs: replace MinIO with filesystem storage throughout
Remove all MinIO/S3 references from documentation and deployment
configs. Silo now uses local filesystem storage exclusively.

Updated files:
- docs/CONFIGURATION.md: storage section now documents filesystem backend
- docs/DEPLOYMENT.md: architecture diagram, external services, troubleshooting
- docs/INSTALL.md: remove MinIO setup, update architecture diagrams
- docs/SPECIFICATION.md: architecture, technology stack, file storage strategy
- docs/STATUS.md: storage backend status
- docs/GAP_ANALYSIS.md: file handling references
- docs/ROADMAP.md: file storage appendix entries
- deployments/config.prod.yaml: filesystem backend config
- deployments/systemd/silod.env.example: remove MinIO credential vars
2026-02-18 14:45:00 -06:00
9dabaf5796 Merge pull request 'feat(scripts): remote migrate-storage script for MinIO to filesystem migration' (#138) from feat-remote-migrate-storage into main
Reviewed-on: #138
2026-02-18 20:33:38 +00:00
Forbes
3bb335397c feat(scripts): remote migrate-storage script for MinIO to filesystem migration
Adds scripts/migrate-storage.sh that follows the same deploy.sh pattern:
cross-compiles the migrate-storage binary locally, uploads it to the
target host via SCP, then runs it over SSH using credentials from
/etc/silo/silod.env.

Usage: ./scripts/migrate-storage.sh <silo-host> <psql-host> <minio-host> [flags...]
2026-02-18 14:29:46 -06:00
344a0cd0a0 Merge pull request 'feat(storage): add MinIO to filesystem migration tool' (#137) from feat/migrate-storage-tool into main
Reviewed-on: #137
2026-02-18 20:16:17 +00:00
forbes
f5b03989ff feat(storage): add MinIO to filesystem migration tool
Standalone binary (cmd/migrate-storage) that downloads all files from
MinIO and writes them to the local filesystem for decommissioning MinIO.

Queries revision files, item file attachments, and item thumbnails from
the database, then downloads each from MinIO preserving the object key
structure as filesystem paths. Supports --dry-run, --verbose, atomic
writes via temp+rename, and idempotent re-runs (skips existing files
with matching size).
2026-02-18 14:12:32 -06:00
8cd92a4025 Merge pull request 'feat(api): direct multipart upload endpoints for filesystem backend' (#136) from feat-direct-upload into main
Reviewed-on: #136
2026-02-17 19:05:39 +00:00
ffa01ebeb7 feat(api): direct multipart upload endpoints for filesystem backend
Add three new endpoints that bypass the MinIO presigned URL flow:
- POST /api/items/{pn}/files/upload — multipart file upload
- POST /api/items/{pn}/thumbnail/upload — multipart thumbnail upload
- GET /api/items/{pn}/files/{fileId}/download — stream file download

Rewrite frontend upload flow: files are held in browser memory on drop
and uploaded directly after item creation via multipart POST. The old
presign+associate endpoints remain for MinIO backward compatibility.

Closes #129
2026-02-17 13:04:44 -06:00
9181673554 Merge pull request 'feat(db): add storage backend metadata columns' (#135) from feat-file-storage-metadata into main
Reviewed-on: #135
2026-02-17 18:32:05 +00:00
8cef4fa55f feat(db): add storage backend metadata columns
Add storage_backend columns to track which backend (minio or filesystem)
holds each file, enabling dual-running during migration.

Migration 017_file_storage_metadata.sql:
- item_files.storage_backend TEXT NOT NULL DEFAULT 'minio'
- revisions.file_storage_backend TEXT NOT NULL DEFAULT 'minio'

DB repository changes:
- Revision struct: add FileStorageBackend field
- ItemFile struct: add StorageBackend field
- All INSERT queries include the new columns
- All SELECT queries read them (COALESCE for pre-migration compat)
- CreateRevisionFromExisting copies the backend from source revision
- Default to 'minio' when field is empty (backward compat)

Existing rows default to 'minio'. New uploads will write 'filesystem'
when the filesystem backend is active.

Closes #128
2026-02-17 12:30:20 -06:00
7a9dd057a5 Merge pull request 'feat(storage): FileStore interface abstraction + filesystem backend' (#134) from feat-storage-interface-filesystem into main
Reviewed-on: #134
2026-02-17 17:55:09 +00:00
9f347e7898 feat(storage): implement filesystem backend
Implement FilesystemStore satisfying the FileStore interface for local
filesystem storage, replacing MinIO for simpler deployments.

- Atomic writes via temp file + os.Rename (no partial files)
- SHA-256 checksum computed on Put via io.MultiWriter
- Get/GetVersion return os.File (GetVersion ignores versionID)
- Delete is idempotent (no error if file missing)
- Copy uses same atomic write pattern
- PresignPut returns ErrPresignNotSupported
- Ping verifies root directory is writable
- Wire NewFilesystemStore in main.go backend switch
- 14 unit tests covering all methods including atomicity

Closes #127
2026-02-17 11:49:42 -06:00
b531617e39 feat(storage): define FileStore interface and refactor to use it
Extract a FileStore interface from the concrete *storage.Storage MinIO
wrapper so the API layer is storage-backend agnostic.

- Define FileStore interface in internal/storage/interface.go
- Add Exists method to MinIO Storage (via StatObject)
- Add compile-time interface satisfaction check
- Change Server.storage and ServerState.storage to FileStore interface
- Update NewServer and NewServerState signatures
- Add Backend and FilesystemConfig fields to StorageConfig
- Add backend selection switch in main.go (minio/filesystem/unknown)
- Update config.example.yaml with backend field

The nil-interface pattern is preserved: when storage is unconfigured,
store remains a true nil FileStore (not a typed nil pointer), so all
existing if s.storage == nil checks continue to work correctly.

Closes #126
2026-02-17 11:49:35 -06:00
906277149e Merge pull request 'feat(web): read-write configuration from admin UI' (#124) from feat-admin-config-ui into main
Reviewed-on: #124
2026-02-15 23:12:04 +00:00
Forbes
fc4826f576 feat(web): read-write configuration from admin UI
Convert all module settings from read-only to editable fields in the
admin settings page:

- Core: host, port, base_url (read-only stays read-only)
- Schemas: directory, default (count stays read-only)
- Database: host, port, name, user, password, sslmode (dropdown),
  max_connections
- Storage: endpoint, bucket, use_ssl (checkbox), region
- Auth: local/ldap/oidc sub-sections with enabled checkboxes,
  connection fields, and secret fields (password input for redacted)

New field components: SelectField (dropdown), CheckboxField (toggle).
Redacted fields now render as password inputs with placeholder.
Auth uses nested key handling to send sub-section objects.

Backend already persists overrides and flags restart-required changes.

Closes #117
2026-02-15 13:33:48 -06:00
fbfc955ccc Merge pull request 'feat(modules): SSE settings.changed event broadcast' (#123) from feat-sse-settings-changed into main
Reviewed-on: #123
2026-02-15 19:14:36 +00:00
e0295e7180 Merge branch 'main' into feat-sse-settings-changed 2026-02-15 19:14:26 +00:00
Forbes
7fec219152 feat(modules): SSE settings.changed event broadcast and UI reactions
Add useSSE hook that connects to /api/events with automatic reconnect
and exponential backoff. On settings.changed events:

- Refresh module state so sidebar nav items show/hide immediately
- Show dismissable toast when another admin updates settings

The backend already publishes settings.changed in HandleUpdateModuleSettings.

Closes #101
2026-02-15 13:11:04 -06:00
fa069eb05c Merge pull request 'feat(web): move edit/delete buttons into tab bar on item detail' (#122) from feat-move-edit-delete-buttons into main
Reviewed-on: #122
2026-02-15 19:03:59 +00:00
Forbes
8735c8341b feat(web): move edit/delete buttons into tab bar on item detail
Relocate Edit and Delete buttons from the header row into the tab bar,
grouping them with tab navigation to reduce mouse travel. Adds Pencil
and Trash2 icons for quick visual recognition.

Header now only shows part number, type badge, and close button.

Closes #119
2026-02-15 12:59:40 -06:00
7a172ce34c Merge pull request 'feat(web): favicon, narrow settings, scrollable token list' (#121) from feat-ui-tweaks into main
Reviewed-on: #121
2026-02-15 18:47:03 +00:00
Forbes
da65d4bc1a feat(web): favicon, narrow settings, scrollable token list
- Add kindred-logo.svg as site favicon (#115)
- Narrow settings page to 66% max-width, centered (#116)
- Add max-height and scroll to API token table (#118)

Closes #115, closes #116, closes #118
2026-02-15 12:38:20 -06:00
57d5a786d0 Merge pull request 'feat(web): collapsible left sidebar, remove top nav bar' (#120) from feat-sidebar-nav into main
Reviewed-on: #120
2026-02-15 18:33:09 +00:00
Forbes
42a901f39c feat(web): collapsible left sidebar, remove top nav bar
- Replace top header with left sidebar navigation
- Sidebar shows module-aware nav items filtered by /api/modules
- Collapsible: expanded shows icon+label, collapsed shows icon only
- Toggle with Ctrl+J or collapse button, state persisted in localStorage
- Keyboard navigable: Arrow Up/Down, Enter to navigate, Escape to collapse
- Bottom section: density toggle, user info with role badge, logout
- Add useModules hook for fetching module state
- Add sidebar density variables to theme.css

Closes #113, closes #114
2026-02-15 12:32:52 -06:00
666cc2b23b Merge pull request 'feat(jobs): wire auto-triggering on bom_changed events' (#112) from feat-job-auto-trigger into main
Reviewed-on: #112
2026-02-15 15:44:42 +00:00
Forbes
747bae8354 feat(jobs): wire auto-triggering on bom_changed events, add module guard
- Add IsEnabled("jobs") guard to triggerJobs() to skip when module disabled
- Fire bom_changed trigger from HandleAddBOMEntry, HandleUpdateBOMEntry,
  HandleDeleteBOMEntry (matching existing HandleMergeBOM pattern)
- Add 4 integration tests: revision trigger, BOM trigger, filter mismatch,
  module disabled
- Fix AppShell overflow: hidden -> auto so Settings page scrolls
- Clean old frontend assets in deploy script before extracting

Closes #107
2026-02-15 09:43:05 -06:00
71603bb6d7 Merge pull request 'feat: location hierarchy CRUD API' (#106) from feat-location-crud into main
Reviewed-on: #106
2026-02-15 09:16:52 +00:00
Forbes
4ef912cf4b feat: location hierarchy CRUD API
Add LocationRepository with CRUD operations, hierarchy traversal
(children, subtree by path prefix), and inventory-safe deletion.

Endpoints:
  GET    /api/locations          — list all or ?tree={path} for subtree
  POST   /api/locations          — create (auto-resolves parent_id, depth)
  GET    /api/locations/{path..} — get by hierarchical path
  PUT    /api/locations/{path..} — update name, type, metadata
  DELETE /api/locations/{path..} — delete (rejects if inventory exists)

Uses chi wildcard routes to support multi-segment paths like
/api/locations/lab/shelf-a/bin-3.

Includes 10 handler integration tests covering CRUD, nesting,
validation, duplicates, tree queries, and delete-not-found.

Closes #81
2026-02-15 03:15:54 -06:00
decb32c3e7 Merge pull request 'feat(web): admin settings page — module cards, toggles, config forms' (#105) from feat-admin-settings-api into main
Reviewed-on: #105
2026-02-15 09:09:17 +00:00
Forbes
0be39065ac feat(web): admin settings page with module cards, toggles, config forms
Add admin-only Module Configuration section to the Settings page.
Each module gets a collapsible card with enable/disable toggle,
status badge, module-specific config fields, save and test
connectivity buttons.

- AdminModules: fetches GET /api/modules + GET /api/admin/settings,
  renders Infrastructure and Features groups, restart banner
- ModuleCard: collapsible card with toggle, status badge, field
  layouts per module, save (PUT) and test (POST) actions
- TypeScript types for ModuleInfo, ModulesResponse, admin settings
  API response shapes

Ref: #100
2026-02-15 03:01:33 -06:00
Forbes
101d04ab6f test(api): admin settings handler tests
- TestGetAllSettings — all module keys present, secrets redacted
- TestGetModuleSettings — single module response
- TestGetModuleSettings_Unknown — 404 for unknown module
- TestToggleModule — disable projects, verify registry state
- TestToggleModule_DependencyError — enable dag without jobs, expect 400
- TestToggleRequiredModule — disable core, expect 400
- TestTestConnectivity_Database — ping database, expect success
- TestTestConnectivity_NotTestable — core module, expect 400
2026-02-15 02:51:00 -06:00
Forbes
8167d9c216 feat(api): admin settings API endpoints
Add four admin-only endpoints under /api/admin/settings:

- GET  /                — full config (secrets redacted)
- GET  /{module}        — single module config
- PUT  /{module}        — toggle modules + persist config overrides
- POST /{module}/test   — test external connectivity (database, storage)

PUT publishes a settings.changed SSE event. Config overrides are
persisted for future hot-reload support; changes to database/storage/
server/schemas namespaces return restart_required: true.

Wires SettingsRepository into Server struct.

Closes #99
2026-02-15 02:51:00 -06:00
Forbes
319a739adb feat(db): add SettingsRepository for module state and config overrides
Provides CRUD operations on the module_state and settings_overrides
tables (created in migration 016).

- GetModuleStates / SetModuleState — upsert module enabled/disabled
- GetOverrides / SetOverride / DeleteOverride — JSONB config overrides

Part of #99
2026-02-15 02:51:00 -06:00
e20252a993 Merge pull request 'feat: module system — registry, middleware, and discovery endpoint' (#102) from feat-module-system into main
Reviewed-on: #102
2026-02-14 20:05:42 +00:00
Forbes
138ce16010 fix: remove unreachable code in testutil.findProjectRoot 2026-02-14 14:02:48 -06:00