Compare commits
2 Commits
feat/kc-de
...
feat/kc-ma
| Author | SHA1 | Date | |
|---|---|---|---|
|
b4b7d326ff | ||
|
|
6e6c9c2c75 |
@@ -1,485 +0,0 @@
|
||||
# .kc Server-Side Metadata Integration
|
||||
|
||||
**Status:** Draft
|
||||
**Date:** February 2026
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
When a `.kc` file is committed to Silo, the server extracts and indexes the `silo/` directory contents so that metadata is queryable, diffable, and streamable without downloading the full file. This document specifies the server-side processing pipeline, database storage, API endpoints, and SSE events that support the Create viewport widgets defined in [SILO_VIEWPORT.md](SILO_VIEWPORT.md).
|
||||
|
||||
The core principle: **the `.kc` file is the transport format; Silo is the index.** The `silo/` directory entries are extracted into database columns on commit and packed back into the ZIP on checkout. The server never modifies the FreeCAD standard zone (`Document.xml`, `.brp` files, `thumbnails/`).
|
||||
|
||||
---
|
||||
|
||||
## 2. Commit Pipeline
|
||||
|
||||
When a `.kc` file is uploaded via `POST /api/items/{partNumber}/file`, the server runs an extraction pipeline before returning success.
|
||||
|
||||
### 2.1 Pipeline Steps
|
||||
|
||||
```
|
||||
Client uploads .kc file
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 1. Store file to disk | (existing behavior -- unchanged)
|
||||
| items/{pn}/rev{N}.kc |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 2. Open ZIP, read silo/ |
|
||||
| Parse each entry |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 3. Validate manifest.json |
|
||||
| - UUID matches item |
|
||||
| - kc_version supported |
|
||||
| - revision_hash present |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 4. Index metadata |
|
||||
| - Upsert item_metadata |
|
||||
| - Upsert dependencies |
|
||||
| - Append history entry |
|
||||
| - Snapshot approvals |
|
||||
| - Register macros |
|
||||
| - Register job defs |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 5. Broadcast SSE events |
|
||||
| - revision.created |
|
||||
| - metadata.updated |
|
||||
| - bom.changed (if deps |
|
||||
| differ from previous) |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
Return 201 Created
|
||||
```
|
||||
|
||||
### 2.2 Validation Rules
|
||||
|
||||
| Check | Failure response |
|
||||
|-------|-----------------|
|
||||
| `silo/manifest.json` missing | `400 Bad Request` -- file is `.fcstd` not `.kc` |
|
||||
| `manifest.uuid` doesn't match item's UUID | `409 Conflict` -- wrong item |
|
||||
| `manifest.kc_version` > server's supported version | `422 Unprocessable` -- client newer than server |
|
||||
| `manifest.revision_hash` matches current head | `200 OK` (no-op, file unchanged) |
|
||||
| Any `silo/` JSON fails to parse | `422 Unprocessable` with path and parse error |
|
||||
|
||||
If validation fails, the blob is still stored (the user uploaded it), but no metadata indexing occurs. The item's revision is created with a `metadata_error` flag so the web UI can surface the problem.
|
||||
|
||||
### 2.3 Backward Compatibility
|
||||
|
||||
Plain `.fcstd` files (no `silo/` directory) continue to work exactly as today -- stored on disk, revision created, no metadata extraction. The pipeline short-circuits at step 2 when no `silo/` directory is found.
|
||||
|
||||
---
|
||||
|
||||
## 3. Database Schema
|
||||
|
||||
### 3.1 `item_metadata` Table
|
||||
|
||||
Stores the indexed contents of `silo/metadata.json` as structured JSONB, searchable and filterable via the existing item query endpoints.
|
||||
|
||||
```sql
|
||||
CREATE TABLE item_metadata (
|
||||
item_id UUID PRIMARY KEY REFERENCES items(id) ON DELETE CASCADE,
|
||||
schema_name TEXT,
|
||||
tags TEXT[] NOT NULL DEFAULT '{}',
|
||||
lifecycle_state TEXT NOT NULL DEFAULT 'draft',
|
||||
fields JSONB NOT NULL DEFAULT '{}',
|
||||
kc_version TEXT,
|
||||
manifest_uuid UUID,
|
||||
silo_instance TEXT,
|
||||
revision_hash TEXT,
|
||||
updated_at TIMESTAMPTZ DEFAULT now(),
|
||||
updated_by TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX idx_item_metadata_tags ON item_metadata USING GIN (tags);
|
||||
CREATE INDEX idx_item_metadata_lifecycle ON item_metadata (lifecycle_state);
|
||||
CREATE INDEX idx_item_metadata_fields ON item_metadata USING GIN (fields);
|
||||
```
|
||||
|
||||
On commit, the server upserts this row from `silo/manifest.json` and `silo/metadata.json`. The `fields` column contains the schema-driven key-value pairs exactly as they appear in the JSON.
|
||||
|
||||
### 3.2 `item_dependencies` Table
|
||||
|
||||
Stores the indexed contents of `silo/dependencies.json`. Replaces the BOM for assembly relationships that originate from the CAD model.
|
||||
|
||||
```sql
|
||||
CREATE TABLE item_dependencies (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
parent_item_id UUID REFERENCES items(id) ON DELETE CASCADE,
|
||||
child_uuid UUID NOT NULL,
|
||||
child_part_number TEXT,
|
||||
child_revision INTEGER,
|
||||
quantity DECIMAL,
|
||||
label TEXT,
|
||||
relationship TEXT NOT NULL DEFAULT 'component',
|
||||
revision_number INTEGER NOT NULL,
|
||||
created_at TIMESTAMPTZ DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_item_deps_parent ON item_dependencies (parent_item_id);
|
||||
CREATE INDEX idx_item_deps_child ON item_dependencies (child_uuid);
|
||||
```
|
||||
|
||||
This table complements the existing `relationships` table. The `relationships` table is the server-authoritative BOM (editable via the web UI and API). The `item_dependencies` table is the CAD-authoritative record extracted from the file. BOM merge (per [BOM_MERGE.md](BOM_MERGE.md)) reconciles the two.
|
||||
|
||||
### 3.3 `item_approvals` Table
|
||||
|
||||
Stores the indexed contents of `silo/approvals.json`. Server-authoritative -- the `.kc` snapshot is a read cache.
|
||||
|
||||
```sql
|
||||
CREATE TABLE item_approvals (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
item_id UUID REFERENCES items(id) ON DELETE CASCADE,
|
||||
eco_number TEXT,
|
||||
state TEXT NOT NULL DEFAULT 'draft',
|
||||
updated_at TIMESTAMPTZ DEFAULT now(),
|
||||
updated_by TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE approval_signatures (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
approval_id UUID REFERENCES item_approvals(id) ON DELETE CASCADE,
|
||||
username TEXT NOT NULL,
|
||||
role TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
signed_at TIMESTAMPTZ,
|
||||
comment TEXT
|
||||
);
|
||||
```
|
||||
|
||||
These tables exist independent of `.kc` commits -- approvals are created and managed through the web UI and API. On `.kc` checkout, the current approval state is serialized into `silo/approvals.json` for offline display.
|
||||
|
||||
### 3.4 `item_macros` Table
|
||||
|
||||
Registers macros from `silo/macros/` for server-side discoverability and the future Macro Store module.
|
||||
|
||||
```sql
|
||||
CREATE TABLE item_macros (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
item_id UUID REFERENCES items(id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
trigger TEXT NOT NULL DEFAULT 'manual',
|
||||
content TEXT NOT NULL,
|
||||
revision_number INTEGER NOT NULL,
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
UNIQUE(item_id, filename)
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. API Endpoints
|
||||
|
||||
These endpoints serve the viewport widgets in Create. All are under `/api/items/{partNumber}` and follow the existing auth model.
|
||||
|
||||
### 4.1 Metadata
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/metadata` | viewer | Get indexed metadata (schema fields, tags, lifecycle) |
|
||||
| `PUT` | `/metadata` | editor | Update metadata fields from client |
|
||||
| `PATCH` | `/metadata/lifecycle` | editor | Transition lifecycle state |
|
||||
| `PATCH` | `/metadata/tags` | editor | Add/remove tags |
|
||||
|
||||
**`GET /api/items/{partNumber}/metadata`**
|
||||
|
||||
Returns the indexed metadata for viewport display. This is the fast path -- reads from `item_metadata` rather than downloading and parsing the `.kc` ZIP.
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_name": "mechanical-part-v2",
|
||||
"lifecycle_state": "draft",
|
||||
"tags": ["structural", "aluminum"],
|
||||
"fields": {
|
||||
"material": "6061-T6",
|
||||
"finish": "anodized",
|
||||
"weight_kg": 0.34,
|
||||
"category": "bracket"
|
||||
},
|
||||
"manifest": {
|
||||
"uuid": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"silo_instance": "https://silo.example.com",
|
||||
"revision_hash": "a1b2c3d4e5f6",
|
||||
"kc_version": "1.0"
|
||||
},
|
||||
"updated_at": "2026-02-13T20:30:00Z",
|
||||
"updated_by": "joseph"
|
||||
}
|
||||
```
|
||||
|
||||
**`PUT /api/items/{partNumber}/metadata`**
|
||||
|
||||
Accepts a partial update of schema fields. The server merges into the existing `fields` JSONB. This is the write-back path for the Metadata Editor widget.
|
||||
|
||||
```json
|
||||
{
|
||||
"fields": {
|
||||
"material": "7075-T6",
|
||||
"weight_kg": 0.31
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The server validates field names against the schema descriptor. Unknown fields are rejected with `422`.
|
||||
|
||||
**`PATCH /api/items/{partNumber}/metadata/lifecycle`**
|
||||
|
||||
Transitions lifecycle state. The server validates the transition is permitted (e.g., `draft` -> `review` is allowed, `released` -> `draft` is not without admin override).
|
||||
|
||||
```json
|
||||
{ "state": "review" }
|
||||
```
|
||||
|
||||
### 4.2 Dependencies
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/dependencies` | viewer | Get CAD-extracted dependency list |
|
||||
| `GET` | `/dependencies/resolve` | viewer | Resolve UUIDs to current part numbers and file status |
|
||||
|
||||
**`GET /api/items/{partNumber}/dependencies`**
|
||||
|
||||
Returns the raw dependency list from the last `.kc` commit.
|
||||
|
||||
**`GET /api/items/{partNumber}/dependencies/resolve`**
|
||||
|
||||
Returns the dependency list with each UUID resolved to its current part number, revision, and whether the file exists on disk. This is what the Dependency Table widget calls to populate the status column.
|
||||
|
||||
```json
|
||||
{
|
||||
"links": [
|
||||
{
|
||||
"uuid": "660e8400-...",
|
||||
"part_number": "KC-BRK-0042",
|
||||
"label": "Base Plate",
|
||||
"revision": 2,
|
||||
"quantity": 1,
|
||||
"resolved": true,
|
||||
"file_available": true
|
||||
},
|
||||
{
|
||||
"uuid": "770e8400-...",
|
||||
"part_number": "KC-HDW-0108",
|
||||
"label": "M6 SHCS",
|
||||
"revision": 1,
|
||||
"quantity": 4,
|
||||
"resolved": true,
|
||||
"file_available": true
|
||||
},
|
||||
{
|
||||
"uuid": "880e8400-...",
|
||||
"part_number": null,
|
||||
"label": "Cover Panel",
|
||||
"revision": 1,
|
||||
"quantity": 1,
|
||||
"resolved": false,
|
||||
"file_available": false
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 Approvals
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/approvals` | viewer | Get current approval state |
|
||||
| `POST` | `/approvals` | editor | Create ECO / start approval workflow |
|
||||
| `POST` | `/approvals/{id}/sign` | editor | Sign (approve/reject) |
|
||||
|
||||
These endpoints power the Approvals Viewer widget. The viewer is read-only in Create -- sign actions happen in the web UI, but the API exists for both.
|
||||
|
||||
### 4.4 Macros
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/macros` | viewer | List registered macros |
|
||||
| `GET` | `/macros/{filename}` | viewer | Get macro source |
|
||||
|
||||
Read-only server-side. Macros are authored in Create and committed inside the `.kc`. The server indexes them for discoverability in the future Macro Store.
|
||||
|
||||
### 4.5 Existing Endpoints (unchanged)
|
||||
|
||||
The viewport widgets also consume these existing endpoints:
|
||||
|
||||
| Widget | Endpoint | Purpose |
|
||||
|--------|----------|---------|
|
||||
| History Viewer | `GET /api/items/{pn}/revisions` | Full revision list |
|
||||
| History Viewer | `GET /api/items/{pn}/revisions/compare` | Property diff |
|
||||
| Job Viewer | `GET /api/jobs?item={pn}&definition={name}&limit=1` | Last job run |
|
||||
| Job Viewer | `POST /api/jobs` | Trigger job |
|
||||
| Job Viewer | `GET /api/jobs/{id}/logs` | Job log |
|
||||
| Manifest Viewer | `GET /api/items/{pn}` | Item details (UUID, etc.) |
|
||||
|
||||
No changes needed to these -- they already exist and return the data the widgets need.
|
||||
|
||||
---
|
||||
|
||||
## 5. Checkout Pipeline
|
||||
|
||||
When a client downloads a `.kc` via `GET /api/items/{partNumber}/file`, the server packs current server-side state into the `silo/` directory before serving the file. This ensures the client always gets the latest metadata, even if it was edited via the web UI since the last commit.
|
||||
|
||||
### 5.1 Pipeline Steps
|
||||
|
||||
```
|
||||
Client requests file download
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 1. Read .kc from disk |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 2. Pack silo/ from DB |
|
||||
| - manifest.json (item) |
|
||||
| - metadata.json (index) |
|
||||
| - history.json (revs) |
|
||||
| - approvals.json (ECO) |
|
||||
| - dependencies.json |
|
||||
| - macros/ (index) |
|
||||
| - jobs/ (job defs) |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
+-----------------------------+
|
||||
| 3. Replace silo/ in ZIP |
|
||||
| Remove old entries |
|
||||
| Write packed entries |
|
||||
+-----------------------------+
|
||||
|
|
||||
v
|
||||
Stream .kc to client
|
||||
```
|
||||
|
||||
### 5.2 Packing Rules
|
||||
|
||||
| `silo/` entry | Source | Notes |
|
||||
|---------------|--------|-------|
|
||||
| `manifest.json` | `item_metadata` + `items` table | UUID from item, revision_hash from latest revision |
|
||||
| `metadata.json` | `item_metadata.fields` + tags + lifecycle | Serialized from indexed columns |
|
||||
| `history.json` | `revisions` table | Last 20 revisions for this item |
|
||||
| `approvals.json` | `item_approvals` + `approval_signatures` | Current ECO state, omitted if no active ECO |
|
||||
| `dependencies.json` | `item_dependencies` | Current revision's dependency list |
|
||||
| `macros/*.py` | `item_macros` | All registered macros |
|
||||
| `jobs/*.yaml` | `job_definitions` filtered by item type | Job definitions matching this item's trigger filters |
|
||||
|
||||
### 5.3 Caching
|
||||
|
||||
Packing the `silo/` directory on every download has a cost. To mitigate:
|
||||
|
||||
- **ETag header**: The response includes an ETag computed from the revision number + metadata `updated_at`. If the client sends `If-None-Match`, the server can return `304 Not Modified`.
|
||||
- **Lazy packing**: If the `.kc` blob's `silo/manifest.json` revision_hash matches the current head *and* `item_metadata.updated_at` is older than the blob's upload time, skip repacking entirely -- the blob is already current.
|
||||
|
||||
---
|
||||
|
||||
## 6. SSE Events
|
||||
|
||||
The viewport widgets subscribe to SSE for live updates. These events are broadcast when server-side metadata changes, whether via `.kc` commit, web UI edit, or API call.
|
||||
|
||||
| Event | Payload | Trigger |
|
||||
|-------|---------|---------|
|
||||
| `metadata.updated` | `{part_number, changed_fields[], lifecycle_state, updated_by}` | Metadata PUT/PATCH |
|
||||
| `metadata.lifecycle` | `{part_number, from_state, to_state, updated_by}` | Lifecycle transition |
|
||||
| `metadata.tags` | `{part_number, added[], removed[]}` | Tag add/remove |
|
||||
| `approval.created` | `{part_number, eco_number, state}` | ECO created |
|
||||
| `approval.signed` | `{part_number, eco_number, user, role, status}` | Approver action |
|
||||
| `approval.completed` | `{part_number, eco_number, final_state}` | All approvers acted |
|
||||
| `dependencies.changed` | `{part_number, added[], removed[], changed[]}` | Dependency diff on commit |
|
||||
|
||||
Existing events (`revision.created`, `job.*`, `bom.changed`) continue to work as documented in [SPECIFICATION.md](SPECIFICATION.md) and [WORKERS.md](WORKERS.md).
|
||||
|
||||
### 6.1 Widget Subscription Map
|
||||
|
||||
| Viewport widget | Subscribes to |
|
||||
|-----------------|---------------|
|
||||
| Manifest Viewer | -- (read-only, no live updates) |
|
||||
| Metadata Editor | `metadata.updated`, `metadata.lifecycle`, `metadata.tags` |
|
||||
| History Viewer | `revision.created` |
|
||||
| Approvals Viewer | `approval.created`, `approval.signed`, `approval.completed` |
|
||||
| Dependency Table | `dependencies.changed` |
|
||||
| Job Viewer | `job.created`, `job.progress`, `job.completed`, `job.failed` |
|
||||
| Macro Editor | -- (local-only until committed) |
|
||||
|
||||
---
|
||||
|
||||
## 7. Web UI Integration
|
||||
|
||||
The Silo web UI also benefits from indexed metadata. These are additions to existing pages, not new pages.
|
||||
|
||||
### 7.1 Items Page
|
||||
|
||||
The item detail panel gains a **Metadata** tab (alongside Main, Properties, Revisions, BOM, Where Used) showing the schema-driven form from `GET /api/items/{pn}/metadata`. Editable for editors.
|
||||
|
||||
### 7.2 Items List
|
||||
|
||||
New filterable columns: `lifecycle_state`, `tags`. The existing search endpoint gains metadata-aware filtering:
|
||||
|
||||
```
|
||||
GET /api/items?lifecycle=released&tag=aluminum
|
||||
GET /api/items/search?q=bracket&lifecycle=draft
|
||||
```
|
||||
|
||||
### 7.3 Approvals Page
|
||||
|
||||
A new page accessible from the top navigation (visible when a future `approvals` module is enabled). Lists all active ECOs with their approval progress.
|
||||
|
||||
---
|
||||
|
||||
## 8. Migration
|
||||
|
||||
### 8.1 Database Migration
|
||||
|
||||
A single migration adds the `item_metadata`, `item_dependencies`, `item_approvals`, `approval_signatures`, and `item_macros` tables. Existing items have no metadata rows -- they're created on first `.kc` commit or via `PUT /api/items/{pn}/metadata`.
|
||||
|
||||
### 8.2 Backfill
|
||||
|
||||
For items that already have `.kc` files stored on disk (committed before this feature), an admin endpoint re-runs the extraction pipeline:
|
||||
|
||||
```
|
||||
POST /api/admin/reindex-metadata
|
||||
```
|
||||
|
||||
This iterates all items with `.kc` files, opens each ZIP, and indexes the `silo/` contents. Idempotent -- safe to run multiple times.
|
||||
|
||||
---
|
||||
|
||||
## 9. Implementation Order
|
||||
|
||||
| Phase | Server work | Supports client phase |
|
||||
|-------|------------|----------------------|
|
||||
| 1 | `item_metadata` table + `GET/PUT /metadata` + commit extraction | SILO_VIEWPORT Phase 1-2 (Manifest, Metadata) |
|
||||
| 2 | Pack `silo/` on checkout + ETag caching | SILO_VIEWPORT Phase 1-3 |
|
||||
| 3 | `item_dependencies` table + `/dependencies/resolve` | SILO_VIEWPORT Phase 5 (Dependency Table) |
|
||||
| 4 | `item_macros` table + `/macros` endpoints | SILO_VIEWPORT Phase 6 (Macro Editor) |
|
||||
| 5 | `item_approvals` tables + `/approvals` endpoints | SILO_VIEWPORT Phase 7 (Approvals Viewer) |
|
||||
| 6 | SSE events for metadata/approvals/dependencies | SILO_VIEWPORT Phase 8 (Live integration) |
|
||||
| 7 | Web UI metadata tab + list filters | Independent of client |
|
||||
|
||||
Phases 1-2 are prerequisite for the viewport to work with live data. Phases 3-6 can be built in parallel with client widget development. Phase 7 is web-UI-only and independent.
|
||||
|
||||
---
|
||||
|
||||
## 10. References
|
||||
|
||||
- [SILO_VIEWPORT.md](SILO_VIEWPORT.md) -- Client-side viewport widget specification
|
||||
- [KC_SPECIFICATION.md](KC_SPECIFICATION.md) -- .kc file format specification
|
||||
- [SPECIFICATION.md](SPECIFICATION.md) -- Silo server API reference
|
||||
- [BOM_MERGE.md](BOM_MERGE.md) -- BOM merge rules (dependency reconciliation)
|
||||
- [WORKERS.md](WORKERS.md) -- Job queue (job viewer data source)
|
||||
- [MODULES.md](MODULES.md) -- Module system (approval module gating)
|
||||
- [ROADMAP.md](ROADMAP.md) -- Platform roadmap tiers
|
||||
749
docs/MODULES.md
Normal file
749
docs/MODULES.md
Normal file
@@ -0,0 +1,749 @@
|
||||
# Module System Specification
|
||||
|
||||
**Status:** Draft
|
||||
**Last Updated:** 2026-02-14
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
Silo's module system defines the boundary between required infrastructure and optional capabilities. Each module groups a set of API endpoints, UI views, and configuration parameters. Modules can be enabled or disabled at runtime by administrators via the web UI, and clients can query which modules are active to adapt their feature set.
|
||||
|
||||
The goal: after initial deployment (where `config.yaml` sets database, storage, and server bind), all further operational configuration happens through the admin settings UI. The YAML file becomes the bootstrap; the database becomes the runtime source of truth.
|
||||
|
||||
---
|
||||
|
||||
## 2. Module Registry
|
||||
|
||||
### 2.1 Required Modules
|
||||
|
||||
These cannot be disabled. They define what Silo *is*.
|
||||
|
||||
| Module ID | Name | Description |
|
||||
|-----------|------|-------------|
|
||||
| `core` | Core PDM | Items, revisions, files, BOM, search, import/export, part number generation |
|
||||
| `schemas` | Schemas | Part numbering schema parsing, segment management, form descriptors |
|
||||
| `storage` | Storage | MinIO/S3 file storage, presigned uploads, versioning |
|
||||
|
||||
### 2.2 Optional Modules
|
||||
|
||||
| Module ID | Name | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `auth` | Authentication | `true` | Local, LDAP, OIDC authentication and RBAC |
|
||||
| `projects` | Projects | `true` | Project management and item tagging |
|
||||
| `audit` | Audit | `true` | Audit logging, completeness scoring |
|
||||
| `odoo` | Odoo ERP | `false` | Odoo integration (config, sync-log, push/pull) |
|
||||
| `freecad` | Create Integration | `true` | URI scheme, executable path, client settings |
|
||||
| `jobs` | Job Queue | `false` | Async compute jobs, runner management |
|
||||
| `dag` | Dependency DAG | `false` | Feature DAG sync, validation states, interference detection |
|
||||
|
||||
### 2.3 Module Dependencies
|
||||
|
||||
Some modules require others to function:
|
||||
|
||||
| Module | Requires |
|
||||
|--------|----------|
|
||||
| `dag` | `jobs` |
|
||||
| `jobs` | `auth` (runner tokens) |
|
||||
| `odoo` | `auth` |
|
||||
|
||||
When enabling a module, its dependencies are validated. The server rejects enabling `dag` without `jobs`. Disabling a module that others depend on shows a warning listing dependents.
|
||||
|
||||
---
|
||||
|
||||
## 3. Endpoint-to-Module Mapping
|
||||
|
||||
### 3.1 `core` (required)
|
||||
|
||||
```
|
||||
# Health
|
||||
GET /health
|
||||
GET /ready
|
||||
|
||||
# Items
|
||||
GET /api/items
|
||||
GET /api/items/search
|
||||
GET /api/items/by-uuid/{uuid}
|
||||
GET /api/items/export.csv
|
||||
GET /api/items/template.csv
|
||||
GET /api/items/export.ods
|
||||
GET /api/items/template.ods
|
||||
POST /api/items
|
||||
POST /api/items/import
|
||||
POST /api/items/import.ods
|
||||
GET /api/items/{partNumber}
|
||||
PUT /api/items/{partNumber}
|
||||
DELETE /api/items/{partNumber}
|
||||
|
||||
# Revisions
|
||||
GET /api/items/{partNumber}/revisions
|
||||
GET /api/items/{partNumber}/revisions/compare
|
||||
GET /api/items/{partNumber}/revisions/{revision}
|
||||
POST /api/items/{partNumber}/revisions
|
||||
PATCH /api/items/{partNumber}/revisions/{revision}
|
||||
POST /api/items/{partNumber}/revisions/{revision}/rollback
|
||||
|
||||
# Files
|
||||
GET /api/items/{partNumber}/files
|
||||
GET /api/items/{partNumber}/file
|
||||
GET /api/items/{partNumber}/file/{revision}
|
||||
POST /api/items/{partNumber}/file
|
||||
POST /api/items/{partNumber}/files
|
||||
DELETE /api/items/{partNumber}/files/{fileId}
|
||||
PUT /api/items/{partNumber}/thumbnail
|
||||
POST /api/uploads/presign
|
||||
|
||||
# BOM
|
||||
GET /api/items/{partNumber}/bom
|
||||
GET /api/items/{partNumber}/bom/expanded
|
||||
GET /api/items/{partNumber}/bom/flat
|
||||
GET /api/items/{partNumber}/bom/cost
|
||||
GET /api/items/{partNumber}/bom/where-used
|
||||
GET /api/items/{partNumber}/bom/export.csv
|
||||
GET /api/items/{partNumber}/bom/export.ods
|
||||
POST /api/items/{partNumber}/bom
|
||||
POST /api/items/{partNumber}/bom/import
|
||||
POST /api/items/{partNumber}/bom/merge
|
||||
PUT /api/items/{partNumber}/bom/{childPartNumber}
|
||||
DELETE /api/items/{partNumber}/bom/{childPartNumber}
|
||||
|
||||
# .kc Metadata
|
||||
GET /api/items/{partNumber}/metadata
|
||||
PUT /api/items/{partNumber}/metadata
|
||||
PATCH /api/items/{partNumber}/metadata/lifecycle
|
||||
PATCH /api/items/{partNumber}/metadata/tags
|
||||
|
||||
# .kc Dependencies
|
||||
GET /api/items/{partNumber}/dependencies
|
||||
GET /api/items/{partNumber}/dependencies/resolve
|
||||
|
||||
# .kc Macros
|
||||
GET /api/items/{partNumber}/macros
|
||||
GET /api/items/{partNumber}/macros/{filename}
|
||||
|
||||
# Part Number Generation
|
||||
POST /api/generate-part-number
|
||||
|
||||
# Sheets
|
||||
POST /api/sheets/diff
|
||||
|
||||
# Settings & Modules (admin)
|
||||
GET /api/modules
|
||||
GET /api/admin/settings
|
||||
GET /api/admin/settings/{module}
|
||||
PUT /api/admin/settings/{module}
|
||||
POST /api/admin/settings/{module}/test
|
||||
```
|
||||
|
||||
### 3.2 `schemas` (required)
|
||||
|
||||
```
|
||||
GET /api/schemas
|
||||
GET /api/schemas/{name}
|
||||
GET /api/schemas/{name}/form
|
||||
POST /api/schemas/{name}/segments/{segment}/values
|
||||
PUT /api/schemas/{name}/segments/{segment}/values/{code}
|
||||
DELETE /api/schemas/{name}/segments/{segment}/values/{code}
|
||||
```
|
||||
|
||||
### 3.3 `storage` (required)
|
||||
|
||||
No dedicated endpoints — storage is consumed internally by file upload/download in `core`. Exposed through admin settings for connection status visibility.
|
||||
|
||||
### 3.4 `auth`
|
||||
|
||||
```
|
||||
# Public (login flow)
|
||||
GET /login
|
||||
POST /login
|
||||
POST /logout
|
||||
GET /auth/oidc
|
||||
GET /auth/callback
|
||||
|
||||
# Authenticated
|
||||
GET /api/auth/me
|
||||
GET /api/auth/tokens
|
||||
POST /api/auth/tokens
|
||||
DELETE /api/auth/tokens/{id}
|
||||
|
||||
# Web UI
|
||||
GET /settings (account info, tokens)
|
||||
POST /settings/tokens
|
||||
POST /settings/tokens/{id}/revoke
|
||||
```
|
||||
|
||||
When `auth` is disabled, all routes are open and a synthetic `dev` admin user is injected (current behavior).
|
||||
|
||||
### 3.5 `projects`
|
||||
|
||||
```
|
||||
GET /api/projects
|
||||
GET /api/projects/{code}
|
||||
GET /api/projects/{code}/items
|
||||
GET /api/projects/{code}/sheet.ods
|
||||
POST /api/projects
|
||||
PUT /api/projects/{code}
|
||||
DELETE /api/projects/{code}
|
||||
|
||||
# Item-project tagging
|
||||
GET /api/items/{partNumber}/projects
|
||||
POST /api/items/{partNumber}/projects
|
||||
DELETE /api/items/{partNumber}/projects/{code}
|
||||
```
|
||||
|
||||
When disabled: project tag endpoints return `404`, project columns are hidden in UI list views, project filter is removed from item search.
|
||||
|
||||
### 3.6 `audit`
|
||||
|
||||
```
|
||||
GET /api/audit/completeness
|
||||
GET /api/audit/completeness/{partNumber}
|
||||
```
|
||||
|
||||
When disabled: audit log table continues to receive writes (it's part of core middleware), but the completeness scoring endpoints and the Audit page in the web UI are hidden. Future: retention policies, export, and compliance reporting endpoints live here.
|
||||
|
||||
### 3.7 `odoo`
|
||||
|
||||
```
|
||||
GET /api/integrations/odoo/config
|
||||
GET /api/integrations/odoo/sync-log
|
||||
PUT /api/integrations/odoo/config
|
||||
POST /api/integrations/odoo/test-connection
|
||||
POST /api/integrations/odoo/sync/push/{partNumber}
|
||||
POST /api/integrations/odoo/sync/pull/{odooId}
|
||||
```
|
||||
|
||||
### 3.8 `freecad`
|
||||
|
||||
No dedicated API endpoints currently. Configures URI scheme and executable path used by the web UI's "Open in Create" links and by CLI operations. Future: client configuration distribution endpoint.
|
||||
|
||||
### 3.9 `jobs`
|
||||
|
||||
```
|
||||
# User-facing
|
||||
GET /api/jobs
|
||||
GET /api/jobs/{jobID}
|
||||
GET /api/jobs/{jobID}/logs
|
||||
POST /api/jobs
|
||||
POST /api/jobs/{jobID}/cancel
|
||||
|
||||
# Job definitions
|
||||
GET /api/job-definitions
|
||||
GET /api/job-definitions/{name}
|
||||
POST /api/job-definitions/reload
|
||||
|
||||
# Runner management (admin)
|
||||
GET /api/runners
|
||||
POST /api/runners
|
||||
DELETE /api/runners/{runnerID}
|
||||
|
||||
# Runner-facing (runner token auth)
|
||||
POST /api/runner/heartbeat
|
||||
POST /api/runner/claim
|
||||
PUT /api/runner/jobs/{jobID}/progress
|
||||
POST /api/runner/jobs/{jobID}/complete
|
||||
POST /api/runner/jobs/{jobID}/fail
|
||||
POST /api/runner/jobs/{jobID}/log
|
||||
PUT /api/runner/jobs/{jobID}/dag
|
||||
```
|
||||
|
||||
### 3.10 `dag`
|
||||
|
||||
```
|
||||
GET /api/items/{partNumber}/dag
|
||||
GET /api/items/{partNumber}/dag/forward-cone/{nodeKey}
|
||||
GET /api/items/{partNumber}/dag/dirty
|
||||
PUT /api/items/{partNumber}/dag
|
||||
POST /api/items/{partNumber}/dag/mark-dirty/{nodeKey}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Disabled Module Behavior
|
||||
|
||||
When a module is disabled:
|
||||
|
||||
1. **API routes** registered by that module return `404 Not Found` with body `{"error": "module '<id>' is not enabled"}`.
|
||||
2. **Web UI** hides the module's navigation entry, page, and any inline UI elements (e.g., project tags on item cards).
|
||||
3. **SSE events** from the module are not broadcast.
|
||||
4. **Background goroutines** (e.g., job timeout sweeper, runner heartbeat checker) are not started.
|
||||
5. **Database tables** are not dropped — they remain for re-enablement. No data loss on disable/enable cycle.
|
||||
|
||||
Implementation: each module's route group is wrapped in a middleware check:
|
||||
|
||||
```go
|
||||
func RequireModule(id string) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if !modules.IsEnabled(id) {
|
||||
http.Error(w, `{"error":"module '`+id+`' is not enabled"}`, 404)
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Configuration Persistence
|
||||
|
||||
### 5.1 Precedence
|
||||
|
||||
```
|
||||
Environment variables (highest — always wins, secrets live here)
|
||||
↓
|
||||
Database overrides (admin UI writes here)
|
||||
↓
|
||||
config.yaml (lowest — bootstrap defaults)
|
||||
```
|
||||
|
||||
### 5.2 Database Table
|
||||
|
||||
```sql
|
||||
-- Migration 014_settings.sql
|
||||
CREATE TABLE settings_overrides (
|
||||
key TEXT PRIMARY KEY, -- dotted path: "auth.ldap.enabled"
|
||||
value JSONB NOT NULL, -- typed value
|
||||
updated_by TEXT NOT NULL, -- username
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE module_state (
|
||||
module_id TEXT PRIMARY KEY, -- "auth", "projects", etc.
|
||||
enabled BOOLEAN NOT NULL,
|
||||
updated_by TEXT NOT NULL,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
```
|
||||
|
||||
### 5.3 Load Sequence
|
||||
|
||||
On startup:
|
||||
|
||||
1. Parse `config.yaml` into Go config struct.
|
||||
2. Query `settings_overrides` — merge each key into the struct using dotted path resolution.
|
||||
3. Apply environment variable overrides (existing `SILO_*` vars).
|
||||
4. Query `module_state` — override default enabled/disabled from YAML.
|
||||
5. Validate module dependencies.
|
||||
6. Register only enabled modules' route groups.
|
||||
7. Start only enabled modules' background goroutines.
|
||||
|
||||
### 5.4 Runtime Updates
|
||||
|
||||
When an admin saves settings via `PUT /api/admin/settings/{module}`:
|
||||
|
||||
1. Validate the payload against the module's config schema.
|
||||
2. Write changed keys to `settings_overrides`.
|
||||
3. Update `module_state` if `enabled` changed.
|
||||
4. Apply changes to the in-memory config (hot reload where safe).
|
||||
5. Broadcast `settings.changed` SSE event with `{module, enabled, changed_keys}`.
|
||||
6. For changes that require restart (e.g., `server.port`, `database.*`), return a `restart_required: true` flag in the response. The UI shows a banner.
|
||||
|
||||
### 5.5 What Requires Restart
|
||||
|
||||
| Config Area | Hot Reload | Restart Required |
|
||||
|-------------|-----------|------------------|
|
||||
| Module enable/disable | Yes | No |
|
||||
| `auth.*` provider toggles | Yes | No |
|
||||
| `auth.cors.allowed_origins` | Yes | No |
|
||||
| `odoo.*` connection settings | Yes | No |
|
||||
| `freecad.*` | Yes | No |
|
||||
| `jobs.*` timeouts, directory | Yes | No |
|
||||
| `server.host`, `server.port` | No | Yes |
|
||||
| `database.*` | No | Yes |
|
||||
| `storage.*` | No | Yes |
|
||||
| `schemas.directory` | No | Yes |
|
||||
|
||||
---
|
||||
|
||||
## 6. Public Module Discovery Endpoint
|
||||
|
||||
```
|
||||
GET /api/modules
|
||||
```
|
||||
|
||||
**No authentication required.** Clients need this pre-login to know whether OIDC is available, whether projects exist, etc.
|
||||
|
||||
### 6.1 Response
|
||||
|
||||
```json
|
||||
{
|
||||
"modules": {
|
||||
"core": {
|
||||
"enabled": true,
|
||||
"required": true,
|
||||
"name": "Core PDM",
|
||||
"version": "0.2"
|
||||
},
|
||||
"schemas": {
|
||||
"enabled": true,
|
||||
"required": true,
|
||||
"name": "Schemas"
|
||||
},
|
||||
"storage": {
|
||||
"enabled": true,
|
||||
"required": true,
|
||||
"name": "Storage"
|
||||
},
|
||||
"auth": {
|
||||
"enabled": true,
|
||||
"required": false,
|
||||
"name": "Authentication",
|
||||
"config": {
|
||||
"local_enabled": true,
|
||||
"ldap_enabled": true,
|
||||
"oidc_enabled": true,
|
||||
"oidc_issuer_url": "https://keycloak.example.com/realms/silo"
|
||||
}
|
||||
},
|
||||
"projects": {
|
||||
"enabled": true,
|
||||
"required": false,
|
||||
"name": "Projects"
|
||||
},
|
||||
"audit": {
|
||||
"enabled": true,
|
||||
"required": false,
|
||||
"name": "Audit"
|
||||
},
|
||||
"odoo": {
|
||||
"enabled": false,
|
||||
"required": false,
|
||||
"name": "Odoo ERP"
|
||||
},
|
||||
"freecad": {
|
||||
"enabled": true,
|
||||
"required": false,
|
||||
"name": "Create Integration",
|
||||
"config": {
|
||||
"uri_scheme": "silo"
|
||||
}
|
||||
},
|
||||
"jobs": {
|
||||
"enabled": false,
|
||||
"required": false,
|
||||
"name": "Job Queue"
|
||||
},
|
||||
"dag": {
|
||||
"enabled": false,
|
||||
"required": false,
|
||||
"name": "Dependency DAG",
|
||||
"depends_on": ["jobs"]
|
||||
}
|
||||
},
|
||||
"server": {
|
||||
"version": "0.2",
|
||||
"read_only": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `config` sub-object exposes only public, non-secret metadata needed by clients. Never includes passwords, tokens, or secret keys.
|
||||
|
||||
---
|
||||
|
||||
## 7. Admin Settings Endpoints
|
||||
|
||||
### 7.1 Get All Settings
|
||||
|
||||
```
|
||||
GET /api/admin/settings
|
||||
Authorization: Bearer <admin token>
|
||||
```
|
||||
|
||||
Returns full config grouped by module with secrets redacted:
|
||||
|
||||
```json
|
||||
{
|
||||
"core": {
|
||||
"server": {
|
||||
"host": "0.0.0.0",
|
||||
"port": 8080,
|
||||
"base_url": "https://silo.example.com",
|
||||
"read_only": false
|
||||
}
|
||||
},
|
||||
"schemas": {
|
||||
"directory": "/etc/silo/schemas",
|
||||
"default": "kindred-rd"
|
||||
},
|
||||
"storage": {
|
||||
"endpoint": "minio:9000",
|
||||
"bucket": "silo-files",
|
||||
"access_key": "****",
|
||||
"secret_key": "****",
|
||||
"use_ssl": false,
|
||||
"region": "us-east-1",
|
||||
"status": "connected"
|
||||
},
|
||||
"database": {
|
||||
"host": "postgres",
|
||||
"port": 5432,
|
||||
"name": "silo",
|
||||
"user": "silo",
|
||||
"password": "****",
|
||||
"sslmode": "disable",
|
||||
"max_connections": 10,
|
||||
"status": "connected"
|
||||
},
|
||||
"auth": {
|
||||
"enabled": true,
|
||||
"session_secret": "****",
|
||||
"local": { "enabled": true },
|
||||
"ldap": {
|
||||
"enabled": true,
|
||||
"url": "ldaps://ipa.example.com",
|
||||
"base_dn": "dc=kindred,dc=internal",
|
||||
"user_search_dn": "cn=users,cn=accounts,dc=kindred,dc=internal",
|
||||
"bind_password": "****",
|
||||
"role_mapping": { "...": "..." }
|
||||
},
|
||||
"oidc": {
|
||||
"enabled": true,
|
||||
"issuer_url": "https://keycloak.example.com/realms/silo",
|
||||
"client_id": "silo",
|
||||
"client_secret": "****",
|
||||
"redirect_url": "https://silo.example.com/auth/callback"
|
||||
},
|
||||
"cors": { "allowed_origins": ["https://silo.example.com"] }
|
||||
},
|
||||
"projects": { "enabled": true },
|
||||
"audit": { "enabled": true },
|
||||
"odoo": { "enabled": false, "url": "", "database": "", "username": "" },
|
||||
"freecad": { "uri_scheme": "silo", "executable": "" },
|
||||
"jobs": {
|
||||
"enabled": false,
|
||||
"directory": "/etc/silo/jobdefs",
|
||||
"runner_timeout": 90,
|
||||
"job_timeout_check": 30,
|
||||
"default_priority": 100
|
||||
},
|
||||
"dag": { "enabled": false }
|
||||
}
|
||||
```
|
||||
|
||||
### 7.2 Get Module Settings
|
||||
|
||||
```
|
||||
GET /api/admin/settings/{module}
|
||||
```
|
||||
|
||||
Returns just the module's config block.
|
||||
|
||||
### 7.3 Update Module Settings
|
||||
|
||||
```
|
||||
PUT /api/admin/settings/{module}
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"enabled": true,
|
||||
"ldap": {
|
||||
"enabled": true,
|
||||
"url": "ldaps://ipa.example.com"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"updated": ["auth.ldap.enabled", "auth.ldap.url"],
|
||||
"restart_required": false
|
||||
}
|
||||
```
|
||||
|
||||
### 7.4 Test Connectivity
|
||||
|
||||
```
|
||||
POST /api/admin/settings/{module}/test
|
||||
```
|
||||
|
||||
Available for modules with external connections:
|
||||
|
||||
| Module | Test Action |
|
||||
|--------|------------|
|
||||
| `storage` | Ping MinIO, verify bucket exists |
|
||||
| `auth` (ldap) | Attempt LDAP bind with configured credentials |
|
||||
| `auth` (oidc) | Fetch OIDC discovery document from issuer URL |
|
||||
| `odoo` | Attempt XML-RPC connection to Odoo |
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "LDAP bind successful",
|
||||
"latency_ms": 42
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Config YAML Changes
|
||||
|
||||
The existing `config.yaml` gains a `modules` section. Existing top-level keys remain for backward compatibility — the module system reads from both locations.
|
||||
|
||||
```yaml
|
||||
# Existing keys (unchanged, still work)
|
||||
server:
|
||||
host: "0.0.0.0"
|
||||
port: 8080
|
||||
|
||||
database:
|
||||
host: postgres
|
||||
port: 5432
|
||||
name: silo
|
||||
user: silo
|
||||
password: silodev
|
||||
sslmode: disable
|
||||
|
||||
storage:
|
||||
endpoint: minio:9000
|
||||
bucket: silo-files
|
||||
access_key: silominio
|
||||
secret_key: silominiosecret
|
||||
use_ssl: false
|
||||
|
||||
schemas:
|
||||
directory: /etc/silo/schemas
|
||||
|
||||
auth:
|
||||
enabled: true
|
||||
session_secret: change-me
|
||||
local:
|
||||
enabled: true
|
||||
|
||||
# New: explicit module toggles (optional, defaults shown)
|
||||
modules:
|
||||
projects:
|
||||
enabled: true
|
||||
audit:
|
||||
enabled: true
|
||||
odoo:
|
||||
enabled: false
|
||||
freecad:
|
||||
enabled: true
|
||||
uri_scheme: silo
|
||||
jobs:
|
||||
enabled: false
|
||||
directory: /etc/silo/jobdefs
|
||||
runner_timeout: 90
|
||||
job_timeout_check: 30
|
||||
default_priority: 100
|
||||
dag:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
If a module is not listed under `modules:`, its default enabled state from Section 2.2 applies. The `auth.enabled` field continues to control the `auth` module (no duplication under `modules:`).
|
||||
|
||||
---
|
||||
|
||||
## 9. SSE Events
|
||||
|
||||
```
|
||||
settings.changed {module, enabled, changed_keys[], updated_by}
|
||||
```
|
||||
|
||||
Broadcast on any admin settings change. The web UI listens for this to:
|
||||
|
||||
- Show/hide navigation entries when modules are toggled.
|
||||
- Display a "Settings updated by another admin" toast.
|
||||
- Show a "Restart required" banner when flagged.
|
||||
|
||||
---
|
||||
|
||||
## 10. Web UI — Admin Settings Page
|
||||
|
||||
The Settings page (`/settings`) is restructured into sections:
|
||||
|
||||
### 10.1 Existing (unchanged)
|
||||
|
||||
- **Account** — username, display name, email, auth source, role badge.
|
||||
- **API Tokens** — create, list, revoke.
|
||||
|
||||
### 10.2 New: Module Configuration (admin only)
|
||||
|
||||
Visible only to admin users. Each module gets a collapsible card:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ [toggle] Authentication [status] │
|
||||
├─────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ── Local Auth ──────────────────────────────────── │
|
||||
│ Enabled: [toggle] │
|
||||
│ │
|
||||
│ ── LDAP / FreeIPA ──────────────────────────────── │
|
||||
│ Enabled: [toggle] │
|
||||
│ URL: [ldaps://ipa.example.com ] │
|
||||
│ Base DN: [dc=kindred,dc=internal ] [Test] │
|
||||
│ │
|
||||
│ ── OIDC / Keycloak ────────────────────────────── │
|
||||
│ Enabled: [toggle] │
|
||||
│ Issuer URL: [https://keycloak.example.com] [Test] │
|
||||
│ Client ID: [silo ] │
|
||||
│ │
|
||||
│ ── CORS ────────────────────────────────────────── │
|
||||
│ Allowed Origins: [tag input] │
|
||||
│ │
|
||||
│ [Save] │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Module cards for required modules (`core`, `schemas`, `storage`) show their status and config but have no enable/disable toggle.
|
||||
|
||||
Status indicators per module:
|
||||
|
||||
| Status | Badge | Meaning |
|
||||
|--------|-------|---------|
|
||||
| Active | `green` | Enabled and operational |
|
||||
| Disabled | `overlay1` | Toggled off |
|
||||
| Error | `red` | Enabled but connectivity or config issue |
|
||||
| Setup Required | `yellow` | Enabled but missing required config (e.g., LDAP URL empty) |
|
||||
|
||||
### 10.3 Infrastructure Section (admin, read-only)
|
||||
|
||||
Shows connection status for required infrastructure:
|
||||
|
||||
- **Database** — host, port, name, connection pool usage, status badge.
|
||||
- **Storage** — endpoint, bucket, SSL, status badge.
|
||||
|
||||
These are read-only in the UI (setup-only via YAML/env). The "Test" button is available to verify connectivity.
|
||||
|
||||
---
|
||||
|
||||
## 11. Implementation Order
|
||||
|
||||
1. **Migration 014** — `settings_overrides` and `module_state` tables.
|
||||
2. **Config loader refactor** — YAML → DB merge → env override pipeline.
|
||||
3. **Module registry** — Go struct defining all modules with metadata, dependencies, defaults.
|
||||
4. **`GET /api/modules`** — public endpoint, no auth.
|
||||
5. **`RequireModule` middleware** — gate route groups by module state.
|
||||
6. **Admin settings API** — `GET/PUT /api/admin/settings/{module}`, test endpoints.
|
||||
7. **Web UI settings page** — module cards with toggles, config forms, test buttons.
|
||||
8. **SSE integration** — `settings.changed` event broadcast.
|
||||
|
||||
---
|
||||
|
||||
## 12. Future Considerations
|
||||
|
||||
- **Module manifest format** — per ROADMAP.md, each module will eventually declare routes, views, hooks, and permissions via a manifest. This spec covers the runtime module registry; the manifest format is TBD.
|
||||
- **Custom modules** — third-party modules that register against the endpoint registry. Requires the manifest contract and a plugin loading mechanism.
|
||||
- **Per-module permissions** — beyond the current role hierarchy, modules may define fine-grained scopes (e.g., `jobs:admin`, `dag:write`).
|
||||
- **Location & Inventory module** — when the Location/Inventory API is implemented (tables already exist), it becomes a new optional module.
|
||||
- **Notifications module** — per ROADMAP.md Tier 1, notifications/subscriptions will be a dedicated module.
|
||||
|
||||
---
|
||||
|
||||
## 13. References
|
||||
|
||||
- [CONFIGURATION.md](CONFIGURATION.md) — Current config reference
|
||||
- [ROADMAP.md](ROADMAP.md) — Module manifest, API endpoint registry
|
||||
- [AUTH.md](AUTH.md) — Authentication architecture
|
||||
- [WORKERS.md](WORKERS.md) — Job queue system
|
||||
- [DAG.md](DAG.md) — Dependency DAG specification
|
||||
- [SPECIFICATION.md](SPECIFICATION.md) — Full endpoint listing
|
||||
@@ -88,7 +88,7 @@ Everything depends on these. They define what Silo *is*.
|
||||
| Component | Description | Status |
|
||||
|-----------|-------------|--------|
|
||||
| **Core Silo** | Part/assembly storage, version control, auth, base REST API | Complete |
|
||||
| **.kc Format Spec** | File format contract between Create and Silo | Not Started |
|
||||
| **.kc Format Spec** | File format contract between Create and Silo | Complete |
|
||||
| **API Endpoint Registry** | Module discovery, dynamic UI rendering, health checks | Not Started |
|
||||
| **Web UI Shell** | App launcher, breadcrumbs, view framework, module rendering | Partial |
|
||||
| **Python Scripting Engine** | Server-side hook execution, module extension point | Not Started |
|
||||
|
||||
@@ -37,7 +37,7 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Silo Server (silod) │
|
||||
│ - REST API (78 endpoints) │
|
||||
│ - REST API (86 endpoints) │
|
||||
│ - Authentication (local, LDAP, OIDC) │
|
||||
│ - Schema parsing and validation │
|
||||
│ - Part number generation engine │
|
||||
@@ -598,7 +598,7 @@ See [AUTH.md](AUTH.md) for full architecture details and [AUTH_USER_GUIDE.md](AU
|
||||
|
||||
## 11. API Design
|
||||
|
||||
### 11.1 REST Endpoints (78 Implemented)
|
||||
### 11.1 REST Endpoints (86 Implemented)
|
||||
|
||||
```
|
||||
# Health (no auth)
|
||||
@@ -697,6 +697,20 @@ POST /api/items/{partNumber}/bom/merge # Merge BOM from ODS
|
||||
PUT /api/items/{partNumber}/bom/{childPartNumber} # Update BOM entry [editor]
|
||||
DELETE /api/items/{partNumber}/bom/{childPartNumber} # Remove BOM entry [editor]
|
||||
|
||||
# .kc Metadata (read: viewer, write: editor)
|
||||
GET /api/items/{partNumber}/metadata # Get indexed .kc metadata
|
||||
PUT /api/items/{partNumber}/metadata # Update metadata fields [editor]
|
||||
PATCH /api/items/{partNumber}/metadata/lifecycle # Transition lifecycle state [editor]
|
||||
PATCH /api/items/{partNumber}/metadata/tags # Add/remove tags [editor]
|
||||
|
||||
# .kc Dependencies (viewer)
|
||||
GET /api/items/{partNumber}/dependencies # List raw dependencies
|
||||
GET /api/items/{partNumber}/dependencies/resolve # Resolve UUIDs to part numbers + file availability
|
||||
|
||||
# .kc Macros (viewer)
|
||||
GET /api/items/{partNumber}/macros # List registered macros
|
||||
GET /api/items/{partNumber}/macros/{filename} # Get macro source content
|
||||
|
||||
# Audit (viewer)
|
||||
GET /api/audit/completeness # Item completeness scores
|
||||
GET /api/audit/completeness/{partNumber} # Item detail breakdown
|
||||
@@ -735,6 +749,139 @@ POST /api/inventory/{partNumber}/move
|
||||
|
||||
---
|
||||
|
||||
## 11.3 .kc File Integration
|
||||
|
||||
Silo supports the `.kc` file format — a ZIP archive that is a superset of FreeCAD's `.fcstd`. A `.kc` file contains everything an `.fcstd` does, plus a `silo/` directory with platform metadata.
|
||||
|
||||
#### Standard entries (preserved as-is)
|
||||
|
||||
`Document.xml`, `GuiDocument.xml`, BREP geometry files (`.brp`), `thumbnails/`
|
||||
|
||||
#### Silo entries (`silo/` directory)
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `silo/manifest.json` | Instance origin, part UUID, revision hash, `.kc` schema version |
|
||||
| `silo/metadata.json` | Custom schema field values, tags, lifecycle state |
|
||||
| `silo/history.json` | Local revision log (server-generated on checkout) |
|
||||
| `silo/dependencies.json` | Assembly link references by Silo UUID |
|
||||
| `silo/macros/*.py` | Embedded macro scripts bound to this part |
|
||||
|
||||
#### Commit-time extraction
|
||||
|
||||
When a `.kc` file is uploaded via `POST /api/items/{partNumber}/file`, the server:
|
||||
|
||||
1. Opens the ZIP and scans for `silo/` entries
|
||||
2. Parses `silo/manifest.json` and validates the UUID matches the item
|
||||
3. Upserts `silo/metadata.json` fields into the `item_metadata` table
|
||||
4. Replaces `silo/dependencies.json` entries in the `item_dependencies` table
|
||||
5. Replaces `silo/macros/*.py` entries in the `item_macros` table
|
||||
6. Broadcasts SSE events: `metadata.updated`, `dependencies.changed`, `macros.changed`
|
||||
|
||||
Extraction is best-effort — failures are logged as warnings but do not block the upload.
|
||||
|
||||
#### Checkout-time packing
|
||||
|
||||
When a `.kc` file is downloaded via `GET /api/items/{partNumber}/file/{revision}`, the server repacks the `silo/` directory with current database state:
|
||||
|
||||
- `silo/manifest.json` — current item UUID and metadata freshness
|
||||
- `silo/metadata.json` — latest schema fields, tags, lifecycle state
|
||||
- `silo/history.json` — last 20 revisions from the database
|
||||
- `silo/dependencies.json` — current dependency list from `item_dependencies`
|
||||
|
||||
Non-silo ZIP entries are passed through unchanged. If the file is a plain `.fcstd` (no `silo/` directory), it is served as-is.
|
||||
|
||||
ETag caching: the server computes an ETag from `revision_number:metadata.updated_at` and returns `304 Not Modified` when the client's `If-None-Match` header matches.
|
||||
|
||||
#### Lifecycle state machine
|
||||
|
||||
The `lifecycle_state` field in `item_metadata` follows this state machine:
|
||||
|
||||
```
|
||||
draft → review → released → obsolete
|
||||
↑ ↓
|
||||
└────────┘
|
||||
```
|
||||
|
||||
Valid transitions are enforced by `PATCH /metadata/lifecycle`. Invalid transitions return `422 Unprocessable Entity`.
|
||||
|
||||
#### Metadata response shape
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_name": "kindred-rd",
|
||||
"lifecycle_state": "draft",
|
||||
"tags": ["prototype", "v2"],
|
||||
"fields": {"material": "AL6061", "finish": "anodized"},
|
||||
"manifest": {
|
||||
"uuid": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"silo_instance": "silo.example.com",
|
||||
"revision_hash": "abc123",
|
||||
"kc_version": "1.0"
|
||||
},
|
||||
"updated_at": "2026-02-18T12:00:00Z",
|
||||
"updated_by": "forbes"
|
||||
}
|
||||
```
|
||||
|
||||
#### Dependency response shape
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"uuid": "550e8400-...",
|
||||
"part_number": "F01-0042",
|
||||
"revision": 3,
|
||||
"quantity": 4.0,
|
||||
"label": "M5 Bolt",
|
||||
"relationship": "component"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
#### Resolved dependency response shape
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"uuid": "550e8400-...",
|
||||
"part_number": "F01-0042",
|
||||
"label": "M5 Bolt",
|
||||
"revision": 3,
|
||||
"quantity": 4.0,
|
||||
"resolved": true,
|
||||
"file_available": true
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
#### Macro list response shape
|
||||
|
||||
```json
|
||||
[
|
||||
{"filename": "validate_dims.py", "trigger": "manual", "revision_number": 5}
|
||||
]
|
||||
```
|
||||
|
||||
#### Macro detail response shape
|
||||
|
||||
```json
|
||||
{
|
||||
"filename": "validate_dims.py",
|
||||
"trigger": "manual",
|
||||
"content": "import FreeCAD\n...",
|
||||
"revision_number": 5
|
||||
}
|
||||
```
|
||||
|
||||
#### Database tables (migration 018)
|
||||
|
||||
- `item_metadata` — schema fields, lifecycle state, tags, manifest info
|
||||
- `item_dependencies` — parent/child UUID references with quantity and relationship type
|
||||
- `item_macros` — filename, trigger type, source content, indexed per item
|
||||
|
||||
---
|
||||
|
||||
## 12. MVP Scope
|
||||
|
||||
### 12.1 Implemented
|
||||
@@ -743,7 +890,7 @@ POST /api/inventory/{partNumber}/move
|
||||
- [x] YAML schema parser for part numbering
|
||||
- [x] Part number generation engine
|
||||
- [x] CLI tool (`cmd/silo`)
|
||||
- [x] API server (`cmd/silod`) with 78 endpoints
|
||||
- [x] API server (`cmd/silod`) with 86 endpoints
|
||||
- [x] Filesystem-based file storage
|
||||
- [x] BOM relationships (component, alternate, reference)
|
||||
- [x] Multi-level BOM (recursive expansion with configurable depth)
|
||||
@@ -765,6 +912,12 @@ POST /api/inventory/{partNumber}/move
|
||||
- [x] Audit logging and completeness scoring
|
||||
- [x] CSRF protection (nosurf)
|
||||
- [x] Fuzzy search
|
||||
- [x] .kc file extraction pipeline (metadata, dependencies, macros indexed on commit)
|
||||
- [x] .kc file packing on checkout (manifest, metadata, history, dependencies)
|
||||
- [x] .kc metadata API (get, update fields, lifecycle transitions, tags)
|
||||
- [x] .kc dependency API (list, resolve with file availability)
|
||||
- [x] .kc macro API (list, get source content)
|
||||
- [x] ETag caching for .kc file downloads
|
||||
- [x] Property schema versioning framework
|
||||
- [x] Docker Compose deployment (dev and prod)
|
||||
- [x] systemd service and deployment scripts
|
||||
|
||||
@@ -10,10 +10,10 @@
|
||||
|
||||
| Component | Status | Notes |
|
||||
|-----------|--------|-------|
|
||||
| PostgreSQL schema | Complete | 13 migrations applied |
|
||||
| PostgreSQL schema | Complete | 18 migrations applied |
|
||||
| YAML schema parser | Complete | Supports enum, serial, constant, string segments |
|
||||
| Part number generator | Complete | Scoped sequences, category-based format |
|
||||
| API server (`silod`) | Complete | 78 REST endpoints via chi/v5 |
|
||||
| API server (`silod`) | Complete | 86 REST endpoints via chi/v5 |
|
||||
| CLI tool (`silo`) | Complete | Item registration and management |
|
||||
| Filesystem file storage | Complete | Upload, download, checksums |
|
||||
| Revision control | Complete | Append-only history, rollback, comparison, status/labels |
|
||||
@@ -30,6 +30,11 @@
|
||||
| Fuzzy search | Complete | sahilm/fuzzy library |
|
||||
| Web UI | Complete | React SPA (Vite + TypeScript), 6 pages, Catppuccin Mocha theme |
|
||||
| File attachments | Complete | Direct uploads, item file association, thumbnails |
|
||||
| .kc extraction pipeline | Complete | Metadata, dependencies, macros indexed on commit |
|
||||
| .kc checkout packing | Complete | Manifest, metadata, history, dependencies repacked on download |
|
||||
| .kc metadata API | Complete | GET/PUT metadata, lifecycle transitions, tag management |
|
||||
| .kc dependency API | Complete | List raw deps, resolve UUIDs to part numbers + file availability |
|
||||
| .kc macro API | Complete | List macros, get source content by filename |
|
||||
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull are stubs |
|
||||
| Docker Compose | Complete | Dev and production configurations |
|
||||
| Deployment scripts | Complete | setup-host, deploy, init-db, setup-ipa-nginx |
|
||||
@@ -96,3 +101,8 @@ The schema defines 170 category codes across 10 groups:
|
||||
| 011_item_files.sql | Item file attachments (item_files table, thumbnail_key column) |
|
||||
| 012_bom_source.sql | BOM entry source tracking |
|
||||
| 013_move_cost_sourcing_to_props.sql | Move sourcing_link and standard_cost from item columns to revision properties |
|
||||
| 014_settings.sql | Settings overrides and module state tables |
|
||||
| 015_jobs.sql | Job queue, runner, and job log tables |
|
||||
| 016_dag.sql | Dependency DAG nodes and edges |
|
||||
| 017_locations.sql | Location hierarchy and inventory tracking |
|
||||
| 018_kc_metadata.sql | .kc metadata tables (item_metadata, item_dependencies, item_macros, item_approvals, approval_signatures) |
|
||||
|
||||
@@ -57,6 +57,7 @@ type Server struct {
|
||||
settings *db.SettingsRepository
|
||||
metadata *db.ItemMetadataRepository
|
||||
deps *db.ItemDependencyRepository
|
||||
macros *db.ItemMacroRepository
|
||||
}
|
||||
|
||||
// NewServer creates a new API server.
|
||||
@@ -87,6 +88,7 @@ func NewServer(
|
||||
locations := db.NewLocationRepository(database)
|
||||
metadata := db.NewItemMetadataRepository(database)
|
||||
itemDeps := db.NewItemDependencyRepository(database)
|
||||
itemMacros := db.NewItemMacroRepository(database)
|
||||
seqStore := &dbSequenceStore{db: database, schemas: schemas}
|
||||
partgen := partnum.NewGenerator(schemas, seqStore)
|
||||
|
||||
@@ -117,6 +119,7 @@ func NewServer(
|
||||
settings: settings,
|
||||
metadata: metadata,
|
||||
deps: itemDeps,
|
||||
macros: itemMacros,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
95
internal/api/macro_handlers.go
Normal file
95
internal/api/macro_handlers.go
Normal file
@@ -0,0 +1,95 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
)
|
||||
|
||||
// MacroListItem is the JSON representation for GET /macros list entries.
|
||||
type MacroListItem struct {
|
||||
Filename string `json:"filename"`
|
||||
Trigger string `json:"trigger"`
|
||||
RevisionNumber int `json:"revision_number"`
|
||||
}
|
||||
|
||||
// MacroResponse is the JSON representation for GET /macros/{filename}.
|
||||
type MacroResponse struct {
|
||||
Filename string `json:"filename"`
|
||||
Trigger string `json:"trigger"`
|
||||
Content string `json:"content"`
|
||||
RevisionNumber int `json:"revision_number"`
|
||||
}
|
||||
|
||||
// HandleGetMacros returns the list of registered macros for an item.
|
||||
// GET /api/items/{partNumber}/macros
|
||||
func (s *Server) HandleGetMacros(w http.ResponseWriter, r *http.Request) {
|
||||
ctx := r.Context()
|
||||
partNumber := chi.URLParam(r, "partNumber")
|
||||
|
||||
item, err := s.items.GetByPartNumber(ctx, partNumber)
|
||||
if err != nil {
|
||||
s.logger.Error().Err(err).Msg("failed to get item")
|
||||
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
|
||||
return
|
||||
}
|
||||
if item == nil {
|
||||
writeError(w, http.StatusNotFound, "not_found", "Item not found")
|
||||
return
|
||||
}
|
||||
|
||||
macros, err := s.macros.ListByItem(ctx, item.ID)
|
||||
if err != nil {
|
||||
s.logger.Error().Err(err).Msg("failed to list macros")
|
||||
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to list macros")
|
||||
return
|
||||
}
|
||||
|
||||
resp := make([]MacroListItem, len(macros))
|
||||
for i, m := range macros {
|
||||
resp[i] = MacroListItem{
|
||||
Filename: m.Filename,
|
||||
Trigger: m.Trigger,
|
||||
RevisionNumber: m.RevisionNumber,
|
||||
}
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusOK, resp)
|
||||
}
|
||||
|
||||
// HandleGetMacro returns a single macro's source content.
|
||||
// GET /api/items/{partNumber}/macros/{filename}
|
||||
func (s *Server) HandleGetMacro(w http.ResponseWriter, r *http.Request) {
|
||||
ctx := r.Context()
|
||||
partNumber := chi.URLParam(r, "partNumber")
|
||||
filename := chi.URLParam(r, "filename")
|
||||
|
||||
item, err := s.items.GetByPartNumber(ctx, partNumber)
|
||||
if err != nil {
|
||||
s.logger.Error().Err(err).Msg("failed to get item")
|
||||
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get item")
|
||||
return
|
||||
}
|
||||
if item == nil {
|
||||
writeError(w, http.StatusNotFound, "not_found", "Item not found")
|
||||
return
|
||||
}
|
||||
|
||||
macro, err := s.macros.GetByFilename(ctx, item.ID, filename)
|
||||
if err != nil {
|
||||
s.logger.Error().Err(err).Msg("failed to get macro")
|
||||
writeError(w, http.StatusInternalServerError, "internal_error", "Failed to get macro")
|
||||
return
|
||||
}
|
||||
if macro == nil {
|
||||
writeError(w, http.StatusNotFound, "not_found", "Macro not found")
|
||||
return
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusOK, MacroResponse{
|
||||
Filename: macro.Filename,
|
||||
Trigger: macro.Trigger,
|
||||
Content: macro.Content,
|
||||
RevisionNumber: macro.RevisionNumber,
|
||||
})
|
||||
}
|
||||
@@ -430,6 +430,27 @@ func (s *Server) extractKCMetadata(ctx context.Context, item *db.Item, fileKey s
|
||||
}
|
||||
}
|
||||
|
||||
// Index macros from silo/macros/*.
|
||||
if len(result.Macros) > 0 {
|
||||
dbMacros := make([]*db.ItemMacro, len(result.Macros))
|
||||
for i, m := range result.Macros {
|
||||
dbMacros[i] = &db.ItemMacro{
|
||||
ItemID: item.ID,
|
||||
Filename: m.Filename,
|
||||
Trigger: "manual",
|
||||
Content: m.Content,
|
||||
}
|
||||
}
|
||||
if err := s.macros.ReplaceForItem(ctx, item.ID, rev.RevisionNumber, dbMacros); err != nil {
|
||||
s.logger.Warn().Err(err).Str("part_number", item.PartNumber).Msg("kc: failed to index macros")
|
||||
} else {
|
||||
s.broker.Publish("macros.changed", mustMarshal(map[string]any{
|
||||
"part_number": item.PartNumber,
|
||||
"count": len(dbMacros),
|
||||
}))
|
||||
}
|
||||
}
|
||||
|
||||
s.logger.Info().Str("part_number", item.PartNumber).Msg("kc: metadata indexed successfully")
|
||||
}
|
||||
|
||||
|
||||
@@ -175,6 +175,8 @@ func NewRouter(server *Server, logger zerolog.Logger) http.Handler {
|
||||
r.Get("/metadata", server.HandleGetMetadata)
|
||||
r.Get("/dependencies", server.HandleGetDependencies)
|
||||
r.Get("/dependencies/resolve", server.HandleResolveDependencies)
|
||||
r.Get("/macros", server.HandleGetMacros)
|
||||
r.Get("/macros/{filename}", server.HandleGetMacro)
|
||||
|
||||
// DAG (gated by dag module)
|
||||
r.Route("/dag", func(r chi.Router) {
|
||||
|
||||
93
internal/db/item_macros.go
Normal file
93
internal/db/item_macros.go
Normal file
@@ -0,0 +1,93 @@
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
// ItemMacro represents a row in the item_macros table.
|
||||
type ItemMacro struct {
|
||||
ID string
|
||||
ItemID string
|
||||
Filename string
|
||||
Trigger string
|
||||
Content string
|
||||
RevisionNumber int
|
||||
CreatedAt time.Time
|
||||
}
|
||||
|
||||
// ItemMacroRepository provides item_macros database operations.
|
||||
type ItemMacroRepository struct {
|
||||
db *DB
|
||||
}
|
||||
|
||||
// NewItemMacroRepository creates a new item macro repository.
|
||||
func NewItemMacroRepository(db *DB) *ItemMacroRepository {
|
||||
return &ItemMacroRepository{db: db}
|
||||
}
|
||||
|
||||
// ReplaceForItem atomically replaces all macros for an item.
|
||||
// Deletes existing rows and inserts the new set.
|
||||
func (r *ItemMacroRepository) ReplaceForItem(ctx context.Context, itemID string, revisionNumber int, macros []*ItemMacro) error {
|
||||
return r.db.Tx(ctx, func(tx pgx.Tx) error {
|
||||
_, err := tx.Exec(ctx, `DELETE FROM item_macros WHERE item_id = $1`, itemID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("deleting old macros: %w", err)
|
||||
}
|
||||
|
||||
for _, m := range macros {
|
||||
_, err := tx.Exec(ctx, `
|
||||
INSERT INTO item_macros (item_id, filename, trigger, content, revision_number)
|
||||
VALUES ($1, $2, $3, $4, $5)
|
||||
`, itemID, m.Filename, m.Trigger, m.Content, revisionNumber)
|
||||
if err != nil {
|
||||
return fmt.Errorf("inserting macro %s: %w", m.Filename, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// ListByItem returns all macros for an item (without content), ordered by filename.
|
||||
func (r *ItemMacroRepository) ListByItem(ctx context.Context, itemID string) ([]*ItemMacro, error) {
|
||||
rows, err := r.db.pool.Query(ctx, `
|
||||
SELECT id, item_id, filename, trigger, revision_number, created_at
|
||||
FROM item_macros
|
||||
WHERE item_id = $1
|
||||
ORDER BY filename
|
||||
`, itemID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("listing macros: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var macros []*ItemMacro
|
||||
for rows.Next() {
|
||||
m := &ItemMacro{}
|
||||
if err := rows.Scan(&m.ID, &m.ItemID, &m.Filename, &m.Trigger, &m.RevisionNumber, &m.CreatedAt); err != nil {
|
||||
return nil, fmt.Errorf("scanning macro: %w", err)
|
||||
}
|
||||
macros = append(macros, m)
|
||||
}
|
||||
return macros, nil
|
||||
}
|
||||
|
||||
// GetByFilename returns a single macro by item ID and filename, including content.
|
||||
func (r *ItemMacroRepository) GetByFilename(ctx context.Context, itemID string, filename string) (*ItemMacro, error) {
|
||||
m := &ItemMacro{}
|
||||
err := r.db.pool.QueryRow(ctx, `
|
||||
SELECT id, item_id, filename, trigger, content, revision_number, created_at
|
||||
FROM item_macros
|
||||
WHERE item_id = $1 AND filename = $2
|
||||
`, itemID, filename).Scan(&m.ID, &m.ItemID, &m.Filename, &m.Trigger, &m.Content, &m.RevisionNumber, &m.CreatedAt)
|
||||
if err != nil {
|
||||
if err == pgx.ErrNoRows {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, fmt.Errorf("getting macro: %w", err)
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
@@ -40,11 +40,18 @@ type Dependency struct {
|
||||
Relationship string `json:"relationship"`
|
||||
}
|
||||
|
||||
// MacroFile represents a script file found under silo/macros/.
|
||||
type MacroFile struct {
|
||||
Filename string
|
||||
Content string
|
||||
}
|
||||
|
||||
// ExtractResult holds the parsed silo/ directory contents from a .kc file.
|
||||
type ExtractResult struct {
|
||||
Manifest *Manifest
|
||||
Metadata *Metadata
|
||||
Dependencies []Dependency
|
||||
Macros []MacroFile
|
||||
}
|
||||
|
||||
// HistoryEntry represents one entry in silo/history.json.
|
||||
@@ -76,6 +83,7 @@ func Extract(data []byte) (*ExtractResult, error) {
|
||||
}
|
||||
|
||||
var manifestFile, metadataFile, dependenciesFile *zip.File
|
||||
var macroFiles []*zip.File
|
||||
hasSiloDir := false
|
||||
|
||||
for _, f := range r.File {
|
||||
@@ -89,6 +97,13 @@ func Extract(data []byte) (*ExtractResult, error) {
|
||||
metadataFile = f
|
||||
case "silo/dependencies.json":
|
||||
dependenciesFile = f
|
||||
default:
|
||||
if strings.HasPrefix(f.Name, "silo/macros/") && !f.FileInfo().IsDir() {
|
||||
name := strings.TrimPrefix(f.Name, "silo/macros/")
|
||||
if name != "" {
|
||||
macroFiles = append(macroFiles, f)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -124,6 +139,22 @@ func Extract(data []byte) (*ExtractResult, error) {
|
||||
}
|
||||
}
|
||||
|
||||
for _, mf := range macroFiles {
|
||||
rc, err := mf.Open()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("kc: open macro %s: %w", mf.Name, err)
|
||||
}
|
||||
content, err := io.ReadAll(rc)
|
||||
rc.Close()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("kc: read macro %s: %w", mf.Name, err)
|
||||
}
|
||||
result.Macros = append(result.Macros, MacroFile{
|
||||
Filename: strings.TrimPrefix(mf.Name, "silo/macros/"),
|
||||
Content: string(content),
|
||||
})
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -1,110 +0,0 @@
|
||||
-- Migration 018: .kc Server-Side Metadata Tables
|
||||
--
|
||||
-- Adds tables for indexing the silo/ directory contents from .kc files.
|
||||
-- See docs/KC_SERVER.md for the full specification.
|
||||
--
|
||||
-- Tables:
|
||||
-- item_metadata - indexed manifest + metadata fields (Section 3.1)
|
||||
-- item_dependencies - CAD-extracted assembly dependencies (Section 3.2)
|
||||
-- item_approvals - ECO workflow state (Section 3.3)
|
||||
-- approval_signatures - individual approval/rejection records (Section 3.3)
|
||||
-- item_macros - registered macros from silo/macros/ (Section 3.4)
|
||||
|
||||
BEGIN;
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
-- item_metadata: indexed silo/manifest.json + silo/metadata.json
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE item_metadata (
|
||||
item_id UUID PRIMARY KEY REFERENCES items(id) ON DELETE CASCADE,
|
||||
schema_name TEXT,
|
||||
tags TEXT[] NOT NULL DEFAULT '{}',
|
||||
lifecycle_state TEXT NOT NULL DEFAULT 'draft',
|
||||
fields JSONB NOT NULL DEFAULT '{}',
|
||||
kc_version TEXT,
|
||||
manifest_uuid UUID,
|
||||
silo_instance TEXT,
|
||||
revision_hash TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_by TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX idx_item_metadata_tags ON item_metadata USING GIN (tags);
|
||||
CREATE INDEX idx_item_metadata_lifecycle ON item_metadata (lifecycle_state);
|
||||
CREATE INDEX idx_item_metadata_fields ON item_metadata USING GIN (fields);
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
-- item_dependencies: indexed silo/dependencies.json
|
||||
--
|
||||
-- Complements the existing `relationships` table.
|
||||
-- relationships = server-authoritative BOM (web UI / API editable)
|
||||
-- item_dependencies = CAD-authoritative record (extracted from .kc)
|
||||
-- BOM merge reconciles the two (see docs/BOM_MERGE.md).
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE item_dependencies (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
parent_item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE,
|
||||
child_uuid UUID NOT NULL,
|
||||
child_part_number TEXT,
|
||||
child_revision INTEGER,
|
||||
quantity DECIMAL,
|
||||
label TEXT,
|
||||
relationship TEXT NOT NULL DEFAULT 'component',
|
||||
revision_number INTEGER NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_item_deps_parent ON item_dependencies (parent_item_id);
|
||||
CREATE INDEX idx_item_deps_child ON item_dependencies (child_uuid);
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
-- item_approvals + approval_signatures: ECO workflow
|
||||
--
|
||||
-- Server-authoritative. The .kc silo/approvals.json is a read cache
|
||||
-- packed on checkout for offline display in Create.
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE item_approvals (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE,
|
||||
eco_number TEXT,
|
||||
state TEXT NOT NULL DEFAULT 'draft',
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_by TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX idx_item_approvals_item ON item_approvals (item_id);
|
||||
CREATE INDEX idx_item_approvals_state ON item_approvals (state);
|
||||
|
||||
CREATE TABLE approval_signatures (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
approval_id UUID NOT NULL REFERENCES item_approvals(id) ON DELETE CASCADE,
|
||||
username TEXT NOT NULL,
|
||||
role TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
signed_at TIMESTAMPTZ,
|
||||
comment TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX idx_approval_sigs_approval ON approval_signatures (approval_id);
|
||||
|
||||
--------------------------------------------------------------------------------
|
||||
-- item_macros: registered macros from silo/macros/
|
||||
--------------------------------------------------------------------------------
|
||||
|
||||
CREATE TABLE item_macros (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
trigger TEXT NOT NULL DEFAULT 'manual',
|
||||
content TEXT NOT NULL,
|
||||
revision_number INTEGER NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE(item_id, filename)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_item_macros_item ON item_macros (item_id);
|
||||
|
||||
COMMIT;
|
||||
Reference in New Issue
Block a user