Merge pull request 'docs: add Silo server documentation with auto-sync workflow (#122)' (#123) from docs/sync-silo-server-docs into main
Reviewed-on: #123
This commit was merged in pull request #123.
This commit is contained in:
@@ -6,6 +6,7 @@ on:
|
||||
paths:
|
||||
- "docs/**"
|
||||
- ".gitea/workflows/docs.yml"
|
||||
- ".gitea/workflows/sync-silo-docs.yml"
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
@@ -30,6 +31,20 @@ jobs:
|
||||
rm mdbook.tar.gz
|
||||
fi
|
||||
|
||||
- name: Fetch Silo server docs
|
||||
run: |
|
||||
git clone --depth 1 --filter=blob:none --sparse \
|
||||
http://gitea:3000/kindred/silo.git /tmp/silo
|
||||
cd /tmp/silo
|
||||
git sparse-checkout set docs README.md ROADMAP.md frontend-spec.md
|
||||
cd -
|
||||
mkdir -p docs/src/silo-server
|
||||
cp /tmp/silo/docs/*.md docs/src/silo-server/
|
||||
cp /tmp/silo/README.md docs/src/silo-server/overview.md
|
||||
cp /tmp/silo/ROADMAP.md docs/src/silo-server/ROADMAP.md 2>/dev/null || true
|
||||
cp /tmp/silo/frontend-spec.md docs/src/silo-server/frontend-spec.md 2>/dev/null || true
|
||||
rm -rf /tmp/silo
|
||||
|
||||
- name: Build mdBook
|
||||
run: mdbook build docs/
|
||||
|
||||
|
||||
62
.gitea/workflows/sync-silo-docs.yml
Normal file
62
.gitea/workflows/sync-silo-docs.yml
Normal file
@@ -0,0 +1,62 @@
|
||||
name: Sync Silo Server Docs
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: "0 6 * * *" # Daily at 06:00 UTC as fallback
|
||||
|
||||
jobs:
|
||||
sync:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
env:
|
||||
DEBIAN_FRONTEND: noninteractive
|
||||
|
||||
steps:
|
||||
- name: Trust Cloudflare origin CA
|
||||
run: |
|
||||
apt-get update -qq
|
||||
apt-get install -y --no-install-recommends ca-certificates git
|
||||
update-ca-certificates
|
||||
|
||||
- name: Checkout create repo
|
||||
uses: https://git.kindred-systems.com/actions/checkout.git@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
token: ${{ secrets.RELEASE_TOKEN }}
|
||||
|
||||
- name: Clone Silo server docs
|
||||
run: |
|
||||
git clone --depth 1 --filter=blob:none --sparse \
|
||||
https://git.kindred-systems.com/kindred/silo.git /tmp/silo
|
||||
cd /tmp/silo
|
||||
git sparse-checkout set docs README.md ROADMAP.md frontend-spec.md
|
||||
|
||||
- name: Copy docs into create repo
|
||||
run: |
|
||||
mkdir -p docs/src/silo-server
|
||||
cp /tmp/silo/docs/*.md docs/src/silo-server/
|
||||
cp /tmp/silo/README.md docs/src/silo-server/overview.md
|
||||
cp /tmp/silo/ROADMAP.md docs/src/silo-server/ROADMAP.md 2>/dev/null || true
|
||||
cp /tmp/silo/frontend-spec.md docs/src/silo-server/frontend-spec.md 2>/dev/null || true
|
||||
rm -rf /tmp/silo
|
||||
|
||||
- name: Check for changes
|
||||
id: diff
|
||||
run: |
|
||||
git add docs/src/silo-server/
|
||||
if git diff --cached --quiet; then
|
||||
echo "changed=false" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "changed=true" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
- name: Commit and push
|
||||
if: steps.diff.outputs.changed == 'true'
|
||||
run: |
|
||||
git config user.name "Kindred Bot"
|
||||
git config user.email "bot@kindred-systems.com"
|
||||
git commit -m "docs: sync Silo server documentation
|
||||
|
||||
Auto-synced from kindred/silo main branch."
|
||||
git push origin main
|
||||
@@ -27,6 +27,21 @@
|
||||
- [Repository Structure](./development/repo-structure.md)
|
||||
- [Build System](./development/build-system.md)
|
||||
|
||||
# Silo Server
|
||||
|
||||
- [Overview](./silo-server/overview.md)
|
||||
- [Specification](./silo-server/SPECIFICATION.md)
|
||||
- [Configuration](./silo-server/CONFIGURATION.md)
|
||||
- [Deployment](./silo-server/DEPLOYMENT.md)
|
||||
- [Authentication](./silo-server/AUTH.md)
|
||||
- [Middleware](./silo-server/AUTH_MIDDLEWARE.md)
|
||||
- [User Guide](./silo-server/AUTH_USER_GUIDE.md)
|
||||
- [BOM Merge](./silo-server/BOM_MERGE.md)
|
||||
- [Calculated Fields](./silo-server/CALC_EXTENSION.md)
|
||||
- [Component Audit](./silo-server/COMPONENT_AUDIT.md)
|
||||
- [Status System](./silo-server/STATUS.md)
|
||||
- [Gap Analysis](./silo-server/GAP_ANALYSIS.md)
|
||||
|
||||
# Reference
|
||||
|
||||
- [Configuration](./reference/configuration.md)
|
||||
|
||||
226
docs/src/silo-server/AUTH.md
Normal file
226
docs/src/silo-server/AUTH.md
Normal file
@@ -0,0 +1,226 @@
|
||||
# Silo Authentication Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
Silo supports three authentication backends that can be enabled independently or in combination:
|
||||
|
||||
| Backend | Use Case | Config Key |
|
||||
|---------|----------|------------|
|
||||
| **Local** | Username/password stored in Silo's database (bcrypt) | `auth.local` |
|
||||
| **LDAP** | FreeIPA / Active Directory via LDAP bind | `auth.ldap` |
|
||||
| **OIDC** | Keycloak or any OpenID Connect provider (redirect flow) | `auth.oidc` |
|
||||
|
||||
When authentication is disabled (`auth.enabled: false`), all routes are open and a synthetic developer user with the `admin` role is injected into every request.
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
### Browser (Session-Based)
|
||||
|
||||
```
|
||||
User -> /login (GET) -> Renders login form
|
||||
User -> /login (POST) -> Validates credentials via backends
|
||||
-> Creates server-side session (PostgreSQL)
|
||||
-> Sets silo_session cookie
|
||||
-> Redirects to / or ?next= URL
|
||||
|
||||
User -> /auth/oidc (GET) -> Generates state, stores in session
|
||||
-> Redirects to Keycloak authorize endpoint
|
||||
Keycloak -> /auth/callback -> Verifies state, exchanges code for token
|
||||
-> Extracts claims, upserts user in DB
|
||||
-> Creates session, redirects to /
|
||||
```
|
||||
|
||||
### API Client (Token-Based)
|
||||
|
||||
```
|
||||
Client -> Authorization: Bearer silo_<hex> -> SHA-256 hash lookup
|
||||
-> Validates expiry + user active
|
||||
-> Injects user into context
|
||||
```
|
||||
|
||||
Both paths converge at the `RequireAuth` middleware, which injects an `auth.User` into the request context. All downstream handlers use `auth.UserFromContext(ctx)` to access the authenticated user.
|
||||
|
||||
## Database Schema
|
||||
|
||||
### users
|
||||
|
||||
The single identity table that all backends resolve to. LDAP and OIDC users are upserted on first login.
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| `id` | UUID | Primary key |
|
||||
| `username` | TEXT | Unique. FreeIPA `uid` or OIDC `preferred_username` |
|
||||
| `display_name` | TEXT | Human-readable name |
|
||||
| `email` | TEXT | Not unique-constrained |
|
||||
| `password_hash` | TEXT | NULL for LDAP/OIDC-only users (bcrypt cost 12) |
|
||||
| `auth_source` | TEXT | `local`, `ldap`, or `oidc` |
|
||||
| `oidc_subject` | TEXT | Stable OIDC `sub` claim (unique partial index) |
|
||||
| `role` | TEXT | `admin`, `editor`, or `viewer` |
|
||||
| `is_active` | BOOLEAN | Deactivated users are rejected at middleware |
|
||||
| `last_login_at` | TIMESTAMPTZ | Updated on each successful login |
|
||||
| `created_at` | TIMESTAMPTZ | |
|
||||
| `updated_at` | TIMESTAMPTZ | |
|
||||
|
||||
### api_tokens
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| `id` | UUID | Primary key |
|
||||
| `user_id` | UUID | FK to users, CASCADE on delete |
|
||||
| `name` | TEXT | Human-readable label |
|
||||
| `token_hash` | TEXT | SHA-256 of raw token (unique) |
|
||||
| `token_prefix` | TEXT | `silo_` + 8 hex chars for display |
|
||||
| `scopes` | TEXT[] | Reserved for future fine-grained permissions |
|
||||
| `last_used_at` | TIMESTAMPTZ | Updated asynchronously on use |
|
||||
| `expires_at` | TIMESTAMPTZ | NULL = never expires |
|
||||
| `created_at` | TIMESTAMPTZ | |
|
||||
|
||||
Raw token format: `silo_` + 64 hex characters (32 random bytes). Shown once at creation. Only the SHA-256 hash is stored.
|
||||
|
||||
### sessions
|
||||
|
||||
Required by `alexedwards/scs` pgxstore:
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| `token` | TEXT | Primary key (session ID) |
|
||||
| `data` | BYTEA | Serialized session data |
|
||||
| `expiry` | TIMESTAMPTZ | Indexed for cleanup |
|
||||
|
||||
Session data contains `user_id` and `username`. Cookie: `silo_session`, HttpOnly, SameSite=Lax, 24h lifetime. `Secure` flag is set when `auth.enabled` is true.
|
||||
|
||||
### audit_log
|
||||
|
||||
| Column | Type | Notes |
|
||||
|--------|------|-------|
|
||||
| `id` | UUID | Primary key |
|
||||
| `timestamp` | TIMESTAMPTZ | Indexed DESC |
|
||||
| `user_id` | UUID | FK to users, SET NULL on delete |
|
||||
| `username` | TEXT | Preserved after user deletion |
|
||||
| `action` | TEXT | `create`, `update`, `delete`, `login`, etc. |
|
||||
| `resource_type` | TEXT | `item`, `revision`, `project`, `relationship` |
|
||||
| `resource_id` | TEXT | |
|
||||
| `details` | JSONB | Arbitrary structured data |
|
||||
| `ip_address` | TEXT | |
|
||||
|
||||
### User Tracking Columns
|
||||
|
||||
Migration 009 adds `created_by` and `updated_by` TEXT columns to:
|
||||
|
||||
- `items` (created_by, updated_by)
|
||||
- `relationships` (created_by, updated_by)
|
||||
- `projects` (created_by)
|
||||
- `sync_log` (triggered_by)
|
||||
|
||||
These store the `username` string (not a foreign key) so audit records survive user deletion and dev mode uses `"dev"`.
|
||||
|
||||
## Role Model
|
||||
|
||||
Three roles with a strict hierarchy:
|
||||
|
||||
```
|
||||
admin > editor > viewer
|
||||
```
|
||||
|
||||
| Permission | viewer | editor | admin |
|
||||
|-----------|--------|--------|-------|
|
||||
| Read items, projects, schemas, BOMs | Yes | Yes | Yes |
|
||||
| Create/update items and revisions | No | Yes | Yes |
|
||||
| Upload files | No | Yes | Yes |
|
||||
| Manage BOMs | No | Yes | Yes |
|
||||
| Create/update projects | No | Yes | Yes |
|
||||
| Import CSV / BOM CSV | No | Yes | Yes |
|
||||
| Generate part numbers | No | Yes | Yes |
|
||||
| Manage own API tokens | Yes | Yes | Yes |
|
||||
| User management (future) | No | No | Yes |
|
||||
|
||||
Role enforcement uses `auth.RoleSatisfies(userRole, minimumRole)` which checks the hierarchy. A user with `admin` satisfies any minimum role.
|
||||
|
||||
### Role Mapping from External Sources
|
||||
|
||||
**LDAP/FreeIPA**: Mapped from group membership. Checked in priority order (admin > editor > viewer). First match wins.
|
||||
|
||||
```yaml
|
||||
ldap:
|
||||
role_mapping:
|
||||
admin:
|
||||
- "cn=silo-admins,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
editor:
|
||||
- "cn=silo-users,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
- "cn=engineers,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
viewer:
|
||||
- "cn=silo-viewers,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
```
|
||||
|
||||
**OIDC/Keycloak**: Mapped from `realm_access.roles` in the ID token.
|
||||
|
||||
```yaml
|
||||
oidc:
|
||||
admin_role: "silo-admin"
|
||||
editor_role: "silo-editor"
|
||||
default_role: "viewer"
|
||||
```
|
||||
|
||||
Roles are re-evaluated on every external login. If an LDAP user's group membership changes in FreeIPA, their Silo role updates on their next login.
|
||||
|
||||
## User Lifecycle
|
||||
|
||||
| Event | What Happens |
|
||||
|-------|-------------|
|
||||
| First LDAP login | User row created with `auth_source=ldap`, role from group mapping |
|
||||
| First OIDC login | User row created with `auth_source=oidc`, `oidc_subject` set |
|
||||
| Subsequent external login | `display_name`, `email`, `role` updated; `last_login_at` updated |
|
||||
| Local account creation | Admin creates user with username, password, role |
|
||||
| Deactivation | `is_active=false` — sessions and tokens rejected at middleware |
|
||||
| Password change | Only for `auth_source=local` users |
|
||||
|
||||
## Default Admin Account
|
||||
|
||||
On startup, if `auth.local.default_admin_username` and `auth.local.default_admin_password` are both set, the daemon checks for an existing user with that username. If none exists, it creates a local admin account. This is idempotent — subsequent startups skip creation.
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
local:
|
||||
enabled: true
|
||||
default_admin_username: "admin"
|
||||
default_admin_password: "" # Use SILO_ADMIN_PASSWORD env var
|
||||
```
|
||||
|
||||
Set via environment variables for production:
|
||||
|
||||
```sh
|
||||
export SILO_ADMIN_USERNAME=admin
|
||||
export SILO_ADMIN_PASSWORD=<strong-password>
|
||||
```
|
||||
|
||||
## API Token Security
|
||||
|
||||
- Raw token: `silo_` + 64 hex characters (32 bytes of `crypto/rand`)
|
||||
- Storage: SHA-256 hash only — the raw token cannot be recovered from the database
|
||||
- Display prefix: `silo_` + first 8 hex characters (for identification in UI)
|
||||
- Tokens inherit the owning user's role. If the user is deactivated, all their tokens stop working
|
||||
- Revocation is immediate (row deletion)
|
||||
- `last_used_at` is updated asynchronously to avoid slowing down API requests
|
||||
|
||||
## Dev Mode
|
||||
|
||||
When `auth.enabled: false`:
|
||||
|
||||
- No login is required
|
||||
- A synthetic user is injected into every request:
|
||||
- Username: `dev`
|
||||
- Role: `admin`
|
||||
- Auth source: `local`
|
||||
- `created_by` fields are set to `"dev"`
|
||||
- CORS allows all origins
|
||||
- Session cookies are not marked `Secure`
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- **Password hashing**: bcrypt with cost factor 12
|
||||
- **Token entropy**: 256 bits (32 bytes from `crypto/rand`)
|
||||
- **LDAP**: Always use `ldaps://` (TLS). `tls_skip_verify` is available but should only be used for testing
|
||||
- **OIDC state parameter**: 128 bits, stored server-side in session, verified on callback
|
||||
- **Session cookies**: HttpOnly, SameSite=Lax, Secure when auth enabled
|
||||
- **CSRF protection**: nosurf library on all web form routes. API routes exempt (use Bearer tokens instead)
|
||||
- **CORS**: Locked down to configured origins when auth is enabled. Credentials allowed for session cookies
|
||||
185
docs/src/silo-server/AUTH_MIDDLEWARE.md
Normal file
185
docs/src/silo-server/AUTH_MIDDLEWARE.md
Normal file
@@ -0,0 +1,185 @@
|
||||
# Silo Auth Middleware
|
||||
|
||||
## Middleware Chain
|
||||
|
||||
Every request passes through this middleware stack in order:
|
||||
|
||||
```
|
||||
RequestID Assigns X-Request-ID header
|
||||
|
|
||||
RealIP Extracts client IP from X-Forwarded-For
|
||||
|
|
||||
RequestLogger Logs method, path, status, duration (zerolog)
|
||||
|
|
||||
Recoverer Catches panics, logs stack trace, returns 500
|
||||
|
|
||||
CORS Validates origin, sets Access-Control headers
|
||||
|
|
||||
SessionLoadAndSave Loads session from PostgreSQL, saves on response
|
||||
|
|
||||
[route group middleware applied per-group below]
|
||||
```
|
||||
|
||||
## Route Groups
|
||||
|
||||
### Public (no auth)
|
||||
|
||||
```
|
||||
GET /health
|
||||
GET /ready
|
||||
GET /login
|
||||
POST /login
|
||||
POST /logout
|
||||
GET /auth/oidc
|
||||
GET /auth/callback
|
||||
```
|
||||
|
||||
No authentication or CSRF middleware. The login form includes a CSRF hidden field but nosurf is not enforced on these routes (they are outside the CSRF-protected group).
|
||||
|
||||
### Web UI (auth + CSRF)
|
||||
|
||||
```
|
||||
RequireAuth -> CSRFProtect -> Handler
|
||||
|
||||
GET / Items page
|
||||
GET /projects Projects page
|
||||
GET /schemas Schemas page
|
||||
GET /settings Settings page (account info, tokens)
|
||||
POST /settings/tokens Create API token (form)
|
||||
POST /settings/tokens/{id}/revoke Revoke token (form)
|
||||
```
|
||||
|
||||
Both `RequireAuth` and `CSRFProtect` are applied. Form submissions must include a `csrf_token` hidden field with the value from `nosurf.Token(r)`.
|
||||
|
||||
### API (auth, no CSRF)
|
||||
|
||||
```
|
||||
RequireAuth -> [RequireRole where needed] -> Handler
|
||||
|
||||
GET /api/auth/me Current user info (viewer)
|
||||
GET /api/auth/tokens List tokens (viewer)
|
||||
POST /api/auth/tokens Create token (viewer)
|
||||
DELETE /api/auth/tokens/{id} Revoke token (viewer)
|
||||
|
||||
GET /api/schemas/* Read schemas (viewer)
|
||||
POST /api/schemas/*/segments/* Modify schema values (editor)
|
||||
|
||||
GET /api/projects/* Read projects (viewer)
|
||||
POST /api/projects Create project (editor)
|
||||
PUT /api/projects/{code} Update project (editor)
|
||||
DELETE /api/projects/{code} Delete project (editor)
|
||||
|
||||
GET /api/items/* Read items (viewer)
|
||||
POST /api/items Create item (editor)
|
||||
PUT /api/items/{partNumber} Update item (editor)
|
||||
DELETE /api/items/{partNumber} Delete item (editor)
|
||||
POST /api/items/{partNumber}/file Upload file (editor)
|
||||
POST /api/items/import CSV import (editor)
|
||||
|
||||
GET /api/items/{partNumber}/bom/* Read BOM (viewer)
|
||||
POST /api/items/{partNumber}/bom Add BOM entry (editor)
|
||||
PUT /api/items/{partNumber}/bom/* Update BOM entry (editor)
|
||||
DELETE /api/items/{partNumber}/bom/* Delete BOM entry (editor)
|
||||
|
||||
POST /api/generate-part-number Generate PN (editor)
|
||||
|
||||
GET /api/integrations/odoo/* Read Odoo config (viewer)
|
||||
PUT /api/integrations/odoo/* Modify Odoo config (editor)
|
||||
POST /api/integrations/odoo/* Odoo sync operations (editor)
|
||||
```
|
||||
|
||||
API routes are exempt from CSRF (they use Bearer token auth). CORS credentials are allowed so browser-based API clients with session cookies work.
|
||||
|
||||
## RequireAuth
|
||||
|
||||
`internal/api/middleware.go`
|
||||
|
||||
Authentication check order:
|
||||
|
||||
1. **Auth disabled?** Inject synthetic dev user (`admin` role) and continue
|
||||
2. **Bearer token?** Extract from `Authorization: Bearer silo_...` header, validate via `auth.Service.ValidateToken()`. On success, inject user into context
|
||||
3. **Session cookie?** Read `user_id` from `scs` session, look up user via `auth.Service.GetUserByID()`. On success, inject user into context. On stale session (user not found), destroy session
|
||||
4. **None of the above?**
|
||||
- API requests (`/api/*`): Return `401 Unauthorized` JSON
|
||||
- Web requests: Redirect to `/login?next=<current-path>`
|
||||
|
||||
## RequireRole
|
||||
|
||||
`internal/api/middleware.go`
|
||||
|
||||
Applied as per-group middleware on routes that require a minimum role:
|
||||
|
||||
```go
|
||||
r.Use(server.RequireRole(auth.RoleEditor))
|
||||
```
|
||||
|
||||
Checks `auth.RoleSatisfies(user.Role, minimum)` against the hierarchy `admin > editor > viewer`. Returns:
|
||||
|
||||
- `401 Unauthorized` if no user in context (should not happen after RequireAuth)
|
||||
- `403 Forbidden` with message `"Insufficient permissions: requires <role> role"` if role is too low
|
||||
|
||||
## CSRFProtect
|
||||
|
||||
`internal/api/middleware.go`
|
||||
|
||||
Wraps the `justinas/nosurf` library:
|
||||
|
||||
- Cookie: `csrf_token`, HttpOnly, SameSite=Lax, Secure when auth enabled
|
||||
- Exempt paths: `/api/*`, `/health`, `/ready`
|
||||
- Form field name: `csrf_token`
|
||||
- Failure: Returns `403 Forbidden` with `"CSRF token validation failed"`
|
||||
|
||||
Templates inject the token via `{{.CSRFToken}}` which is populated from `nosurf.Token(r)`.
|
||||
|
||||
## CORS Configuration
|
||||
|
||||
Configured in `internal/api/routes.go`:
|
||||
|
||||
| Setting | Auth Disabled | Auth Enabled |
|
||||
|---------|---------------|--------------|
|
||||
| Allowed Origins | `*` | From `auth.cors.allowed_origins` config |
|
||||
| Allow Credentials | `false` | `true` (needed for session cookies) |
|
||||
| Allowed Methods | GET, POST, PUT, PATCH, DELETE, OPTIONS | Same |
|
||||
| Allowed Headers | Accept, Authorization, Content-Type, X-CSRF-Token, X-Request-ID | Same |
|
||||
| Max Age | 300 seconds | 300 seconds |
|
||||
|
||||
FreeCAD uses direct HTTP (not browser), so CORS does not affect it. Browser-based tools on other origins need their origin in the allowed list.
|
||||
|
||||
## Request Flow Examples
|
||||
|
||||
### Browser Login
|
||||
|
||||
```
|
||||
GET /projects
|
||||
-> RequestLogger
|
||||
-> CORS (pass)
|
||||
-> Session (loads empty session)
|
||||
-> RequireAuth: no token, no session user_id
|
||||
-> Redirect 303 /login?next=/projects
|
||||
|
||||
POST /login (username=alice, password=...)
|
||||
-> RequestLogger
|
||||
-> CORS (pass)
|
||||
-> Session (loads)
|
||||
-> HandleLogin: auth.Authenticate(ctx, "alice", password)
|
||||
-> Session: put user_id, renew token
|
||||
-> Redirect 303 /projects
|
||||
|
||||
GET /projects
|
||||
-> Session (loads, has user_id)
|
||||
-> RequireAuth: session -> GetUserByID -> inject alice
|
||||
-> CSRFProtect: GET request, passes
|
||||
-> HandleProjectsPage: renders with alice's info
|
||||
```
|
||||
|
||||
### API Token Request
|
||||
|
||||
```
|
||||
GET /api/items
|
||||
Authorization: Bearer silo_a1b2c3d4...
|
||||
|
||||
-> RequireAuth: Bearer token found
|
||||
-> ValidateToken: SHA-256 hash lookup, check expiry, check user active
|
||||
-> Inject user into context
|
||||
-> HandleListItems: returns JSON
|
||||
```
|
||||
257
docs/src/silo-server/AUTH_USER_GUIDE.md
Normal file
257
docs/src/silo-server/AUTH_USER_GUIDE.md
Normal file
@@ -0,0 +1,257 @@
|
||||
# Silo Authentication User Guide
|
||||
|
||||
## Logging In
|
||||
|
||||
### Username and Password
|
||||
|
||||
Navigate to the Silo web UI. If authentication is enabled, you'll be redirected to the login page.
|
||||
|
||||
Enter your username and password. This works for both local accounts and LDAP/FreeIPA accounts — Silo tries local authentication first, then LDAP if configured.
|
||||
|
||||
### Keycloak / OIDC
|
||||
|
||||
If your deployment has OIDC enabled, the login page will show a "Sign in with Keycloak" button. Click it to be redirected to your identity provider. After authenticating there, you'll be redirected back to Silo with a session.
|
||||
|
||||
## Roles
|
||||
|
||||
Your role determines what you can do in Silo:
|
||||
|
||||
| Role | Permissions |
|
||||
|------|-------------|
|
||||
| **viewer** | Read all items, projects, schemas, BOMs. Manage own API tokens. |
|
||||
| **editor** | Everything a viewer can do, plus: create/update/delete items, upload files, manage BOMs, import CSV, generate part numbers. |
|
||||
| **admin** | Everything an editor can do, plus: user management (future), configuration changes. |
|
||||
|
||||
Your role is shown as a badge next to your name in the header. For LDAP and OIDC users, the role is determined by group membership or token claims and re-evaluated on each login.
|
||||
|
||||
## API Tokens
|
||||
|
||||
API tokens allow the FreeCAD plugin, scripts, and CI pipelines to authenticate with Silo without a browser session. Tokens inherit your role.
|
||||
|
||||
### Creating a Token (Web UI)
|
||||
|
||||
1. Click **Settings** in the navigation bar
|
||||
2. Under **API Tokens**, enter a name (e.g., "FreeCAD workstation") and click **Create Token**
|
||||
3. The raw token is displayed once — copy it immediately
|
||||
4. Store the token securely. It cannot be shown again.
|
||||
|
||||
### Creating a Token (CLI)
|
||||
|
||||
```sh
|
||||
export SILO_API_URL=https://silo.kindred.internal
|
||||
export SILO_API_TOKEN=silo_<your-existing-token>
|
||||
|
||||
silo token create --name "CI pipeline"
|
||||
```
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
Token created: CI pipeline
|
||||
API Token: silo_a1b2c3d4e5f6...
|
||||
Save this token — it will not be shown again.
|
||||
```
|
||||
|
||||
### Listing Tokens
|
||||
|
||||
```sh
|
||||
silo token list
|
||||
```
|
||||
|
||||
### Revoking a Token
|
||||
|
||||
Via the web UI settings page (click **Revoke** next to the token), or via CLI:
|
||||
|
||||
```sh
|
||||
silo token revoke <token-id>
|
||||
```
|
||||
|
||||
Revocation is immediate. Any in-flight requests using the token will fail.
|
||||
|
||||
## FreeCAD Plugin Configuration
|
||||
|
||||
The FreeCAD plugin reads the API token from two sources (checked in order):
|
||||
|
||||
1. **FreeCAD Preferences**: `Tools > Edit parameters > BaseApp/Preferences/Mod/Silo > ApiToken`
|
||||
2. **Environment variable**: `SILO_API_TOKEN`
|
||||
|
||||
To set the token in FreeCAD preferences:
|
||||
|
||||
1. Open FreeCAD
|
||||
2. Go to `Edit > Preferences > General > Macro` or use the parameter editor
|
||||
3. Navigate to `BaseApp/Preferences/Mod/Silo`
|
||||
4. Set `ApiToken` to your token string (e.g., `silo_a1b2c3d4...`)
|
||||
|
||||
Or set the environment variable before launching FreeCAD:
|
||||
|
||||
```sh
|
||||
export SILO_API_TOKEN=silo_a1b2c3d4...
|
||||
freecad
|
||||
```
|
||||
|
||||
The API URL is configured the same way via the `ApiUrl` preference or `SILO_API_URL` environment variable.
|
||||
|
||||
## Default Admin Account
|
||||
|
||||
On first deployment, configure a default admin account to bootstrap access:
|
||||
|
||||
```yaml
|
||||
# config.yaml
|
||||
auth:
|
||||
enabled: true
|
||||
local:
|
||||
enabled: true
|
||||
default_admin_username: "admin"
|
||||
default_admin_password: "" # Set via SILO_ADMIN_PASSWORD env var
|
||||
```
|
||||
|
||||
```sh
|
||||
export SILO_ADMIN_PASSWORD=<strong-password>
|
||||
```
|
||||
|
||||
The admin account is created on the first startup if it doesn't already exist. Subsequent startups skip creation. Change the password after first login via the database or a future admin UI.
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Minimal (Local Auth Only)
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: true
|
||||
session_secret: "" # Set via SILO_SESSION_SECRET
|
||||
|
||||
local:
|
||||
enabled: true
|
||||
default_admin_username: "admin"
|
||||
default_admin_password: "" # Set via SILO_ADMIN_PASSWORD
|
||||
```
|
||||
|
||||
### LDAP / FreeIPA
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: true
|
||||
session_secret: ""
|
||||
|
||||
local:
|
||||
enabled: true
|
||||
default_admin_username: "admin"
|
||||
default_admin_password: ""
|
||||
|
||||
ldap:
|
||||
enabled: true
|
||||
url: "ldaps://ipa.kindred.internal"
|
||||
base_dn: "dc=kindred,dc=internal"
|
||||
user_search_dn: "cn=users,cn=accounts,dc=kindred,dc=internal"
|
||||
user_attr: "uid"
|
||||
email_attr: "mail"
|
||||
display_attr: "displayName"
|
||||
group_attr: "memberOf"
|
||||
role_mapping:
|
||||
admin:
|
||||
- "cn=silo-admins,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
editor:
|
||||
- "cn=silo-users,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
- "cn=engineers,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
viewer:
|
||||
- "cn=silo-viewers,cn=groups,cn=accounts,dc=kindred,dc=internal"
|
||||
tls_skip_verify: false
|
||||
```
|
||||
|
||||
### OIDC / Keycloak
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: true
|
||||
session_secret: ""
|
||||
|
||||
local:
|
||||
enabled: true
|
||||
|
||||
oidc:
|
||||
enabled: true
|
||||
issuer_url: "https://keycloak.kindred.internal/realms/silo"
|
||||
client_id: "silo"
|
||||
client_secret: "" # Set via SILO_OIDC_CLIENT_SECRET
|
||||
redirect_url: "https://silo.kindred.internal/auth/callback"
|
||||
scopes: ["openid", "profile", "email"]
|
||||
admin_role: "silo-admin"
|
||||
editor_role: "silo-editor"
|
||||
default_role: "viewer"
|
||||
```
|
||||
|
||||
### CORS (Production)
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "https://silo.kindred.internal"
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description | Config Path |
|
||||
|----------|-------------|-------------|
|
||||
| `SILO_SESSION_SECRET` | Session encryption key | `auth.session_secret` |
|
||||
| `SILO_ADMIN_USERNAME` | Default admin username | `auth.local.default_admin_username` |
|
||||
| `SILO_ADMIN_PASSWORD` | Default admin password | `auth.local.default_admin_password` |
|
||||
| `SILO_OIDC_CLIENT_SECRET` | OIDC client secret | `auth.oidc.client_secret` |
|
||||
| `SILO_LDAP_BIND_PASSWORD` | LDAP service account password | `auth.ldap.bind_password` |
|
||||
| `SILO_API_URL` | API base URL (CLI and FreeCAD) | — |
|
||||
| `SILO_API_TOKEN` | API token (CLI and FreeCAD) | — |
|
||||
|
||||
Environment variables override config file values.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Authentication required" on every request
|
||||
|
||||
- Verify `auth.enabled: true` in your config
|
||||
- Check that the `sessions` table exists in PostgreSQL (migration 009)
|
||||
- Ensure `SILO_SESSION_SECRET` is set (empty string is allowed for dev but not recommended)
|
||||
- Check browser cookies — `silo_session` should be present after login
|
||||
|
||||
### API token returns 401
|
||||
|
||||
- Tokens are case-sensitive. Ensure no trailing whitespace
|
||||
- Check token expiry with `silo token list`
|
||||
- Verify the user account is still active
|
||||
- Ensure the `Authorization` header format is exactly `Bearer silo_<hex>`
|
||||
|
||||
### LDAP login fails
|
||||
|
||||
- Check `ldaps://` URL is reachable from the Silo server
|
||||
- Verify `base_dn` and `user_search_dn` match your FreeIPA tree
|
||||
- Test with `ldapsearch` from the command line first
|
||||
- Set `tls_skip_verify: true` temporarily to rule out certificate issues
|
||||
- Check Silo logs for the specific LDAP error message
|
||||
|
||||
### OIDC redirect loops
|
||||
|
||||
- Verify `redirect_url` matches the Keycloak client configuration exactly
|
||||
- Check that `issuer_url` is reachable from the Silo server
|
||||
- Ensure the Keycloak client has the correct redirect URI registered
|
||||
- Check for clock skew between Silo and Keycloak servers (JWT validation is time-sensitive)
|
||||
|
||||
### Locked out (no admin account)
|
||||
|
||||
If you've lost access to all admin accounts:
|
||||
|
||||
1. Set `auth.local.default_admin_password` to a new password via `SILO_ADMIN_PASSWORD`
|
||||
2. Use a different username (e.g., `default_admin_username: "recovery-admin"`)
|
||||
3. Restart Silo — the new account will be created
|
||||
4. Log in and fix the original accounts
|
||||
|
||||
Or directly update the database:
|
||||
|
||||
```sql
|
||||
-- Reset a local user's password (generate bcrypt hash externally)
|
||||
UPDATE users SET password_hash = '<bcrypt-hash>', is_active = true WHERE username = 'admin';
|
||||
```
|
||||
|
||||
### FreeCAD plugin gets 401
|
||||
|
||||
- Verify the token is set in FreeCAD preferences or `SILO_API_TOKEN`
|
||||
- Check the API URL points to the correct server
|
||||
- Test with curl: `curl -H "Authorization: Bearer silo_..." https://silo.kindred.internal/api/items`
|
||||
0
docs/src/silo-server/BOM_MERGE.md
Normal file
0
docs/src/silo-server/BOM_MERGE.md
Normal file
16
docs/src/silo-server/CALC_EXTENSION.md
Normal file
16
docs/src/silo-server/CALC_EXTENSION.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# LibreOffice Calc Extension
|
||||
|
||||
The Silo Calc extension has been moved to its own repository: [silo-calc](https://git.kindred-systems.com/kindred/silo-calc).
|
||||
|
||||
## Server-Side ODS Support
|
||||
|
||||
The server-side ODS library (`internal/ods/`) and ODS endpoints remain in this repository. See `docs/SPECIFICATION.md` Section 11 for the full endpoint listing.
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/items/export.ods` | Items as ODS |
|
||||
| GET | `/api/items/template.ods` | Blank import template |
|
||||
| POST | `/api/items/import.ods` | Import from ODS |
|
||||
| GET | `/api/items/{pn}/bom/export.ods` | BOM as formatted ODS |
|
||||
| GET | `/api/projects/{code}/sheet.ods` | Multi-sheet project workbook |
|
||||
| POST | `/api/sheets/diff` | Upload ODS, return JSON diff |
|
||||
523
docs/src/silo-server/COMPONENT_AUDIT.md
Normal file
523
docs/src/silo-server/COMPONENT_AUDIT.md
Normal file
@@ -0,0 +1,523 @@
|
||||
# Component Audit Tool
|
||||
|
||||
**Last Updated:** 2026-02-01
|
||||
**Status:** Design
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The parts database has grown organically. Many items were created with only a
|
||||
part number, description, and category. The property schema defines dozens of
|
||||
fields per category (material, finish, manufacturer, supplier, cost, etc.) but
|
||||
most items have few or none of these populated. There is no way to see which
|
||||
items are missing data or to prioritize what needs filling in.
|
||||
|
||||
Currently, adding or updating properties requires either:
|
||||
- Editing each item individually through the web UI detail panel
|
||||
- Bulk CSV export, manual editing, re-import
|
||||
- The Calc extension (new, not yet widely used)
|
||||
|
||||
None of these approaches give visibility into what's missing across the
|
||||
database. Engineers don't know which items need attention until they encounter
|
||||
a blank field during a design review or procurement cycle.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
1. Show a per-item completeness score based on the property schema
|
||||
2. Surface the least-complete items so they can be prioritized
|
||||
3. Let users fill in missing fields directly from the audit view
|
||||
4. Filter by project, category, completeness threshold
|
||||
5. Track improvement over time
|
||||
|
||||
---
|
||||
|
||||
## Design
|
||||
|
||||
The audit tool is a page in the web UI (`/audit`), built with the React SPA
|
||||
(same architecture as the items, projects, and schemas pages). It adds one
|
||||
new API endpoint for the completeness data and reuses existing endpoints for
|
||||
updates.
|
||||
|
||||
### Completeness Scoring
|
||||
|
||||
Each item's completeness is computed against its category's property schema.
|
||||
The schema defines both **global defaults** (12 fields, all categories) and
|
||||
**category-specific properties** (varies: 9 fields for fasteners, 20+ for
|
||||
motion components, etc.).
|
||||
|
||||
**Score formula:**
|
||||
|
||||
```
|
||||
score = sum(weight for each filled field) / sum(weight for all applicable fields)
|
||||
```
|
||||
|
||||
Score is 0.0 to 1.0, displayed as a percentage. Fields are weighted
|
||||
differently depending on sourcing type.
|
||||
|
||||
**Purchased parts (`sourcing_type = "purchased"`):**
|
||||
|
||||
| Weight | Fields | Rationale |
|
||||
|--------|--------|-----------|
|
||||
| 3 | manufacturer_pn, sourcing_link | Can't procure without these |
|
||||
| 2 | manufacturer, supplier, supplier_pn, standard_cost | Core procurement data |
|
||||
| 1 | description, sourcing_type, lead_time_days, minimum_order_qty, lifecycle_status | Important but less blocking |
|
||||
| 1 | All category-specific properties | Engineering detail |
|
||||
| 0.5 | rohs_compliant, country_of_origin, notes, long_description | Nice to have |
|
||||
|
||||
**Manufactured parts (`sourcing_type = "manufactured"`):**
|
||||
|
||||
| Weight | Fields | Rationale |
|
||||
|--------|--------|-----------|
|
||||
| 3 | has_bom (at least one BOM child) | Can't manufacture without a BOM |
|
||||
| 2 | description, standard_cost | Core identification |
|
||||
| 1 | All category-specific properties | Engineering detail |
|
||||
| 0.5 | manufacturer, supplier, notes, long_description | Less relevant for in-house |
|
||||
|
||||
The `has_bom` check for manufactured parts queries the `relationships`
|
||||
table for at least one `rel_type = 'component'` child. This is not a
|
||||
property field -- it's a structural check. A manufactured part with no BOM
|
||||
children is flagged as critically incomplete regardless of how many other
|
||||
fields are filled.
|
||||
|
||||
**Assemblies (categories A01-A07):**
|
||||
|
||||
Assembly scores are partially computed from children:
|
||||
|
||||
| Field | Source | Notes |
|
||||
|-------|--------|-------|
|
||||
| weight | Sum of child weights | Computed if all children have weight |
|
||||
| standard_cost | Sum of child (cost * qty) | Computed from BOM |
|
||||
| component_count | Count of BOM children | Always computable |
|
||||
| has_bom | BOM children exist | Required (weight 3) |
|
||||
|
||||
A computed field counts as "filled" if the data needed to compute it is
|
||||
available. If a computed value exists, it is shown alongside the stored
|
||||
value so engineers can verify or override.
|
||||
|
||||
Assembly-specific properties that cannot be computed (assembly_time,
|
||||
test_procedure, ip_rating, dimensions) are scored normally.
|
||||
|
||||
**Field filled criteria:**
|
||||
|
||||
- String fields: non-empty after trimming
|
||||
- Number fields: non-null and non-zero
|
||||
- Boolean fields: non-null (false is a valid answer)
|
||||
- has_bom: at least one component relationship exists
|
||||
|
||||
Item-level fields (`description`, `sourcing_type`, `sourcing_link`,
|
||||
`standard_cost`, `long_description`) are checked on the items table.
|
||||
Property fields (`manufacturer`, `material`, etc.) are checked on the
|
||||
current revision's `properties` JSONB column. BOM existence is checked
|
||||
on the `relationships` table.
|
||||
|
||||
### Tiers
|
||||
|
||||
Items are grouped into completeness tiers for dashboard display:
|
||||
|
||||
| Tier | Range | Color | Label |
|
||||
|------|-------|-------|-------|
|
||||
| Critical | 0-25% | Red | Missing critical data |
|
||||
| Low | 25-50% | Orange | Needs attention |
|
||||
| Partial | 50-75% | Yellow | Partially complete |
|
||||
| Good | 75-99% | Light green | Nearly complete |
|
||||
| Complete | 100% | Green | All fields populated |
|
||||
|
||||
---
|
||||
|
||||
## API
|
||||
|
||||
### `GET /api/audit/completeness`
|
||||
|
||||
Returns completeness scores for all items (or filtered subset).
|
||||
|
||||
**Query parameters:**
|
||||
|
||||
| Param | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `project` | string | Filter by project code |
|
||||
| `category` | string | Filter by category prefix (e.g. `F`, `F01`) |
|
||||
| `max_score` | float | Only items below this score (e.g. `0.5`) |
|
||||
| `min_score` | float | Only items above this score |
|
||||
| `sort` | string | `score_asc` (default), `score_desc`, `part_number`, `updated_at` |
|
||||
| `limit` | int | Pagination limit (default 100) |
|
||||
| `offset` | int | Pagination offset |
|
||||
|
||||
**Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"items": [
|
||||
{
|
||||
"part_number": "F01-0042",
|
||||
"description": "M3x10 Socket Head Cap Screw",
|
||||
"category": "F01",
|
||||
"category_name": "Screws and Bolts",
|
||||
"sourcing_type": "purchased",
|
||||
"projects": ["3DX10", "PROTO"],
|
||||
"score": 0.41,
|
||||
"weighted_filled": 12.5,
|
||||
"weighted_total": 30.5,
|
||||
"has_bom": false,
|
||||
"bom_children": 0,
|
||||
"missing_critical": ["manufacturer_pn", "sourcing_link"],
|
||||
"missing": [
|
||||
"manufacturer_pn",
|
||||
"sourcing_link",
|
||||
"supplier",
|
||||
"supplier_pn",
|
||||
"finish",
|
||||
"strength_grade",
|
||||
"torque_spec"
|
||||
],
|
||||
"updated_at": "2026-01-15T10:30:00Z"
|
||||
},
|
||||
{
|
||||
"part_number": "A01-0003",
|
||||
"description": "3DX10 Line Assembly",
|
||||
"category": "A01",
|
||||
"category_name": "Mechanical Assembly",
|
||||
"sourcing_type": "manufactured",
|
||||
"projects": ["3DX10"],
|
||||
"score": 0.68,
|
||||
"weighted_filled": 15.0,
|
||||
"weighted_total": 22.0,
|
||||
"has_bom": true,
|
||||
"bom_children": 12,
|
||||
"computed_fields": {
|
||||
"standard_cost": 7538.61,
|
||||
"component_count": 12,
|
||||
"weight": null
|
||||
},
|
||||
"missing_critical": [],
|
||||
"missing": ["assembly_time", "test_procedure", "weight", "ip_rating"],
|
||||
"updated_at": "2026-01-28T14:20:00Z"
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
"total_items": 847,
|
||||
"avg_score": 0.42,
|
||||
"manufactured_without_bom": 31,
|
||||
"by_tier": {
|
||||
"critical": 123,
|
||||
"low": 298,
|
||||
"partial": 251,
|
||||
"good": 142,
|
||||
"complete": 33
|
||||
},
|
||||
"by_category": {
|
||||
"F": {"count": 156, "avg_score": 0.51},
|
||||
"C": {"count": 89, "avg_score": 0.38},
|
||||
"R": {"count": 201, "avg_score": 0.29}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `GET /api/audit/completeness/{partNumber}`
|
||||
|
||||
Single-item detail with field-by-field breakdown.
|
||||
|
||||
```json
|
||||
{
|
||||
"part_number": "F01-0042",
|
||||
"description": "M3x10 Socket Head Cap Screw",
|
||||
"category": "F01",
|
||||
"sourcing_type": "purchased",
|
||||
"score": 0.41,
|
||||
"has_bom": false,
|
||||
"fields": [
|
||||
{"key": "description", "source": "item", "weight": 1, "value": "M3x10 Socket Head Cap Screw", "filled": true},
|
||||
{"key": "sourcing_type", "source": "item", "weight": 1, "value": "purchased", "filled": true},
|
||||
{"key": "standard_cost", "source": "item", "weight": 2, "value": 0.12, "filled": true},
|
||||
{"key": "sourcing_link", "source": "item", "weight": 3, "value": "", "filled": false},
|
||||
{"key": "manufacturer", "source": "property", "weight": 2, "value": null, "filled": false},
|
||||
{"key": "manufacturer_pn", "source": "property", "weight": 3, "value": null, "filled": false},
|
||||
{"key": "supplier", "source": "property", "weight": 2, "value": null, "filled": false},
|
||||
{"key": "supplier_pn", "source": "property", "weight": 2, "value": null, "filled": false},
|
||||
{"key": "material", "source": "property", "weight": 1, "value": "18-8 Stainless Steel", "filled": true},
|
||||
{"key": "finish", "source": "property", "weight": 1, "value": null, "filled": false},
|
||||
{"key": "thread_size", "source": "property", "weight": 1, "value": "M3", "filled": true},
|
||||
{"key": "thread_pitch", "source": "property", "weight": 1, "value": null, "filled": false},
|
||||
{"key": "length", "source": "property", "weight": 1, "value": "10mm", "filled": true},
|
||||
{"key": "head_type", "source": "property", "weight": 1, "value": "Socket", "filled": true}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
For assemblies, the detail response includes a `computed_fields` section
|
||||
showing values derived from children (cost rollup, weight rollup,
|
||||
component count). These are presented alongside stored values in the UI
|
||||
so engineers can compare and choose to accept the computed value.
|
||||
|
||||
Existing `PUT /api/items/{pn}` and revision property updates handle writes.
|
||||
|
||||
---
|
||||
|
||||
## Web UI
|
||||
|
||||
### Audit Page (`/audit`)
|
||||
|
||||
New page accessible from the top navigation bar (fourth tab after Items,
|
||||
Projects, Schemas).
|
||||
|
||||
**Layout:**
|
||||
|
||||
```
|
||||
+------------------------------------------------------------------+
|
||||
| Items | Projects | Schemas | Audit |
|
||||
+------------------------------------------------------------------+
|
||||
| [Project: ___] [Category: ___] [Max Score: ___] [Search] |
|
||||
+------------------------------------------------------------------+
|
||||
| Summary Bar |
|
||||
| [===Critical: 123===|===Low: 298===|==Partial: 251==|Good|Done] |
|
||||
+------------------------------------------------------------------+
|
||||
| Score | PN | Description | Category | Missing|
|
||||
|-------|-----------|--------------------------|----------|--------|
|
||||
| 12% | R01-0003 | Bearing, Deep Groove 6205| Bearings | 18 |
|
||||
| 15% | E14-0001 | NTC Thermistor 10K | Sensors | 16 |
|
||||
| 23% | C03-0012 | 1/4" NPT Ball Valve SS | Valves | 14 |
|
||||
| 35% | F01-0042 | M3x10 Socket Head Cap | Screws | 7 |
|
||||
| ... | | | | |
|
||||
+------------------------------------------------------------------+
|
||||
```
|
||||
|
||||
**Interactions:**
|
||||
|
||||
- Click a row to open an inline edit panel (right side, same split-panel
|
||||
pattern as the items page)
|
||||
- The edit panel shows all applicable fields for the category, with empty
|
||||
fields highlighted
|
||||
- Editing a field and pressing Enter/Tab saves immediately via API
|
||||
- Score updates live after each save
|
||||
- Summary bar updates as items are completed
|
||||
- Click a tier segment in the summary bar to filter to that tier
|
||||
|
||||
### Inline Edit Panel
|
||||
|
||||
```
|
||||
+----------------------------------+
|
||||
| F01-0042 Score: 35% |
|
||||
| M3x10 Socket Head Cap Screw |
|
||||
+----------------------------------+
|
||||
| -- Required -- |
|
||||
| Description [M3x10 Socket H..] |
|
||||
| Sourcing [purchased v ] |
|
||||
+----------------------------------+
|
||||
| -- Procurement -- |
|
||||
| Manufacturer [________________] |
|
||||
| Mfr PN [________________] |
|
||||
| Supplier [________________] |
|
||||
| Supplier PN [________________] |
|
||||
| Cost [$0.12 ] |
|
||||
| Sourcing Link[________________] |
|
||||
| Lead Time [____ days ] |
|
||||
+----------------------------------+
|
||||
| -- Fastener Properties -- |
|
||||
| Material [18-8 Stainless ] |
|
||||
| Finish [________________] |
|
||||
| Thread Size [M3 ] |
|
||||
| Thread Pitch [________________] |
|
||||
| Length [10mm ] |
|
||||
| Head Type [Socket ] |
|
||||
| Drive Type [________________] |
|
||||
| Strength [________________] |
|
||||
| Torque Spec [________________] |
|
||||
+----------------------------------+
|
||||
| [Save All] |
|
||||
+----------------------------------+
|
||||
```
|
||||
|
||||
Fields are grouped into sections: Required, Procurement (global defaults),
|
||||
and category-specific properties. Empty fields have a subtle red left border.
|
||||
Filled fields have a green left border. The score bar at the top updates as
|
||||
fields are filled in.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: API endpoint and scoring engine
|
||||
|
||||
New file: `internal/api/audit_handlers.go`
|
||||
|
||||
- `HandleAuditCompleteness` -- query items, join current revision properties,
|
||||
compute scores against schema, return paginated JSON
|
||||
- `HandleAuditItemDetail` -- single item with field-by-field breakdown
|
||||
- Scoring logic in a helper function that takes item fields + revision
|
||||
properties + category schema and returns score + missing list
|
||||
|
||||
Register routes:
|
||||
- `GET /api/audit/completeness` (viewer role)
|
||||
- `GET /api/audit/completeness/{partNumber}` (viewer role)
|
||||
|
||||
### Phase 2: Web UI page
|
||||
|
||||
New template: `internal/api/templates/audit.html`
|
||||
|
||||
- Same base template, Catppuccin Mocha theme, nav bar with Audit tab
|
||||
- Summary bar with tier counts (colored segments)
|
||||
- Sortable, filterable table
|
||||
- Split-panel detail view on row click
|
||||
- Vanilla JS fetch calls to audit and item update endpoints
|
||||
|
||||
Update `internal/api/web.go`:
|
||||
- Add `HandleAuditPage` handler
|
||||
- Register `GET /audit` route
|
||||
|
||||
Update `internal/api/templates/base.html`:
|
||||
- Add Audit tab to navigation
|
||||
|
||||
### Phase 3: Inline editing
|
||||
|
||||
- Field save on blur/Enter via `PUT /api/items/{pn}` for item fields
|
||||
- Property updates via `POST /api/items/{pn}/revisions` with updated
|
||||
properties map
|
||||
- Live score recalculation after save (re-fetch from audit detail endpoint)
|
||||
- Batch "Save All" button for multiple field changes
|
||||
|
||||
### Phase 4: Tracking and reporting
|
||||
|
||||
- Store periodic score snapshots (daily cron or on-demand) in a new
|
||||
`audit_snapshots` table for trend tracking
|
||||
- Dashboard chart showing completeness improvement over time
|
||||
- Per-project completeness summary on the projects page
|
||||
- CSV export of audit results for offline review
|
||||
|
||||
### Phase 5: Batch AI assistance
|
||||
|
||||
Server-side OpenRouter integration for bulk property inference from existing
|
||||
sourcing data. This extends the Calc extension's AI client pattern to the
|
||||
backend.
|
||||
|
||||
**Workflow:**
|
||||
|
||||
1. Audit page shows items with sourcing links but missing properties
|
||||
2. Engineer selects items (or filters to a category/project) and clicks
|
||||
"AI Fill Properties"
|
||||
3. Server fetches each item's sourcing link page content (or uses the
|
||||
seller description from the item's metadata)
|
||||
4. OpenRouter API call per item: system prompt describes the category's
|
||||
property schema, user prompt provides the scraped/stored description
|
||||
5. AI returns structured JSON with suggested property values
|
||||
6. Results shown in a review table: item, field, current value, suggested
|
||||
value, confidence indicator
|
||||
7. Engineer checks/unchecks suggestions, clicks "Apply Selected"
|
||||
8. Server writes accepted values as property updates (new revision)
|
||||
|
||||
**AI prompt structure:**
|
||||
|
||||
```
|
||||
System: You are a parts data specialist. Given a product description
|
||||
and a list of property fields with types, extract values for as many
|
||||
fields as possible. Return JSON only.
|
||||
|
||||
User:
|
||||
Category: F01 (Screws and Bolts)
|
||||
Product: {seller_description or scraped page text}
|
||||
|
||||
Fields to extract:
|
||||
- material (string): Material specification
|
||||
- finish (string): Surface finish
|
||||
- thread_size (string): Thread size designation
|
||||
- thread_pitch (string): Thread pitch
|
||||
- length (string): Fastener length with unit
|
||||
- head_type (string): Head style
|
||||
- drive_type (string): Drive type
|
||||
- strength_grade (string): Strength/property class
|
||||
```
|
||||
|
||||
**Rate limiting:** Queue items and process in batches of 10 with 1s delay
|
||||
between batches to stay within OpenRouter rate limits. Show progress bar
|
||||
in the UI.
|
||||
|
||||
**Cost control:** Use `openai/gpt-4.1-nano` by default (cheapest). Show
|
||||
estimated cost before starting batch. Allow model override in settings.
|
||||
|
||||
---
|
||||
|
||||
## Database Changes
|
||||
|
||||
### Phase 1: None
|
||||
|
||||
Completeness is computed at query time from existing `items` +
|
||||
`revisions.properties` data joined against the in-memory schema definition.
|
||||
No new tables needed for the core feature.
|
||||
|
||||
### Phase 4: New table
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS audit_snapshots (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
captured_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
total_items INTEGER NOT NULL,
|
||||
avg_score DECIMAL(5,4) NOT NULL,
|
||||
by_tier JSONB NOT NULL,
|
||||
by_category JSONB NOT NULL,
|
||||
by_project JSONB NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
### Phase 5: None
|
||||
|
||||
AI suggestions are ephemeral (computed per request, not stored). Accepted
|
||||
suggestions are written through the existing revision/property update path.
|
||||
|
||||
---
|
||||
|
||||
## Scoring Examples
|
||||
|
||||
### Purchased Fastener (F01)
|
||||
|
||||
**Weighted total: ~30.5 points**
|
||||
|
||||
| Field | Weight | Filled? | Points |
|
||||
|-------|--------|---------|--------|
|
||||
| manufacturer_pn | 3 | no | 0/3 |
|
||||
| sourcing_link | 3 | no | 0/3 |
|
||||
| manufacturer | 2 | no | 0/2 |
|
||||
| supplier | 2 | no | 0/2 |
|
||||
| supplier_pn | 2 | no | 0/2 |
|
||||
| standard_cost | 2 | yes | 2/2 |
|
||||
| description | 1 | yes | 1/1 |
|
||||
| sourcing_type | 1 | yes | 1/1 |
|
||||
| material | 1 | yes | 1/1 |
|
||||
| thread_size | 1 | yes | 1/1 |
|
||||
| length | 1 | yes | 1/1 |
|
||||
| head_type | 1 | yes | 1/1 |
|
||||
| drive_type | 1 | no | 0/1 |
|
||||
| finish | 1 | no | 0/1 |
|
||||
| ... (remaining) | 0.5-1 | no | 0/... |
|
||||
|
||||
**Score: 8/30.5 = 26%** -- "Low" tier, flagged because weight-3 fields
|
||||
(manufacturer_pn, sourcing_link) are missing.
|
||||
|
||||
### Manufactured Assembly (A01)
|
||||
|
||||
**Weighted total: ~22 points**
|
||||
|
||||
| Field | Weight | Source | Points |
|
||||
|-------|--------|--------|--------|
|
||||
| has_bom | 3 | BOM query | 3/3 (12 children) |
|
||||
| description | 2 | item | 2/2 |
|
||||
| standard_cost | 2 | computed from children | 2/2 |
|
||||
| component_count | 1 | computed (= 12) | 1/1 |
|
||||
| weight | 1 | computed (needs children) | 0/1 (not all children have weight) |
|
||||
| assembly_time | 1 | property | 0/1 |
|
||||
| test_procedure | 1 | property | 0/1 |
|
||||
| dimensions | 1 | property | 0/1 |
|
||||
| ip_rating | 1 | property | 0/1 |
|
||||
| ... (globals) | 0.5-1 | property | .../... |
|
||||
|
||||
**Score: ~15/22 = 68%** -- "Partial" tier, mostly complete because BOM
|
||||
and cost are covered through children.
|
||||
|
||||
### Motor (R01) -- highest field count
|
||||
|
||||
30+ applicable fields across global defaults + motion-specific properties
|
||||
(load, speed, power, voltage, current, torque, encoder, gear ratio...).
|
||||
A motor with only description + cost + sourcing_type scores under 10%
|
||||
because of the large denominator. Motors are the category most likely to
|
||||
benefit from batch AI extraction from datasheets.
|
||||
364
docs/src/silo-server/CONFIGURATION.md
Normal file
364
docs/src/silo-server/CONFIGURATION.md
Normal file
@@ -0,0 +1,364 @@
|
||||
# Configuration Reference
|
||||
|
||||
**Last Updated:** 2026-02-06
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Silo is configured via a YAML file. Copy the example and edit for your environment:
|
||||
|
||||
```bash
|
||||
cp config.example.yaml config.yaml
|
||||
```
|
||||
|
||||
The server reads the config file at startup:
|
||||
|
||||
```bash
|
||||
./silod -config config.yaml # default: config.yaml
|
||||
go run ./cmd/silod -config config.yaml
|
||||
```
|
||||
|
||||
YAML values support environment variable expansion using `${VAR_NAME}` syntax. Environment variable overrides (listed per-key below) take precedence over YAML values.
|
||||
|
||||
---
|
||||
|
||||
## Server
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `server.host` | string | `"0.0.0.0"` | Bind address |
|
||||
| `server.port` | int | `8080` | HTTP port |
|
||||
| `server.base_url` | string | — | External URL (e.g. `https://silo.example.com`). Used for OIDC callback URLs and session cookie domain. Required when OIDC is enabled. |
|
||||
| `server.read_only` | bool | `false` | Start in read-only mode. All write endpoints return 503. Can be toggled at runtime with `SIGUSR1`. |
|
||||
|
||||
```yaml
|
||||
server:
|
||||
host: "0.0.0.0"
|
||||
port: 8080
|
||||
base_url: "https://silo.example.com"
|
||||
read_only: false
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database
|
||||
|
||||
| Key | Type | Default | Env Override | Description |
|
||||
|-----|------|---------|-------------|-------------|
|
||||
| `database.host` | string | — | `SILO_DB_HOST` | PostgreSQL host |
|
||||
| `database.port` | int | `5432` | — | PostgreSQL port |
|
||||
| `database.name` | string | — | `SILO_DB_NAME` | Database name |
|
||||
| `database.user` | string | — | `SILO_DB_USER` | Database user |
|
||||
| `database.password` | string | — | `SILO_DB_PASSWORD` | Database password |
|
||||
| `database.sslmode` | string | `"require"` | — | SSL mode: `disable`, `require`, `verify-ca`, `verify-full` |
|
||||
| `database.max_connections` | int | `10` | — | Connection pool size |
|
||||
|
||||
**SSL mode guidance:**
|
||||
- `disable` — development only, no encryption
|
||||
- `require` — encrypted but no certificate verification (default)
|
||||
- `verify-ca` — verify server certificate is signed by trusted CA
|
||||
- `verify-full` — verify CA and hostname match (recommended for production)
|
||||
|
||||
```yaml
|
||||
database:
|
||||
host: "localhost"
|
||||
port: 5432
|
||||
name: "silo"
|
||||
user: "silo"
|
||||
password: "" # use SILO_DB_PASSWORD env var
|
||||
sslmode: "require"
|
||||
max_connections: 10
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Storage (MinIO/S3)
|
||||
|
||||
| Key | Type | Default | Env Override | Description |
|
||||
|-----|------|---------|-------------|-------------|
|
||||
| `storage.endpoint` | string | — | `SILO_MINIO_ENDPOINT` | MinIO/S3 endpoint (`host:port`) |
|
||||
| `storage.access_key` | string | — | `SILO_MINIO_ACCESS_KEY` | Access key |
|
||||
| `storage.secret_key` | string | — | `SILO_MINIO_SECRET_KEY` | Secret key |
|
||||
| `storage.bucket` | string | — | — | S3 bucket name (created automatically if missing) |
|
||||
| `storage.use_ssl` | bool | `false` | — | Use HTTPS for MinIO connections |
|
||||
| `storage.region` | string | `"us-east-1"` | — | S3 region |
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
endpoint: "localhost:9000"
|
||||
access_key: "" # use SILO_MINIO_ACCESS_KEY env var
|
||||
secret_key: "" # use SILO_MINIO_SECRET_KEY env var
|
||||
bucket: "silo-files"
|
||||
use_ssl: false
|
||||
region: "us-east-1"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Schemas
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `schemas.directory` | string | `"/etc/silo/schemas"` | Path to directory containing YAML schema files |
|
||||
| `schemas.default` | string | — | Default schema name for part number generation |
|
||||
|
||||
Schema files define part numbering formats, category codes, and property definitions. See `schemas/kindred-rd.yaml` for an example.
|
||||
|
||||
```yaml
|
||||
schemas:
|
||||
directory: "./schemas"
|
||||
default: "kindred-rd"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## FreeCAD
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `freecad.uri_scheme` | string | `"silo"` | URI scheme for "Open in FreeCAD" links in the web UI |
|
||||
| `freecad.executable` | string | — | Path to FreeCAD binary (for CLI operations) |
|
||||
|
||||
```yaml
|
||||
freecad:
|
||||
uri_scheme: "silo"
|
||||
executable: "/usr/bin/freecad"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Odoo ERP Integration
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `odoo.enabled` | bool | `false` | Enable Odoo integration |
|
||||
| `odoo.url` | string | — | Odoo server URL |
|
||||
| `odoo.database` | string | — | Odoo database name |
|
||||
| `odoo.username` | string | — | Odoo username |
|
||||
| `odoo.api_key` | string | — | Odoo API key |
|
||||
|
||||
The Odoo integration currently supports configuration and sync-log CRUD. Push/pull sync operations are stubs.
|
||||
|
||||
```yaml
|
||||
odoo:
|
||||
enabled: false
|
||||
url: "https://odoo.example.com"
|
||||
database: "odoo"
|
||||
username: "silo-service"
|
||||
api_key: ""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
Authentication has a master toggle and three independent backends. When `auth.enabled` is `false`, all routes are accessible without login and a synthetic admin user (`dev`) is injected into every request.
|
||||
|
||||
| Key | Type | Default | Env Override | Description |
|
||||
|-----|------|---------|-------------|-------------|
|
||||
| `auth.enabled` | bool | `false` | — | Master toggle. Set `true` for production. |
|
||||
| `auth.session_secret` | string | — | `SILO_SESSION_SECRET` | Secret for signing session cookies. Required when auth is enabled. |
|
||||
|
||||
### Local Auth
|
||||
|
||||
Built-in username/password accounts stored in the Silo database with bcrypt-hashed passwords.
|
||||
|
||||
| Key | Type | Default | Env Override | Description |
|
||||
|-----|------|---------|-------------|-------------|
|
||||
| `auth.local.enabled` | bool | — | — | Enable local accounts |
|
||||
| `auth.local.default_admin_username` | string | — | `SILO_ADMIN_USERNAME` | Default admin account created on first startup |
|
||||
| `auth.local.default_admin_password` | string | — | `SILO_ADMIN_PASSWORD` | Password for default admin (bcrypt-hashed on creation) |
|
||||
|
||||
The default admin account is only created if both username and password are set and the user does not already exist. This is idempotent.
|
||||
|
||||
### LDAP / FreeIPA
|
||||
|
||||
| Key | Type | Default | Env Override | Description |
|
||||
|-----|------|---------|-------------|-------------|
|
||||
| `auth.ldap.enabled` | bool | `false` | — | Enable LDAP authentication |
|
||||
| `auth.ldap.url` | string | — | — | LDAP server URL (e.g. `ldaps://ipa.example.com`) |
|
||||
| `auth.ldap.base_dn` | string | — | — | Base DN for the LDAP tree |
|
||||
| `auth.ldap.user_search_dn` | string | — | — | DN under which to search for users |
|
||||
| `auth.ldap.bind_dn` | string | — | — | Service account DN for user lookups (optional; omit for direct user bind) |
|
||||
| `auth.ldap.bind_password` | string | — | `SILO_LDAP_BIND_PASSWORD` | Service account password |
|
||||
| `auth.ldap.user_attr` | string | `"uid"` | — | LDAP attribute for username |
|
||||
| `auth.ldap.email_attr` | string | `"mail"` | — | LDAP attribute for email |
|
||||
| `auth.ldap.display_attr` | string | `"displayName"` | — | LDAP attribute for display name |
|
||||
| `auth.ldap.group_attr` | string | `"memberOf"` | — | LDAP attribute for group membership |
|
||||
| `auth.ldap.role_mapping` | map | — | — | Maps LDAP group DNs to Silo roles (see example below) |
|
||||
| `auth.ldap.tls_skip_verify` | bool | `false` | — | Skip TLS certificate verification (testing only) |
|
||||
|
||||
**Role mapping** maps LDAP group DNs to Silo roles. Groups are checked in priority order: admin, then editor, then viewer. The first match wins.
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
ldap:
|
||||
enabled: true
|
||||
url: "ldaps://ipa.example.com"
|
||||
base_dn: "dc=example,dc=com"
|
||||
user_search_dn: "cn=users,cn=accounts,dc=example,dc=com"
|
||||
role_mapping:
|
||||
admin:
|
||||
- "cn=silo-admins,cn=groups,cn=accounts,dc=example,dc=com"
|
||||
editor:
|
||||
- "cn=silo-users,cn=groups,cn=accounts,dc=example,dc=com"
|
||||
- "cn=engineers,cn=groups,cn=accounts,dc=example,dc=com"
|
||||
viewer:
|
||||
- "cn=silo-viewers,cn=groups,cn=accounts,dc=example,dc=com"
|
||||
```
|
||||
|
||||
### OIDC / Keycloak
|
||||
|
||||
| Key | Type | Default | Env Override | Description |
|
||||
|-----|------|---------|-------------|-------------|
|
||||
| `auth.oidc.enabled` | bool | `false` | — | Enable OIDC authentication |
|
||||
| `auth.oidc.issuer_url` | string | — | — | OIDC provider issuer URL (e.g. Keycloak realm URL) |
|
||||
| `auth.oidc.client_id` | string | — | — | OAuth2 client ID |
|
||||
| `auth.oidc.client_secret` | string | — | `SILO_OIDC_CLIENT_SECRET` | OAuth2 client secret |
|
||||
| `auth.oidc.redirect_url` | string | — | — | OAuth2 callback URL (typically `{base_url}/auth/callback`) |
|
||||
| `auth.oidc.scopes` | []string | `["openid", "profile", "email"]` | — | OAuth2 scopes to request |
|
||||
| `auth.oidc.admin_role` | string | — | — | Keycloak realm role that grants admin access |
|
||||
| `auth.oidc.editor_role` | string | — | — | Keycloak realm role that grants editor access |
|
||||
| `auth.oidc.default_role` | string | `"viewer"` | — | Fallback role when no role claim matches |
|
||||
|
||||
Roles are extracted from the Keycloak `realm_access.roles` claim. If the user has the `admin_role`, they get admin. Otherwise if they have `editor_role`, they get editor. Otherwise `default_role` applies.
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
oidc:
|
||||
enabled: true
|
||||
issuer_url: "https://keycloak.example.com/realms/silo"
|
||||
client_id: "silo"
|
||||
client_secret: "" # use SILO_OIDC_CLIENT_SECRET env var
|
||||
redirect_url: "https://silo.example.com/auth/callback"
|
||||
admin_role: "silo-admin"
|
||||
editor_role: "silo-editor"
|
||||
default_role: "viewer"
|
||||
```
|
||||
|
||||
### CORS
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `auth.cors.allowed_origins` | []string | — | Origins allowed for cross-origin requests from browser-based clients |
|
||||
|
||||
FreeCAD and other non-browser clients use direct HTTP and are not affected by CORS. This setting is for browser-based tools running on different origins.
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
cors:
|
||||
allowed_origins:
|
||||
- "https://silo.example.com"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All environment variable overrides. These take precedence over values in `config.yaml`.
|
||||
|
||||
| Variable | Config Key | Description |
|
||||
|----------|-----------|-------------|
|
||||
| `SILO_DB_HOST` | `database.host` | PostgreSQL host |
|
||||
| `SILO_DB_NAME` | `database.name` | PostgreSQL database name |
|
||||
| `SILO_DB_USER` | `database.user` | PostgreSQL user |
|
||||
| `SILO_DB_PASSWORD` | `database.password` | PostgreSQL password |
|
||||
| `SILO_MINIO_ENDPOINT` | `storage.endpoint` | MinIO endpoint |
|
||||
| `SILO_MINIO_ACCESS_KEY` | `storage.access_key` | MinIO access key |
|
||||
| `SILO_MINIO_SECRET_KEY` | `storage.secret_key` | MinIO secret key |
|
||||
| `SILO_SESSION_SECRET` | `auth.session_secret` | Session cookie signing secret |
|
||||
| `SILO_ADMIN_USERNAME` | `auth.local.default_admin_username` | Default admin username |
|
||||
| `SILO_ADMIN_PASSWORD` | `auth.local.default_admin_password` | Default admin password |
|
||||
| `SILO_LDAP_BIND_PASSWORD` | `auth.ldap.bind_password` | LDAP service account password |
|
||||
| `SILO_OIDC_CLIENT_SECRET` | `auth.oidc.client_secret` | OIDC client secret |
|
||||
|
||||
Additionally, YAML values can reference environment variables directly using `${VAR_NAME}` syntax, which is expanded at load time via `os.ExpandEnv()`.
|
||||
|
||||
---
|
||||
|
||||
## Deployment Examples
|
||||
|
||||
### Development (no auth)
|
||||
|
||||
```yaml
|
||||
server:
|
||||
host: "0.0.0.0"
|
||||
port: 8080
|
||||
base_url: "http://localhost:8080"
|
||||
|
||||
database:
|
||||
host: "localhost"
|
||||
port: 5432
|
||||
name: "silo"
|
||||
user: "silo"
|
||||
password: "silodev"
|
||||
sslmode: "disable"
|
||||
|
||||
storage:
|
||||
endpoint: "localhost:9000"
|
||||
access_key: "minioadmin"
|
||||
secret_key: "minioadmin"
|
||||
bucket: "silo-files"
|
||||
use_ssl: false
|
||||
|
||||
schemas:
|
||||
directory: "./schemas"
|
||||
default: "kindred-rd"
|
||||
|
||||
auth:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
### Local Auth Only
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: true
|
||||
session_secret: "change-me-to-a-random-string"
|
||||
local:
|
||||
enabled: true
|
||||
default_admin_username: "admin"
|
||||
default_admin_password: "change-me"
|
||||
```
|
||||
|
||||
### LDAP / FreeIPA
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: true
|
||||
session_secret: "${SILO_SESSION_SECRET}"
|
||||
local:
|
||||
enabled: false
|
||||
ldap:
|
||||
enabled: true
|
||||
url: "ldaps://ipa.example.com"
|
||||
base_dn: "dc=example,dc=com"
|
||||
user_search_dn: "cn=users,cn=accounts,dc=example,dc=com"
|
||||
role_mapping:
|
||||
admin:
|
||||
- "cn=silo-admins,cn=groups,cn=accounts,dc=example,dc=com"
|
||||
editor:
|
||||
- "cn=engineers,cn=groups,cn=accounts,dc=example,dc=com"
|
||||
viewer:
|
||||
- "cn=silo-viewers,cn=groups,cn=accounts,dc=example,dc=com"
|
||||
```
|
||||
|
||||
### OIDC / Keycloak
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: true
|
||||
session_secret: "${SILO_SESSION_SECRET}"
|
||||
local:
|
||||
enabled: false
|
||||
oidc:
|
||||
enabled: true
|
||||
issuer_url: "https://keycloak.example.com/realms/silo"
|
||||
client_id: "silo"
|
||||
client_secret: "${SILO_OIDC_CLIENT_SECRET}"
|
||||
redirect_url: "https://silo.example.com/auth/callback"
|
||||
admin_role: "silo-admin"
|
||||
editor_role: "silo-editor"
|
||||
default_role: "viewer"
|
||||
```
|
||||
466
docs/src/silo-server/DEPLOYMENT.md
Normal file
466
docs/src/silo-server/DEPLOYMENT.md
Normal file
@@ -0,0 +1,466 @@
|
||||
# Silo Production Deployment Guide
|
||||
|
||||
This guide covers deploying Silo to a dedicated VM using external PostgreSQL and MinIO services.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Architecture](#architecture)
|
||||
- [External Services](#external-services)
|
||||
- [Quick Start](#quick-start)
|
||||
- [Initial Setup](#initial-setup)
|
||||
- [Deployment](#deployment)
|
||||
- [Configuration](#configuration)
|
||||
- [Maintenance](#maintenance)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ silo.kindred.internal │
|
||||
│ ┌───────────────────────────────────────────────────────────┐ │
|
||||
│ │ silod │ │
|
||||
│ │ (Silo API Server) │ │
|
||||
│ │ :8080 │ │
|
||||
│ └───────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────────────┐ ┌─────────────────────────────────┐
|
||||
│ psql.kindred.internal │ │ minio.kindred.internal │
|
||||
│ PostgreSQL 16 │ │ MinIO S3 │
|
||||
│ :5432 │ │ :9000 (API) │
|
||||
│ │ │ :9001 (Console) │
|
||||
└─────────────────────────┘ └─────────────────────────────────┘
|
||||
```
|
||||
|
||||
## External Services
|
||||
|
||||
The following external services are already configured:
|
||||
|
||||
| Service | Host | Database/Bucket | User |
|
||||
|---------|------|-----------------|------|
|
||||
| PostgreSQL | psql.kindred.internal:5432 | silo | silo |
|
||||
| MinIO | minio.kindred.internal:9000 | silo-files | silouser |
|
||||
|
||||
Migrations have been applied to the database.
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
For a fresh VM, run these commands:
|
||||
|
||||
```bash
|
||||
# 1. SSH to the target host
|
||||
ssh root@silo.kindred.internal
|
||||
|
||||
# 2. Download and run setup script
|
||||
curl -fsSL https://gitea.kindred.internal/kindred/silo-0062/raw/branch/main/scripts/setup-host.sh | bash
|
||||
|
||||
# 3. Configure credentials
|
||||
nano /etc/silo/silod.env
|
||||
|
||||
# 4. Deploy
|
||||
/opt/silo/src/scripts/deploy.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Initial Setup
|
||||
|
||||
Run the setup script once on `silo.kindred.internal` to prepare the host:
|
||||
|
||||
```bash
|
||||
# Option 1: If you have the repo locally
|
||||
scp scripts/setup-host.sh root@silo.kindred.internal:/tmp/
|
||||
ssh root@silo.kindred.internal 'bash /tmp/setup-host.sh'
|
||||
|
||||
# Option 2: Direct on the host
|
||||
ssh root@silo.kindred.internal
|
||||
curl -fsSL https://git.kindred.internal/kindred/silo/raw/branch/main/scripts/setup-host.sh -o /tmp/setup-host.sh
|
||||
bash /tmp/setup-host.sh
|
||||
```
|
||||
|
||||
The setup script:
|
||||
- Installs dependencies (git, Go 1.23)
|
||||
- Creates the `silo` system user
|
||||
- Creates directory structure (`/opt/silo`, `/etc/silo`)
|
||||
- Clones the repository to `/opt/silo/src`
|
||||
- Creates the environment file template
|
||||
|
||||
### Configure Credentials
|
||||
|
||||
After setup, edit the environment file with your credentials:
|
||||
|
||||
```bash
|
||||
sudo nano /etc/silo/silod.env
|
||||
```
|
||||
|
||||
Fill in the values:
|
||||
|
||||
```bash
|
||||
# Database credentials (psql.kindred.internal)
|
||||
SILO_DB_PASSWORD=your-database-password
|
||||
|
||||
# MinIO credentials (minio.kindred.internal)
|
||||
SILO_MINIO_ACCESS_KEY=silouser
|
||||
SILO_MINIO_SECRET_KEY=your-minio-secret-key
|
||||
```
|
||||
|
||||
### Verify External Services
|
||||
|
||||
Before deploying, verify connectivity to external services:
|
||||
|
||||
```bash
|
||||
# Test PostgreSQL
|
||||
psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1'
|
||||
|
||||
# Test MinIO
|
||||
curl -I http://minio.kindred.internal:9000/minio/health/live
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
### Deploy (or Update)
|
||||
|
||||
To deploy or update Silo, run the deploy script on the target host:
|
||||
|
||||
```bash
|
||||
ssh root@silo.kindred.internal
|
||||
/opt/silo/src/scripts/deploy.sh
|
||||
```
|
||||
|
||||
The deploy script:
|
||||
1. Pulls the latest code from git
|
||||
2. Builds the `silod` binary
|
||||
3. Installs configuration and schemas
|
||||
4. Installs/updates the systemd service
|
||||
5. Restarts the service
|
||||
6. Verifies health endpoints
|
||||
|
||||
### Deploy Options
|
||||
|
||||
```bash
|
||||
# Full deployment (pull, build, deploy, restart)
|
||||
sudo /opt/silo/src/scripts/deploy.sh
|
||||
|
||||
# Skip git pull (use current checkout)
|
||||
sudo /opt/silo/src/scripts/deploy.sh --no-pull
|
||||
|
||||
# Skip build (use existing binary)
|
||||
sudo /opt/silo/src/scripts/deploy.sh --no-build
|
||||
|
||||
# Just restart the service
|
||||
sudo /opt/silo/src/scripts/deploy.sh --restart-only
|
||||
|
||||
# Check service status
|
||||
sudo /opt/silo/src/scripts/deploy.sh --status
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
You can override the git repository URL and branch:
|
||||
|
||||
```bash
|
||||
export SILO_REPO_URL=https://git.kindred.internal/kindred/silo.git
|
||||
export SILO_BRANCH=main
|
||||
sudo -E /opt/silo/src/scripts/deploy.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### File Locations
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `/opt/silo/bin/silod` | Server binary |
|
||||
| `/opt/silo/src/` | Git repository checkout |
|
||||
| `/etc/silo/config.yaml` | Server configuration |
|
||||
| `/etc/silo/silod.env` | Environment variables (secrets) |
|
||||
| `/etc/silo/schemas/` | Part numbering schemas |
|
||||
| `/var/log/silo/` | Log directory |
|
||||
|
||||
### Configuration File
|
||||
|
||||
The configuration file `/etc/silo/config.yaml` is installed on first deployment and not overwritten on updates. To update it manually:
|
||||
|
||||
```bash
|
||||
sudo cp /opt/silo/src/deployments/config.prod.yaml /etc/silo/config.yaml
|
||||
sudo systemctl restart silod
|
||||
```
|
||||
|
||||
### Schemas
|
||||
|
||||
Schemas in `/etc/silo/schemas/` are updated on every deployment from the repository.
|
||||
|
||||
---
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Service Management
|
||||
|
||||
```bash
|
||||
# Check status
|
||||
sudo systemctl status silod
|
||||
|
||||
# Start/stop/restart
|
||||
sudo systemctl start silod
|
||||
sudo systemctl stop silod
|
||||
sudo systemctl restart silod
|
||||
|
||||
# Enable/disable auto-start
|
||||
sudo systemctl enable silod
|
||||
sudo systemctl disable silod
|
||||
```
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# Follow logs
|
||||
sudo journalctl -u silod -f
|
||||
|
||||
# Recent logs
|
||||
sudo journalctl -u silod -n 100
|
||||
|
||||
# Logs since a time
|
||||
sudo journalctl -u silod --since "1 hour ago"
|
||||
sudo journalctl -u silod --since "2024-01-15 10:00:00"
|
||||
```
|
||||
|
||||
### Health Checks
|
||||
|
||||
```bash
|
||||
# Basic health check
|
||||
curl http://localhost:8080/health
|
||||
|
||||
# Full readiness check (includes DB and MinIO)
|
||||
curl http://localhost:8080/ready
|
||||
```
|
||||
|
||||
### Update Deployment
|
||||
|
||||
To update to the latest version:
|
||||
|
||||
```bash
|
||||
ssh root@silo.kindred.internal
|
||||
/opt/silo/src/scripts/deploy.sh
|
||||
```
|
||||
|
||||
To deploy a specific branch or tag:
|
||||
|
||||
```bash
|
||||
cd /opt/silo/src
|
||||
git fetch --all --tags
|
||||
git checkout v1.2.3 # or a branch name
|
||||
sudo /opt/silo/src/scripts/deploy.sh --no-pull
|
||||
```
|
||||
|
||||
### Database Migrations
|
||||
|
||||
When new migrations are added, run them manually:
|
||||
|
||||
```bash
|
||||
# Check for new migrations
|
||||
ls -la /opt/silo/src/migrations/
|
||||
|
||||
# Run a specific migration
|
||||
psql -h psql.kindred.internal -U silo -d silo -f /opt/silo/src/migrations/008_new_feature.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Service Won't Start
|
||||
|
||||
1. Check logs for errors:
|
||||
```bash
|
||||
sudo journalctl -u silod -n 50
|
||||
```
|
||||
|
||||
2. Verify configuration:
|
||||
```bash
|
||||
cat /etc/silo/config.yaml
|
||||
```
|
||||
|
||||
3. Check environment file permissions:
|
||||
```bash
|
||||
ls -la /etc/silo/silod.env
|
||||
# Should be: -rw------- root silo
|
||||
```
|
||||
|
||||
4. Verify binary exists:
|
||||
```bash
|
||||
ls -la /opt/silo/bin/silod
|
||||
```
|
||||
|
||||
### Connection Refused to PostgreSQL
|
||||
|
||||
1. Test network connectivity:
|
||||
```bash
|
||||
nc -zv psql.kindred.internal 5432
|
||||
```
|
||||
|
||||
2. Test credentials:
|
||||
```bash
|
||||
source /etc/silo/silod.env
|
||||
PGPASSWORD=$SILO_DB_PASSWORD psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1'
|
||||
```
|
||||
|
||||
3. Check `pg_hba.conf` on PostgreSQL server allows connections from this host.
|
||||
|
||||
### Connection Refused to MinIO
|
||||
|
||||
1. Test network connectivity:
|
||||
```bash
|
||||
nc -zv minio.kindred.internal 9000
|
||||
```
|
||||
|
||||
2. Test with curl:
|
||||
```bash
|
||||
curl -I http://minio.kindred.internal:9000/minio/health/live
|
||||
```
|
||||
|
||||
3. Check SSL settings in config match MinIO setup:
|
||||
```yaml
|
||||
storage:
|
||||
use_ssl: true # or false
|
||||
```
|
||||
|
||||
### Health Check Fails
|
||||
|
||||
```bash
|
||||
# Check individual endpoints
|
||||
curl -v http://localhost:8080/health
|
||||
curl -v http://localhost:8080/ready
|
||||
|
||||
# If ready fails but health passes, check external services
|
||||
psql -h psql.kindred.internal -U silo -d silo -c 'SELECT 1'
|
||||
curl http://minio.kindred.internal:9000/minio/health/live
|
||||
```
|
||||
|
||||
### Build Fails
|
||||
|
||||
1. Check Go is installed:
|
||||
```bash
|
||||
go version
|
||||
# Should be 1.23+
|
||||
```
|
||||
|
||||
2. Check source is present:
|
||||
```bash
|
||||
ls -la /opt/silo/src/
|
||||
```
|
||||
|
||||
3. Try manual build:
|
||||
```bash
|
||||
cd /opt/silo/src
|
||||
go build -v ./cmd/silod
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## SSL/TLS with FreeIPA and Nginx
|
||||
|
||||
For production deployments, Silo should be served over HTTPS using nginx as a reverse proxy with certificates from FreeIPA.
|
||||
|
||||
### Setup IPA and Nginx
|
||||
|
||||
Run the IPA/nginx setup script after the basic host setup:
|
||||
|
||||
```bash
|
||||
sudo /opt/silo/src/scripts/setup-ipa-nginx.sh
|
||||
```
|
||||
|
||||
This script:
|
||||
1. Installs FreeIPA client and nginx
|
||||
2. Enrolls the host in FreeIPA domain
|
||||
3. Requests SSL certificate from IPA CA (auto-renewed by certmonger)
|
||||
4. Configures nginx as reverse proxy (HTTP → HTTPS redirect)
|
||||
5. Opens firewall ports 80 and 443
|
||||
|
||||
### Manual Steps After Script
|
||||
|
||||
1. Verify certificate was issued:
|
||||
```bash
|
||||
getcert list
|
||||
```
|
||||
|
||||
2. The silo config is already updated to use `https://silo.kindred.internal` as base URL. Restart silo:
|
||||
```bash
|
||||
sudo systemctl restart silod
|
||||
```
|
||||
|
||||
3. Test the setup:
|
||||
```bash
|
||||
curl https://silo.kindred.internal/health
|
||||
```
|
||||
|
||||
### Certificate Management
|
||||
|
||||
Certificates are automatically renewed by certmonger. Check status:
|
||||
|
||||
```bash
|
||||
# List all tracked certificates
|
||||
getcert list
|
||||
|
||||
# Check specific certificate
|
||||
getcert list -f /etc/ssl/silo/silo.crt
|
||||
|
||||
# Manual renewal if needed
|
||||
getcert resubmit -f /etc/ssl/silo/silo.crt
|
||||
```
|
||||
|
||||
### Trusting IPA CA on Clients
|
||||
|
||||
For clients to trust the Silo HTTPS certificate, they need the IPA CA:
|
||||
|
||||
```bash
|
||||
# Download CA cert
|
||||
curl -o /tmp/ipa-ca.crt https://ipa.kindred.internal/ipa/config/ca.crt
|
||||
|
||||
# Ubuntu/Debian
|
||||
sudo cp /tmp/ipa-ca.crt /usr/local/share/ca-certificates/ipa-ca.crt
|
||||
sudo update-ca-certificates
|
||||
|
||||
# RHEL/Fedora
|
||||
sudo cp /tmp/ipa-ca.crt /etc/pki/ca-trust/source/anchors/
|
||||
sudo update-ca-trust
|
||||
```
|
||||
|
||||
### Nginx Configuration
|
||||
|
||||
The nginx config is installed at `/etc/nginx/sites-available/silo`. Key settings:
|
||||
|
||||
- HTTP redirects to HTTPS
|
||||
- TLS 1.2/1.3 only with strong ciphers
|
||||
- Proxies to `127.0.0.1:8080` (silod)
|
||||
- 100MB max upload size for CAD files
|
||||
- Security headers (X-Frame-Options, etc.)
|
||||
|
||||
To modify:
|
||||
```bash
|
||||
sudo nano /etc/nginx/sites-available/silo
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- [ ] `/etc/silo/silod.env` has mode 600 (`chmod 600`)
|
||||
- [ ] Database password is strong and unique
|
||||
- [ ] MinIO credentials are specific to silo (not admin)
|
||||
- [ ] SSL/TLS enabled for PostgreSQL (`sslmode: require`)
|
||||
- [ ] SSL/TLS enabled for MinIO (`use_ssl: true`) if available
|
||||
- [ ] HTTPS enabled via nginx reverse proxy
|
||||
- [ ] Silod listens on localhost only (`host: 127.0.0.1`)
|
||||
- [ ] Firewall allows only ports 80, 443 (not 8080)
|
||||
- [ ] Service runs as non-root `silo` user
|
||||
- [ ] Host enrolled in FreeIPA for centralized auth (future)
|
||||
452
docs/src/silo-server/GAP_ANALYSIS.md
Normal file
452
docs/src/silo-server/GAP_ANALYSIS.md
Normal file
@@ -0,0 +1,452 @@
|
||||
# Silo Gap Analysis and Revision Control Roadmap
|
||||
|
||||
**Date:** 2026-02-08
|
||||
**Status:** Analysis Complete (Updated)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document analyzes the current state of the Silo project against its specification, identifies documentation and feature gaps, and outlines a roadmap for enhanced revision control capabilities.
|
||||
|
||||
---
|
||||
|
||||
## 1. Documentation Gap Analysis
|
||||
|
||||
### 1.1 Current Documentation
|
||||
|
||||
| Document | Coverage | Status |
|
||||
|----------|----------|--------|
|
||||
| `README.md` | Quick start, overview, component map | Current |
|
||||
| `docs/SPECIFICATION.md` | Design specification, API reference | Current |
|
||||
| `docs/STATUS.md` | Implementation status summary | Current |
|
||||
| `docs/DEPLOYMENT.md` | Production deployment guide | Current |
|
||||
| `docs/CONFIGURATION.md` | Configuration reference (all config.yaml options) | Current |
|
||||
| `docs/AUTH.md` | Authentication system design | Current |
|
||||
| `docs/AUTH_USER_GUIDE.md` | User guide for login, tokens, and roles | Current |
|
||||
| `docs/GAP_ANALYSIS.md` | Revision control roadmap | Current |
|
||||
| `ROADMAP.md` | Feature roadmap and SOLIDWORKS PDM comparison | Current |
|
||||
| `frontend-spec.md` | React SPA frontend specification | Current |
|
||||
|
||||
### 1.2 Documentation Gaps (Priority Order)
|
||||
|
||||
#### High Priority
|
||||
|
||||
| Gap | Impact | Effort | Status |
|
||||
|-----|--------|--------|--------|
|
||||
| **API Reference** | Users cannot integrate programmatically | Medium | Addressed in SPECIFICATION.md Section 11 |
|
||||
| **Deployment Guide** | Cannot deploy to production | Medium | Complete (`docs/DEPLOYMENT.md`) |
|
||||
| **Database Schema Guide** | Migration troubleshooting difficult | Low | Open |
|
||||
| **Configuration Reference** | config.yaml options undocumented | Low | Complete (`docs/CONFIGURATION.md`) |
|
||||
|
||||
#### Medium Priority
|
||||
|
||||
| Gap | Impact | Effort |
|
||||
|-----|--------|--------|
|
||||
| **User Workflows** | Users lack step-by-step guidance | Medium |
|
||||
| **Troubleshooting Guide** | Support burden increases | Medium |
|
||||
| **Developer Setup Guide** | Onboarding friction | Low |
|
||||
|
||||
#### Lower Priority
|
||||
|
||||
| Gap | Impact | Effort |
|
||||
|-----|--------|--------|
|
||||
| **CHANGELOG.md** | Version history unclear | Low |
|
||||
| **Architecture Decision Records** | Design rationale lost | Medium |
|
||||
| **Integration Guide** | Third-party integration unclear | High |
|
||||
|
||||
### 1.3 Recommended Actions
|
||||
|
||||
1. ~~**Consolidate specs**: Remove `silo-spec.md` duplicate~~ Done (deleted)
|
||||
2. ~~**Create API reference**: Full REST endpoint documentation~~ Addressed in SPECIFICATION.md
|
||||
3. ~~**Create `docs/DEPLOYMENT.md`**: Production deployment guide~~ Done
|
||||
4. ~~**Create configuration reference**: Document all `config.yaml` options~~ Done (`docs/CONFIGURATION.md`)
|
||||
5. **Create database schema guide**: Document migrations and troubleshooting
|
||||
|
||||
---
|
||||
|
||||
## 2. Current Revision Control Implementation
|
||||
|
||||
### 2.1 What's Implemented
|
||||
|
||||
| Feature | Status | Location |
|
||||
|---------|--------|----------|
|
||||
| Append-only revision history | Complete | `internal/db/items.go` |
|
||||
| Sequential revision numbering | Complete | Database trigger |
|
||||
| Property snapshots (JSONB) | Complete | `revisions.properties` |
|
||||
| File versioning (MinIO) | Complete | `internal/storage/` |
|
||||
| SHA256 checksums | Complete | Captured on upload |
|
||||
| Revision comments | Complete | `revisions.comment` |
|
||||
| User attribution | Complete | `revisions.created_by` |
|
||||
| Property schema versioning | Complete | Migration 005 |
|
||||
| BOM revision pinning | Complete | `relationships.child_revision` |
|
||||
|
||||
### 2.2 Database Schema
|
||||
|
||||
```sql
|
||||
-- Current revision schema (migrations/001_initial.sql)
|
||||
CREATE TABLE revisions (
|
||||
id UUID PRIMARY KEY,
|
||||
item_id UUID REFERENCES items(id) ON DELETE CASCADE,
|
||||
revision_number INTEGER NOT NULL,
|
||||
properties JSONB NOT NULL DEFAULT '{}',
|
||||
file_key TEXT,
|
||||
file_version TEXT, -- MinIO version ID
|
||||
file_checksum TEXT, -- SHA256
|
||||
file_size BIGINT,
|
||||
thumbnail_key TEXT,
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
created_by TEXT,
|
||||
comment TEXT,
|
||||
property_schema_version INTEGER DEFAULT 1,
|
||||
UNIQUE(item_id, revision_number)
|
||||
);
|
||||
```
|
||||
|
||||
### 2.3 API Endpoints
|
||||
|
||||
| Endpoint | Method | Status |
|
||||
|----------|--------|--------|
|
||||
| `/api/items/{pn}/revisions` | GET | Implemented |
|
||||
| `/api/items/{pn}/revisions` | POST | Implemented |
|
||||
| `/api/items/{pn}/revisions/{rev}` | GET | Implemented |
|
||||
| `/api/items/{pn}/file` | POST | Implemented |
|
||||
| `/api/items/{pn}/file` | GET | Implemented (latest) |
|
||||
| `/api/items/{pn}/file/{rev}` | GET | Implemented |
|
||||
|
||||
### 2.4 Client Integration
|
||||
|
||||
FreeCAD workbench maintained in separate [silo-mod](https://git.kindred-systems.com/kindred/silo-mod) repository. The server provides the revision and file endpoints consumed by the workbench.
|
||||
|
||||
---
|
||||
|
||||
## 3. Revision Control Gaps
|
||||
|
||||
### 3.1 Critical Gaps
|
||||
|
||||
| Gap | Description | Impact | Status |
|
||||
|-----|-------------|--------|--------|
|
||||
| ~~**No rollback**~~ | ~~Cannot revert to previous revision~~ | ~~Data recovery difficult~~ | **Implemented** |
|
||||
| ~~**No comparison**~~ | ~~Cannot diff between revisions~~ | ~~Change tracking manual~~ | **Implemented** |
|
||||
| **No locking** | No concurrent edit protection | Multi-user unsafe | Open |
|
||||
| **No approval workflow** | No release/sign-off process | Quality control gap | Open |
|
||||
|
||||
### 3.2 Important Gaps
|
||||
|
||||
| Gap | Description | Impact | Status |
|
||||
|-----|-------------|--------|--------|
|
||||
| **No branching** | Linear history only | No experimental variants | Open |
|
||||
| ~~**No tagging**~~ | ~~No named milestones~~ | ~~Release tracking manual~~ | **Implemented** (revision labels) |
|
||||
| ~~**No audit log**~~ | ~~Actions not logged separately~~ | ~~Compliance gap~~ | **Implemented** (migration 009, `audit_log` table + completeness scoring) |
|
||||
| **Thumbnail missing** | Schema exists, not populated | No visual preview | Open |
|
||||
|
||||
### 3.3 Nice-to-Have Gaps
|
||||
|
||||
| Gap | Description | Impact |
|
||||
|-----|-------------|--------|
|
||||
| **No search** | Cannot search revision comments | Discovery limited |
|
||||
| **No retention policy** | Revisions never expire | Storage grows unbounded |
|
||||
| **No delta storage** | Full file per revision | Storage inefficient |
|
||||
| **No notifications** | No change alerts | Manual monitoring required |
|
||||
|
||||
---
|
||||
|
||||
## 4. Revision Control Roadmap
|
||||
|
||||
### Phase 1: Foundation -- COMPLETE
|
||||
|
||||
**Goal:** Enable safe single-user revision management
|
||||
|
||||
All Phase 1 items have been implemented:
|
||||
|
||||
- **Rollback**: `POST /api/items/{pn}/revisions/{rev}/rollback` - creates new revision from old
|
||||
- **Revision Comparison**: `GET /api/items/{pn}/revisions/compare?from={rev1}&to={rev2}` - property and file diffs
|
||||
- **Revision Labels/Status**: `PATCH /api/items/{pn}/revisions/{rev}` - status (draft/review/released/obsolete) and arbitrary labels via migration 007
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Multi-User Support
|
||||
|
||||
**Goal:** Enable safe concurrent editing
|
||||
|
||||
#### 2.1 Pessimistic Locking
|
||||
```
|
||||
Effort: High | Priority: High | Risk: Medium | Status: Not Started
|
||||
```
|
||||
|
||||
**Database Migration:**
|
||||
```sql
|
||||
CREATE TABLE item_locks (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE,
|
||||
locked_by TEXT NOT NULL,
|
||||
locked_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
expires_at TIMESTAMPTZ NOT NULL,
|
||||
lock_type TEXT NOT NULL DEFAULT 'exclusive',
|
||||
comment TEXT,
|
||||
UNIQUE(item_id)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_locks_expires ON item_locks(expires_at);
|
||||
```
|
||||
|
||||
**API Endpoints:**
|
||||
```
|
||||
POST /api/items/{pn}/lock # Acquire lock
|
||||
DELETE /api/items/{pn}/lock # Release lock
|
||||
GET /api/items/{pn}/lock # Check lock status
|
||||
```
|
||||
|
||||
#### 2.2 Authentication -- COMPLETE
|
||||
|
||||
Authentication is fully implemented with three backends (local/bcrypt, LDAP/FreeIPA, OIDC/Keycloak), role-based access control (admin > editor > viewer), API token management, and PostgreSQL-backed sessions. See `docs/AUTH.md` for full details.
|
||||
|
||||
- Migration: `009_auth.sql`
|
||||
- Code: `internal/auth/`, `internal/api/middleware.go`
|
||||
|
||||
#### 2.3 Audit Logging -- COMPLETE
|
||||
|
||||
Audit logging is implemented via migration 009 with the `audit_log` table and completeness scoring system. Endpoints:
|
||||
- `GET /api/audit/completeness` — summary of all items
|
||||
- `GET /api/audit/completeness/{partNumber}` — per-item scoring with weighted fields and tier classification
|
||||
|
||||
Code: `internal/api/audit_handlers.go`
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Advanced Features
|
||||
|
||||
**Goal:** Enhanced revision management capabilities
|
||||
|
||||
#### 3.1 Branching Support
|
||||
```
|
||||
Effort: High | Priority: Low | Risk: High
|
||||
```
|
||||
|
||||
**Concept:** Named revision streams per item
|
||||
|
||||
**Database Migration:**
|
||||
```sql
|
||||
CREATE TABLE revision_branches (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE,
|
||||
name TEXT NOT NULL, -- 'main', 'experimental', 'customer-variant'
|
||||
base_revision INTEGER, -- Branch point
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
created_by TEXT,
|
||||
UNIQUE(item_id, name)
|
||||
);
|
||||
|
||||
ALTER TABLE revisions ADD COLUMN branch_id UUID REFERENCES revision_branches(id);
|
||||
```
|
||||
|
||||
**Complexity:** Requires merge logic, conflict resolution UI
|
||||
|
||||
#### 3.2 Release Management
|
||||
```
|
||||
Effort: Medium | Priority: Medium | Risk: Low
|
||||
```
|
||||
|
||||
**Database Migration:**
|
||||
```sql
|
||||
CREATE TABLE releases (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
name TEXT NOT NULL UNIQUE, -- 'v1.0', '2026-Q1'
|
||||
description TEXT,
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
created_by TEXT,
|
||||
status TEXT DEFAULT 'draft' -- 'draft', 'released', 'archived'
|
||||
);
|
||||
|
||||
CREATE TABLE release_items (
|
||||
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
|
||||
release_id UUID REFERENCES releases(id) ON DELETE CASCADE,
|
||||
item_id UUID REFERENCES items(id) ON DELETE CASCADE,
|
||||
revision_number INTEGER NOT NULL,
|
||||
UNIQUE(release_id, item_id)
|
||||
);
|
||||
```
|
||||
|
||||
**Use Case:** Snapshot a set of items at specific revisions for a product release
|
||||
|
||||
#### 3.3 Thumbnail Generation
|
||||
```
|
||||
Effort: Medium | Priority: Low | Risk: Low
|
||||
```
|
||||
|
||||
**Implementation Options:**
|
||||
1. FreeCAD headless rendering (python script)
|
||||
2. External service (e.g., CAD thumbnail microservice)
|
||||
3. User-uploaded thumbnails
|
||||
|
||||
**Changes:**
|
||||
- Add thumbnail generation on file upload
|
||||
- Store in MinIO at `thumbnails/{part_number}/rev{n}.png`
|
||||
- Expose via `GET /api/items/{pn}/thumbnail/{rev}`
|
||||
|
||||
---
|
||||
|
||||
## 5. Recommended Implementation Order
|
||||
|
||||
### Completed
|
||||
|
||||
1. ~~**Revision Comparison API**~~ - Implemented
|
||||
2. ~~**Rollback Support**~~ - Implemented
|
||||
3. ~~**Revision Labels/Status**~~ - Implemented (migration 007)
|
||||
|
||||
### Recently Completed
|
||||
|
||||
4. ~~**Authentication**~~ - Implemented (3 backends: local, LDAP, OIDC; RBAC; API tokens; sessions)
|
||||
5. ~~**Audit Logging**~~ - Implemented (audit_log table, completeness scoring)
|
||||
|
||||
### Next (Short-term)
|
||||
|
||||
6. **Pessimistic Locking** - Required before multi-user
|
||||
|
||||
### Medium-term (3-6 Months)
|
||||
|
||||
7. **Release Management** - Product milestone tracking
|
||||
8. **Thumbnail Generation** - Visual preview capability
|
||||
|
||||
### Long-term (Future)
|
||||
|
||||
10. **Branching** - Complex, defer until needed
|
||||
11. **Delta Storage** - Optimization, not critical
|
||||
12. **Notifications** - Nice-to-have workflow enhancement
|
||||
|
||||
---
|
||||
|
||||
## 6. Migration Considerations
|
||||
|
||||
### Part Number Format Migration (Completed)
|
||||
|
||||
The recent migration from `XXXXX-CCC-NNNN` to `CCC-NNNN` format has been completed:
|
||||
|
||||
- Database migration: `migrations/006_project_tags.sql`
|
||||
- Schema update: `schemas/kindred-rd.yaml` v3
|
||||
- Projects: Now many-to-many tags instead of embedded in part number
|
||||
- File paths: `~/projects/cad/{category}_{name}/{part_number}_{description}.FCStd`
|
||||
|
||||
### Future Schema Migrations
|
||||
|
||||
The property schema versioning framework (`property_schema_version`, `property_migrations` table) is in place but lacks:
|
||||
|
||||
- Automated migration runners
|
||||
- Rollback capability for failed migrations
|
||||
- Dry-run validation mode
|
||||
|
||||
**Recommendation:** Build migration tooling before adding complex property schemas.
|
||||
|
||||
---
|
||||
|
||||
## 7. Open Questions (from Specification)
|
||||
|
||||
These design decisions remain unresolved:
|
||||
|
||||
1. **Property change triggers** - Should editing properties auto-create revision?
|
||||
2. **Revision metadata editing** - Allow comment updates post-creation?
|
||||
3. **Soft delete behavior** - Archive or hard delete revisions?
|
||||
4. **File diff strategy** - Exploded FCStd storage for better diffing?
|
||||
5. **Retention policy** - How long to keep old revisions?
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: File Structure
|
||||
|
||||
Revision endpoints, status, labels, authentication, audit logging, and file attachments are implemented. Current structure:
|
||||
|
||||
```
|
||||
internal/
|
||||
api/
|
||||
audit_handlers.go # Audit/completeness endpoints
|
||||
auth_handlers.go # Login, tokens, OIDC
|
||||
bom_handlers.go # Flat BOM, cost roll-up
|
||||
file_handlers.go # Presigned uploads, item files, thumbnails
|
||||
handlers.go # Items, schemas, projects, revisions
|
||||
middleware.go # Auth middleware
|
||||
odoo_handlers.go # Odoo integration endpoints
|
||||
routes.go # Route registration (75 endpoints)
|
||||
search.go # Fuzzy search
|
||||
auth/
|
||||
auth.go # Auth service: local, LDAP, OIDC
|
||||
db/
|
||||
items.go # Item and revision repository
|
||||
item_files.go # File attachment repository
|
||||
relationships.go # BOM repository
|
||||
projects.go # Project repository
|
||||
storage/
|
||||
storage.go # MinIO file storage helpers
|
||||
migrations/
|
||||
001_initial.sql # Core schema
|
||||
...
|
||||
011_item_files.sql # Item file attachments (latest)
|
||||
```
|
||||
|
||||
Future features would add:
|
||||
|
||||
```
|
||||
internal/
|
||||
api/
|
||||
lock_handlers.go # Locking endpoints
|
||||
db/
|
||||
locks.go # Lock repository
|
||||
releases.go # Release repository
|
||||
migrations/
|
||||
012_item_locks.sql # Locking table
|
||||
013_releases.sql # Release management
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix B: API Additions Summary
|
||||
|
||||
### Phase 1 Endpoints (Implemented)
|
||||
```
|
||||
GET /api/items/{pn}/revisions/compare # Diff two revisions
|
||||
POST /api/items/{pn}/revisions/{rev}/rollback # Create revision from old
|
||||
PATCH /api/items/{pn}/revisions/{rev} # Update status/labels
|
||||
```
|
||||
|
||||
### Phase 2 Endpoints
|
||||
|
||||
**Authentication (Implemented):**
|
||||
```
|
||||
GET /api/auth/me # Current user info
|
||||
GET /api/auth/tokens # List API tokens
|
||||
POST /api/auth/tokens # Create API token
|
||||
DELETE /api/auth/tokens/{id} # Revoke API token
|
||||
```
|
||||
|
||||
**Audit (Implemented):**
|
||||
```
|
||||
GET /api/audit/completeness # All items completeness summary
|
||||
GET /api/audit/completeness/{partNumber} # Per-item scoring
|
||||
```
|
||||
|
||||
**File Attachments (Implemented):**
|
||||
```
|
||||
GET /api/auth/config # Auth config (public)
|
||||
POST /api/uploads/presign # Presigned upload URL
|
||||
GET /api/items/{pn}/files # List item files
|
||||
POST /api/items/{pn}/files # Associate file with item
|
||||
DELETE /api/items/{pn}/files/{fileId} # Delete file association
|
||||
PUT /api/items/{pn}/thumbnail # Set item thumbnail
|
||||
GET /api/items/{pn}/bom/flat # Flattened BOM
|
||||
GET /api/items/{pn}/bom/cost # Assembly cost roll-up
|
||||
```
|
||||
|
||||
**Locking (Not Implemented):**
|
||||
```
|
||||
POST /api/items/{pn}/lock # Acquire lock
|
||||
DELETE /api/items/{pn}/lock # Release lock
|
||||
GET /api/items/{pn}/lock # Check lock status
|
||||
```
|
||||
|
||||
### Phase 3 Endpoints (Not Implemented)
|
||||
```
|
||||
GET /api/releases # List releases
|
||||
POST /api/releases # Create release
|
||||
GET /api/releases/{name} # Get release details
|
||||
POST /api/releases/{name}/items # Add items to release
|
||||
GET /api/items/{pn}/thumbnail/{rev} # Get thumbnail
|
||||
```
|
||||
536
docs/src/silo-server/ROADMAP.md
Normal file
536
docs/src/silo-server/ROADMAP.md
Normal file
@@ -0,0 +1,536 @@
|
||||
# Silo Roadmap
|
||||
|
||||
**Version:** 1.1
|
||||
**Date:** February 2026
|
||||
**Purpose:** Project inventory, SOLIDWORKS PDM gap analysis, and development roadmap
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Executive Summary](#executive-summary)
|
||||
2. [Current Project Inventory](#current-project-inventory)
|
||||
3. [SOLIDWORKS PDM Gap Analysis](#solidworks-pdm-gap-analysis)
|
||||
4. [Feature Roadmap](#feature-roadmap)
|
||||
5. [Implementation Phases](#implementation-phases)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Silo is an R&D-oriented item database and part management system. It provides configurable part number generation, revision tracking, BOM management, and file versioning through MinIO storage. CAD integration (FreeCAD workbench, LibreOffice Calc extension) is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)).
|
||||
|
||||
This document compares Silo's current capabilities against SOLIDWORKS PDM—the industry-leading product data management solution—to identify gaps and prioritize future development.
|
||||
|
||||
### Key Differentiators
|
||||
|
||||
| Aspect | Silo | SOLIDWORKS PDM |
|
||||
|--------|------|----------------|
|
||||
| **Target CAD** | FreeCAD / Kindred Create (open source) | SOLIDWORKS (proprietary) |
|
||||
| **Part Numbering** | Schema-as-configuration (YAML) | Fixed format with some customization |
|
||||
| **Licensing** | Open source / Kindred Proprietary | Commercial ($3,000-$10,000+ per seat) |
|
||||
| **Storage** | PostgreSQL + MinIO (S3-compatible) | SQL Server + File Archive |
|
||||
| **Philosophy** | R&D-oriented, lightweight | Enterprise-grade, comprehensive |
|
||||
|
||||
---
|
||||
|
||||
## Current Project Inventory
|
||||
|
||||
### Implemented Features (MVP Complete)
|
||||
|
||||
#### Core Database System
|
||||
- PostgreSQL schema with 11 migrations
|
||||
- UUID-based identifiers throughout
|
||||
- Soft delete support via `archived_at` timestamps
|
||||
- Atomic sequence generation for part numbers
|
||||
|
||||
#### Part Number Generation
|
||||
- YAML schema parser with validation
|
||||
- Segment types: `string`, `enum`, `serial`, `constant`
|
||||
- Scope templates for serial counters (e.g., `{category}`, `{project}`)
|
||||
- Format templates for custom output
|
||||
|
||||
#### Item Management
|
||||
- Full CRUD operations for items
|
||||
- Item types: part, assembly, drawing, document, tooling, purchased, electrical, software
|
||||
- Custom properties via JSONB storage
|
||||
- Project tagging with many-to-many relationships
|
||||
|
||||
#### Revision Control
|
||||
- Append-only revision history
|
||||
- Revision metadata: properties, file reference, checksum, comment
|
||||
- Status tracking: draft, review, released, obsolete
|
||||
- Labels/tags per revision
|
||||
- Revision comparison (diff)
|
||||
- Rollback functionality
|
||||
|
||||
#### File Management
|
||||
- MinIO integration with versioning
|
||||
- File upload/download via REST API
|
||||
- SHA256 checksums for integrity
|
||||
- Storage path: `items/{partNumber}/rev{N}.FCStd`
|
||||
|
||||
#### Bill of Materials (BOM)
|
||||
- Relationship types: component, alternate, reference
|
||||
- Multi-level BOM (recursive expansion with configurable depth)
|
||||
- Where-used queries (reverse parent lookup)
|
||||
- BOM CSV and ODS export/import with cycle detection
|
||||
- Reference designators for electronics
|
||||
- Quantity tracking with units
|
||||
- Revision-specific child linking
|
||||
|
||||
#### Project Management
|
||||
- Project CRUD operations
|
||||
- Unique project codes (2-10 characters)
|
||||
- Item-to-project tagging
|
||||
- Project-filtered queries
|
||||
|
||||
#### Data Import/Export
|
||||
- CSV export with configurable properties
|
||||
- CSV import with dry-run validation
|
||||
- ODS spreadsheet import/export (items, BOMs, project sheets)
|
||||
- Template generation for import formatting
|
||||
|
||||
#### API & Web Interface
|
||||
- REST API with 75 endpoints
|
||||
- Authentication: local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak
|
||||
- Role-based access control (admin > editor > viewer)
|
||||
- API token management (SHA-256 hashed)
|
||||
- Session management (PostgreSQL-backed, 24h lifetime)
|
||||
- CSRF protection (nosurf on web forms)
|
||||
- Middleware: logging, CORS, recovery, request ID
|
||||
- Web UI — React SPA (Vite + TypeScript, Catppuccin Mocha theme)
|
||||
- Fuzzy search
|
||||
- Health and readiness probes
|
||||
|
||||
#### Audit & Completeness
|
||||
- Audit logging (database table with user/action/resource tracking)
|
||||
- Item completeness scoring with weighted fields
|
||||
- Category-specific property validation
|
||||
- Tier classification (critical/low/partial/good/complete)
|
||||
|
||||
#### Configuration
|
||||
- YAML configuration with environment variable overrides
|
||||
- Multi-schema support
|
||||
- Docker Compose deployment ready
|
||||
|
||||
### Partially Implemented
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull sync operations are stubs |
|
||||
| Date segment type | Not started | Schema parser placeholder exists |
|
||||
| Part number validation | Not started | API accepts but doesn't validate format |
|
||||
| Location hierarchy CRUD | Schema only | Tables exist, no API endpoints |
|
||||
| Inventory tracking | Schema only | Tables exist, no API endpoints |
|
||||
| Unit tests | Partial | 9 Go test files across api, db, ods, partnum, schema packages |
|
||||
|
||||
### Infrastructure Status
|
||||
|
||||
| Component | Status |
|
||||
|-----------|--------|
|
||||
| PostgreSQL | Running (psql.kindred.internal) |
|
||||
| MinIO | Configured in Docker Compose |
|
||||
| Silo API Server | Builds successfully |
|
||||
| Docker Compose | Complete (dev and production) |
|
||||
| systemd service | Unit file and env template ready |
|
||||
| Deployment scripts | setup-host, deploy, init-db, setup-ipa-nginx |
|
||||
|
||||
---
|
||||
|
||||
## SOLIDWORKS PDM Gap Analysis
|
||||
|
||||
This section compares Silo's capabilities against SOLIDWORKS PDM features. Gaps are categorized by priority and implementation complexity.
|
||||
|
||||
### Legend
|
||||
- **Silo Status:** Full / Partial / None
|
||||
- **Priority:** Critical / High / Medium / Low
|
||||
- **Complexity:** Simple / Moderate / Complex
|
||||
|
||||
---
|
||||
|
||||
### 1. Version Control & Revision Management
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| Check-in/check-out | Full pessimistic locking | None | High | Moderate |
|
||||
| Version history | Complete with branching | Full (linear) | - | - |
|
||||
| Revision labels | A, B, C or custom schemes | Full (custom labels) | - | - |
|
||||
| Rollback/restore | Full | Full | - | - |
|
||||
| Compare revisions | Visual + metadata diff | Metadata diff only | Medium | Complex |
|
||||
| Get Latest Revision | One-click retrieval | Partial (API only) | Medium | Simple |
|
||||
|
||||
**Gap Analysis:**
|
||||
Silo lacks pessimistic locking (check-out), which is critical for multi-user CAD environments where file merging is impractical. Visual diff comparison would require FreeCAD integration for CAD file visualization.
|
||||
|
||||
---
|
||||
|
||||
### 2. Workflow Management
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| Custom workflows | Full visual designer | None | Critical | Complex |
|
||||
| State transitions | Configurable with permissions | Basic (status field only) | Critical | Complex |
|
||||
| Parallel approvals | Multiple approvers required | None | High | Complex |
|
||||
| Automatic transitions | Timer/condition-based | None | Medium | Moderate |
|
||||
| Email notifications | On state change | None | High | Moderate |
|
||||
| ECO process | Built-in change management | None | High | Complex |
|
||||
| Child state conditions | Block parent if children invalid | None | Medium | Moderate |
|
||||
|
||||
**Gap Analysis:**
|
||||
Workflow management is the largest functional gap. SOLIDWORKS PDM offers sophisticated state machines with parallel approvals, automatic transitions, and deep integration with engineering change processes. Silo currently has only a simple status field (draft/review/released/obsolete) with no transition rules or approval processes.
|
||||
|
||||
---
|
||||
|
||||
### 3. User Management & Security
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| User authentication | Windows AD, LDAP | Full (local, LDAP, OIDC) | - | - |
|
||||
| Role-based permissions | Granular per folder/state | Partial (3-tier role model) | Medium | Moderate |
|
||||
| Group management | Full | None | Medium | Moderate |
|
||||
| Folder permissions | Read/write/delete per folder | None | Medium | Moderate |
|
||||
| State permissions | Actions allowed per state | None | High | Moderate |
|
||||
| Audit trail | Complete action logging | Full | - | - |
|
||||
| Private files | Pre-check-in visibility control | None | Low | Simple |
|
||||
|
||||
**Gap Analysis:**
|
||||
Authentication is implemented with three backends (local, LDAP/FreeIPA, OIDC/Keycloak) and a 3-tier role model (admin > editor > viewer). Audit logging captures user actions. Remaining gaps: group management, folder-level permissions, and state-based permission rules.
|
||||
|
||||
---
|
||||
|
||||
### 4. Search & Discovery
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| Metadata search | Full with custom cards | Partial (API query params + fuzzy) | High | Moderate |
|
||||
| Full-text content search | iFilters for Office, CAD | None | Medium | Complex |
|
||||
| Quick search | Toolbar with history | Partial (fuzzy search API) | Medium | Simple |
|
||||
| Saved searches | User-defined favorites | None | Medium | Simple |
|
||||
| Advanced operators | AND, OR, NOT, wildcards | None | Medium | Simple |
|
||||
| Multi-variable search | Search across multiple fields | None | Medium | Simple |
|
||||
| Where-used search | Find all assemblies using part | Full | - | - |
|
||||
|
||||
**Gap Analysis:**
|
||||
Silo has API-level filtering, fuzzy search, and where-used queries. Remaining gaps: saved searches, advanced search operators, and a richer search UI. Content search (searching within CAD files) is not planned for the server.
|
||||
|
||||
---
|
||||
|
||||
### 5. BOM Management
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| Single-level BOM | Yes | Full | - | - |
|
||||
| Multi-level BOM | Indented/exploded views | Full (recursive, configurable depth) | - | - |
|
||||
| BOM comparison | Between revisions | None | Medium | Moderate |
|
||||
| BOM export | Excel, XML, ERP formats | Full (CSV, ODS) | - | - |
|
||||
| BOM import | Bulk BOM loading | Full (CSV with upsert) | - | - |
|
||||
| Calculated BOMs | Quantities rolled up | None | Medium | Moderate |
|
||||
| Reference designators | Full support | Full | - | - |
|
||||
| Alternate parts | Substitute tracking | Full | - | - |
|
||||
|
||||
**Gap Analysis:**
|
||||
Multi-level BOM retrieval (recursive CTE with configurable depth) and BOM export (CSV, ODS) are implemented. BOM import supports CSV with upsert and cycle detection. Remaining gap: BOM comparison between revisions.
|
||||
|
||||
---
|
||||
|
||||
### 6. CAD Integration
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| Native CAD add-in | Deep SOLIDWORKS integration | FreeCAD workbench (silo-mod) | Medium | Complex |
|
||||
| Property mapping | Bi-directional sync | Planned (silo-mod) | Medium | Moderate |
|
||||
| Task pane | Embedded in CAD UI | Auth dock panel (silo-mod) | Medium | Complex |
|
||||
| Lightweight components | Handle without full load | N/A | - | - |
|
||||
| Drawing/model linking | Automatic association | Manual | Medium | Moderate |
|
||||
| Multi-CAD support | Third-party formats | FreeCAD only | Low | - |
|
||||
|
||||
**Gap Analysis:**
|
||||
CAD integration is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The Silo server provides the REST API endpoints consumed by those clients.
|
||||
|
||||
---
|
||||
|
||||
### 7. External Integrations
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| ERP integration | SAP, Dynamics, etc. | Partial (Odoo stubs) | Medium | Complex |
|
||||
| API access | Full COM/REST API | Full REST API (75 endpoints) | - | - |
|
||||
| Dispatch scripts | Automation without coding | None | Medium | Moderate |
|
||||
| Task scheduler | Background processing | None | Medium | Moderate |
|
||||
| Email system | SMTP integration | None | High | Simple |
|
||||
| Web portal | Browser access | Full (React SPA + auth) | - | - |
|
||||
|
||||
**Gap Analysis:**
|
||||
Silo has a comprehensive REST API (75 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation.
|
||||
|
||||
---
|
||||
|
||||
### 8. Reporting & Analytics
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| Standard reports | Inventory, usage, activity | None | Medium | Moderate |
|
||||
| Custom reports | User-defined queries | None | Medium | Moderate |
|
||||
| Dashboard | Visual KPIs | None | Low | Moderate |
|
||||
| Export formats | PDF, Excel, CSV | CSV and ODS | Medium | Simple |
|
||||
|
||||
**Gap Analysis:**
|
||||
Reporting capabilities are absent. Basic reports (item counts, revision activity, where-used) would provide immediate value.
|
||||
|
||||
---
|
||||
|
||||
### 9. File Handling
|
||||
|
||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||
|---------|---------------|-------------|----------|------------|
|
||||
| File versioning | Automatic | Full (MinIO) | - | - |
|
||||
| File preview | Thumbnails, 3D preview | None | Medium | Complex |
|
||||
| File conversion | PDF, DXF generation | None | Medium | Complex |
|
||||
| Replication | Multi-site sync | None | Low | Complex |
|
||||
| File copy with refs | Copy tree with references | None | Medium | Moderate |
|
||||
|
||||
**Gap Analysis:**
|
||||
File storage works well. Thumbnail generation and file preview would significantly improve the web UI experience. Automatic conversion to PDF/DXF is valuable for sharing with non-CAD users.
|
||||
|
||||
---
|
||||
|
||||
### Gap Summary by Priority
|
||||
|
||||
#### Completed (Previously Critical/High)
|
||||
1. ~~**User authentication**~~ - Implemented: local, LDAP, OIDC
|
||||
2. ~~**Role-based permissions**~~ - Implemented: 3-tier role model (admin/editor/viewer)
|
||||
3. ~~**Audit trail**~~ - Implemented: audit_log table with completeness scoring
|
||||
4. ~~**Where-used search**~~ - Implemented: reverse parent lookup API
|
||||
5. ~~**Multi-level BOM API**~~ - Implemented: recursive expansion with configurable depth
|
||||
6. ~~**BOM export**~~ - Implemented: CSV and ODS formats
|
||||
|
||||
#### Critical Gaps (Required for Team Use)
|
||||
1. **Workflow engine** - State machines with transitions and approvals
|
||||
2. **Check-out locking** - Pessimistic locking for CAD files
|
||||
|
||||
#### High Priority Gaps (Significant Value)
|
||||
1. **Email notifications** - Alert users on state changes
|
||||
2. **Web UI search** - Advanced search interface with saved searches
|
||||
3. **Folder/state permissions** - Granular access control beyond role model
|
||||
|
||||
#### Medium Priority Gaps (Nice to Have)
|
||||
1. **Saved searches** - Frequently used queries
|
||||
2. **File preview/thumbnails** - Visual browsing
|
||||
3. **Reporting** - Activity and inventory reports
|
||||
4. **Scheduled tasks** - Background automation
|
||||
5. **BOM comparison** - Revision diff for assemblies
|
||||
|
||||
---
|
||||
|
||||
## Feature Roadmap
|
||||
|
||||
### Phase 1: Foundation (Current - Q2 2026)
|
||||
*Complete MVP and stabilize core functionality*
|
||||
|
||||
| Feature | Description | Status |
|
||||
|---------|-------------|--------|
|
||||
| MinIO integration | File upload/download with versioning and checksums | Complete |
|
||||
| Revision control | Rollback, comparison, status/labels | Complete |
|
||||
| CSV import/export | Dry-run validation, template generation | Complete |
|
||||
| ODS import/export | Items, BOMs, project sheets, templates | Complete |
|
||||
| Project management | CRUD, many-to-many item tagging | Complete |
|
||||
| Multi-level BOM | Recursive expansion, where-used, export | Complete |
|
||||
| Authentication | Local, LDAP, OIDC with role-based access | Complete |
|
||||
| Audit logging | Action logging, completeness scoring | Complete |
|
||||
| Unit tests | Core API and database operations | Not Started |
|
||||
| Date segment type | Support date-based part number segments | Not Started |
|
||||
| Part number validation | Validate format on creation | Not Started |
|
||||
| Location CRUD API | Expose location hierarchy via REST | Not Started |
|
||||
| Inventory API | Expose inventory operations via REST | Not Started |
|
||||
|
||||
### Phase 2: Multi-User (Q2-Q3 2026)
|
||||
*Enable team collaboration*
|
||||
|
||||
| Feature | Description | Status |
|
||||
|---------|-------------|--------|
|
||||
| LDAP authentication | Integrate with FreeIPA/Active Directory | **Complete** |
|
||||
| OIDC authentication | Keycloak / OpenID Connect | **Complete** |
|
||||
| Audit logging | Record all user actions with timestamps | **Complete** |
|
||||
| Session management | Token-based and session-based API authentication | **Complete** |
|
||||
| User/group management | Create, assign, manage users and groups | Not Started |
|
||||
| Folder permissions | Read/write/delete per folder hierarchy | Not Started |
|
||||
| Check-out locking | Pessimistic locks with timeout | Not Started |
|
||||
|
||||
### Phase 3: Workflow Engine (Q3-Q4 2026)
|
||||
*Implement engineering change processes*
|
||||
|
||||
| Feature | Description | Complexity |
|
||||
|---------|-------------|------------|
|
||||
| Workflow designer | YAML-defined state machines | Complex |
|
||||
| State transitions | Configurable transition rules | Complex |
|
||||
| Transition permissions | Who can execute which transitions | Moderate |
|
||||
| Single approvals | Basic approval workflow | Moderate |
|
||||
| Parallel approvals | Multi-approver gates | Complex |
|
||||
| Automatic transitions | Timer and condition-based | Complex |
|
||||
| Email notifications | SMTP integration for alerts | Simple |
|
||||
| Child state conditions | Block parent transitions | Moderate |
|
||||
|
||||
### Phase 4: Search & Discovery (Q4 2026 - Q1 2027)
|
||||
*Improve findability and navigation*
|
||||
|
||||
| Feature | Description | Status |
|
||||
|---------|-------------|--------|
|
||||
| Where-used queries | Find parent assemblies | **Complete** |
|
||||
| Fuzzy search | Quick search across items | **Complete** |
|
||||
| Advanced search UI | Web interface with filters | Not Started |
|
||||
| Search operators | AND, OR, NOT, wildcards | Not Started |
|
||||
| Saved searches | User favorites | Not Started |
|
||||
| Content search | Search within file content | Not Started |
|
||||
|
||||
### Phase 5: BOM & Reporting (Q1-Q2 2027)
|
||||
*Enhanced BOM management and analytics*
|
||||
|
||||
| Feature | Description | Status |
|
||||
|---------|-------------|--------|
|
||||
| Multi-level BOM API | Recursive assembly retrieval | **Complete** |
|
||||
| BOM export | CSV and ODS formats | **Complete** |
|
||||
| BOM import | CSV with upsert and cycle detection | **Complete** |
|
||||
| BOM comparison | Diff between revisions | Not Started |
|
||||
| Standard reports | Activity, inventory, usage | Not Started |
|
||||
| Custom queries | User-defined report builder | Not Started |
|
||||
| Dashboard | Visual KPIs and metrics | Not Started |
|
||||
|
||||
### Phase 6: Advanced Features (Q2-Q4 2027)
|
||||
*Enterprise capabilities*
|
||||
|
||||
| Feature | Description | Complexity |
|
||||
|---------|-------------|------------|
|
||||
| File preview | Thumbnail generation | Complex |
|
||||
| File conversion | Auto-generate PDF/DXF | Complex |
|
||||
| ERP integration | Adapter framework | Complex |
|
||||
| Task scheduler | Background job processing | Moderate |
|
||||
| Webhooks | Event notifications to external systems | Moderate |
|
||||
| API rate limiting | Protect against abuse | Simple |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1 Detailed Tasks
|
||||
|
||||
#### 1.1 MinIO Integration -- COMPLETE
|
||||
- [x] MinIO service configured in Docker Compose
|
||||
- [x] File upload via REST API
|
||||
- [x] File download via REST API (latest and by revision)
|
||||
- [x] SHA256 checksums on upload
|
||||
|
||||
#### 1.2 Authentication & Authorization -- COMPLETE
|
||||
- [x] Local authentication (bcrypt)
|
||||
- [x] LDAP/FreeIPA authentication
|
||||
- [x] OIDC/Keycloak authentication
|
||||
- [x] Role-based access control (admin/editor/viewer)
|
||||
- [x] API token management (SHA-256 hashed)
|
||||
- [x] Session management (PostgreSQL-backed)
|
||||
- [x] CSRF protection (nosurf)
|
||||
- [x] Audit logging (database table)
|
||||
|
||||
#### 1.3 Multi-level BOM & Export -- COMPLETE
|
||||
- [x] Recursive BOM expansion with configurable depth
|
||||
- [x] Where-used reverse lookup
|
||||
- [x] BOM CSV export/import with cycle detection
|
||||
- [x] BOM ODS export
|
||||
- [x] ODS item export/import/template
|
||||
|
||||
#### 1.4 Unit Test Suite
|
||||
- [ ] Database connection and transaction tests
|
||||
- [ ] Item CRUD operation tests
|
||||
- [ ] Revision creation and retrieval tests
|
||||
- [ ] Part number generation tests
|
||||
- [ ] File upload/download tests
|
||||
- [ ] CSV import/export tests
|
||||
- [ ] API endpoint tests
|
||||
|
||||
#### 1.5 Missing Segment Types
|
||||
- [ ] Implement date segment type
|
||||
- [ ] Add strftime-style format support
|
||||
|
||||
#### 1.6 Location & Inventory APIs
|
||||
- [ ] `GET /api/locations` - List locations
|
||||
- [ ] `POST /api/locations` - Create location
|
||||
- [ ] `GET /api/locations/{path}` - Get location
|
||||
- [ ] `DELETE /api/locations/{path}` - Delete location
|
||||
- [ ] `GET /api/inventory/{partNumber}` - Get inventory
|
||||
- [ ] `POST /api/inventory/{partNumber}/adjust` - Adjust quantity
|
||||
- [ ] `POST /api/inventory/{partNumber}/move` - Move between locations
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Phase 1 (Foundation)
|
||||
- All existing tests pass
|
||||
- File upload/download works end-to-end
|
||||
- FreeCAD users can checkout, modify, commit parts
|
||||
|
||||
### Phase 2 (Multi-User)
|
||||
- 5+ concurrent users supported
|
||||
- No data corruption under concurrent access
|
||||
- Audit log captures all modifications
|
||||
|
||||
### Phase 3 (Workflow)
|
||||
- Engineering change process completable in Silo
|
||||
- Email notifications delivered reliably
|
||||
- Workflow state visible in web UI
|
||||
|
||||
### Phase 4+ (Advanced)
|
||||
- Search returns results in <2 seconds
|
||||
- Where-used queries complete in <5 seconds
|
||||
- BOM export matches assembly structure
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
### SOLIDWORKS PDM Documentation
|
||||
- [SOLIDWORKS PDM Product Page](https://www.solidworks.com/product/solidworks-pdm)
|
||||
- [What's New in SOLIDWORKS PDM 2025](https://blogs.solidworks.com/solidworksblog/2024/10/whats-new-in-solidworks-pdm-2025.html)
|
||||
- [Top 5 Enhancements in SOLIDWORKS PDM 2024](https://blogs.solidworks.com/solidworksblog/2023/10/top-5-enhancements-in-solidworks-pdm-2024.html)
|
||||
- [SOLIDWORKS PDM Workflow Transitions](https://help.solidworks.com/2023/english/EnterprisePDM/Admin/c_workflow_transition.htm)
|
||||
- [Ultimate Guide to SOLIDWORKS PDM Permissions](https://www.goengineer.com/blog/ultimate-guide-to-solidworks-pdm-permissions)
|
||||
- [Searching in SOLIDWORKS PDM](https://help.solidworks.com/2021/english/EnterprisePDM/fileexplorer/c_searches.htm)
|
||||
- [SOLIDWORKS PDM API Getting Started](https://3dswym.3dexperience.3ds.com/wiki/solidworks-news-info/getting-started-with-the-solidworks-pdm-api-solidpractices_gBCYaM75RgORBcpSO1m_Mw)
|
||||
|
||||
### Silo Documentation
|
||||
- [Specification](docs/SPECIFICATION.md)
|
||||
- [Development Status](docs/STATUS.md)
|
||||
- [Deployment Guide](docs/DEPLOYMENT.md)
|
||||
- [Gap Analysis](docs/GAP_ANALYSIS.md)
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Feature Comparison Matrix
|
||||
|
||||
| Category | Feature | SW PDM Standard | SW PDM Pro | Silo Current | Silo Planned |
|
||||
|----------|---------|-----------------|------------|--------------|--------------|
|
||||
| **Version Control** | Check-in/out | Yes | Yes | No | Phase 2 |
|
||||
| | Version history | Yes | Yes | Yes | - |
|
||||
| | Rollback | Yes | Yes | Yes | - |
|
||||
| | Revision labels/status | Yes | Yes | Yes | - |
|
||||
| | Revision comparison | Yes | Yes | Yes (metadata) | - |
|
||||
| **Workflow** | Custom workflows | Limited | Yes | No | Phase 3 |
|
||||
| | Parallel approval | No | Yes | No | Phase 3 |
|
||||
| | Notifications | No | Yes | No | Phase 3 |
|
||||
| **Security** | User auth | Windows | Windows/LDAP | Yes (local, LDAP, OIDC) | - |
|
||||
| | Permissions | Basic | Granular | Partial (role-based) | Phase 2 |
|
||||
| | Audit trail | Basic | Full | Yes | - |
|
||||
| **Search** | Metadata search | Yes | Yes | Partial (API + fuzzy) | Phase 4 |
|
||||
| | Content search | No | Yes | No | Phase 4 |
|
||||
| | Where-used | Yes | Yes | Yes | - |
|
||||
| **BOM** | Single-level | Yes | Yes | Yes | - |
|
||||
| | Multi-level | Yes | Yes | Yes (recursive) | - |
|
||||
| | BOM export | Yes | Yes | Yes (CSV, ODS) | - |
|
||||
| **Data** | CSV import/export | Yes | Yes | Yes | - |
|
||||
| | ODS import/export | No | No | Yes | - |
|
||||
| | Project management | Yes | Yes | Yes | - |
|
||||
| **Integration** | API | Limited | Full | Full REST (75) | - |
|
||||
| | ERP connectors | No | Yes | Partial (Odoo stubs) | Phase 6 |
|
||||
| | Web access | No | Yes | Yes (React SPA + auth) | - |
|
||||
| **Files** | Versioning | Yes | Yes | Yes | - |
|
||||
| | Preview | Yes | Yes | No | Phase 6 |
|
||||
| | Multi-site | No | Yes | No | Not Planned |
|
||||
943
docs/src/silo-server/SPECIFICATION.md
Normal file
943
docs/src/silo-server/SPECIFICATION.md
Normal file
@@ -0,0 +1,943 @@
|
||||
# Silo: Item Database and Part Management System
|
||||
|
||||
**Version:** 0.2
|
||||
**Date:** February 2026
|
||||
**Author:** Kindred Systems LLC
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Silo is an item database with configurable part number generation, designed for R&D-oriented workflows. It provides revision tracking, BOM management, file versioning, and physical inventory location management through a REST API and web UI. CAD and workflow integration (FreeCAD workbench, LibreOffice Calc extension) is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)).
|
||||
|
||||
### 1.1 Core Philosophy
|
||||
|
||||
Silo treats **part numbering schemas as configuration, not code**. Multiple numbering schemes can coexist, each defined in YAML. The system is schema-agnostic—it doesn't impose a particular part numbering philosophy (intelligent vs. non-intelligent numbers) but instead provides the machinery to implement whatever scheme the organization requires.
|
||||
|
||||
### 1.2 Key Principles
|
||||
|
||||
- **Items are the atomic unit**: Everything is an item (parts, assemblies, drawings, documents)
|
||||
- **Schemas are mutable**: Part numbering schemas can evolve, though migration tooling is out of scope for MVP
|
||||
- **Append-only history**: All parameter changes are recorded; item state is reconstructable at any point in time
|
||||
- **Configuration over convention**: Hierarchies, relationships, and behaviors are YAML-defined
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture
|
||||
|
||||
### 2.1 Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ CAD Clients (silo-mod, silo-calc) │
|
||||
│ FreeCAD Workbench · LibreOffice Calc Extension │
|
||||
│ (maintained in separate repositories) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│ REST API
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Silo Server (silod) │
|
||||
│ - REST API (75 endpoints) │
|
||||
│ - Authentication (local, LDAP, OIDC) │
|
||||
│ - Schema parsing and validation │
|
||||
│ - Part number generation engine │
|
||||
│ - Revision management │
|
||||
│ - Relationship graph / BOM │
|
||||
│ - Web UI (React SPA) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌───────────────┴───────────────┐
|
||||
▼ ▼
|
||||
┌─────────────────────────┐ ┌─────────────────────────────┐
|
||||
│ PostgreSQL │ │ MinIO │
|
||||
│ (psql.kindred.internal)│ │ - File storage │
|
||||
│ - Item metadata │ │ - Versioned objects │
|
||||
│ - Relationships │ │ - Thumbnails │
|
||||
│ - Revision history │ │ │
|
||||
│ - Auth / Sessions │ │ │
|
||||
│ - Audit log │ │ │
|
||||
└─────────────────────────┘ └─────────────────────────────┘
|
||||
```
|
||||
|
||||
### 2.2 Technology Stack
|
||||
|
||||
| Component | Technology | Notes |
|
||||
|-----------|------------|-------|
|
||||
| Database | PostgreSQL 16 | Existing instance at psql.kindred.internal |
|
||||
| File Storage | MinIO | S3-compatible, versioning enabled |
|
||||
| CLI & API Server | Go (1.24) | chi/v5 router, pgx/v5 driver, zerolog |
|
||||
| Authentication | Multi-backend | Local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak |
|
||||
| Sessions | PostgreSQL pgxstore | alexedwards/scs, 24h lifetime |
|
||||
| Web UI | React 19, Vite 6, TypeScript 5.7 | Catppuccin Mocha theme, inline styles |
|
||||
|
||||
---
|
||||
|
||||
## 3. Data Model
|
||||
|
||||
### 3.1 Items
|
||||
|
||||
An **item** is the fundamental entity. Items have:
|
||||
|
||||
- A **part number** (generated according to a schema)
|
||||
- A **type** (part, assembly, drawing, document, etc.)
|
||||
- **Properties** (key-value pairs, schema-defined and custom)
|
||||
- **Relationships** to other items
|
||||
- **Revisions** (append-only history)
|
||||
- **Files** (optional, stored in MinIO)
|
||||
- **Location** (optional physical inventory location)
|
||||
|
||||
### 3.2 Database Schema (Conceptual)
|
||||
|
||||
```sql
|
||||
-- Part numbering schemas (YAML stored as text, parsed at runtime)
|
||||
CREATE TABLE schemas (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
name TEXT UNIQUE NOT NULL,
|
||||
version INTEGER NOT NULL DEFAULT 1,
|
||||
definition JSONB NOT NULL, -- parsed YAML
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ DEFAULT now()
|
||||
);
|
||||
|
||||
-- Items (core entity)
|
||||
CREATE TABLE items (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
part_number TEXT UNIQUE NOT NULL,
|
||||
schema_id UUID REFERENCES schemas(id),
|
||||
item_type TEXT NOT NULL, -- 'part', 'assembly', 'drawing', etc.
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
current_revision_id UUID -- points to latest revision
|
||||
);
|
||||
|
||||
-- Append-only revision history
|
||||
CREATE TABLE revisions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
item_id UUID REFERENCES items(id) NOT NULL,
|
||||
revision_number INTEGER NOT NULL,
|
||||
properties JSONB NOT NULL, -- all properties at this revision
|
||||
file_version TEXT, -- MinIO version ID if applicable
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
created_by TEXT, -- user identifier (future: LDAP DN)
|
||||
comment TEXT,
|
||||
UNIQUE(item_id, revision_number)
|
||||
);
|
||||
|
||||
-- Item relationships (BOM structure)
|
||||
CREATE TABLE relationships (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
parent_item_id UUID REFERENCES items(id) NOT NULL,
|
||||
child_item_id UUID REFERENCES items(id) NOT NULL,
|
||||
relationship_type TEXT NOT NULL, -- 'component', 'alternate', 'reference'
|
||||
quantity DECIMAL,
|
||||
reference_designator TEXT, -- e.g., "R1", "C3" for electronics
|
||||
metadata JSONB, -- assembly-specific relationship config
|
||||
revision_id UUID REFERENCES revisions(id), -- which revision this applies to
|
||||
created_at TIMESTAMPTZ DEFAULT now()
|
||||
);
|
||||
|
||||
-- Location hierarchy (configurable via YAML)
|
||||
CREATE TABLE locations (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
path TEXT UNIQUE NOT NULL, -- e.g., "lab/shelf-a/bin-3"
|
||||
name TEXT NOT NULL,
|
||||
parent_id UUID REFERENCES locations(id),
|
||||
location_type TEXT NOT NULL, -- defined in location schema
|
||||
metadata JSONB,
|
||||
created_at TIMESTAMPTZ DEFAULT now()
|
||||
);
|
||||
|
||||
-- Item inventory (quantity at location)
|
||||
CREATE TABLE inventory (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
item_id UUID REFERENCES items(id) NOT NULL,
|
||||
location_id UUID REFERENCES locations(id) NOT NULL,
|
||||
quantity DECIMAL NOT NULL DEFAULT 0,
|
||||
updated_at TIMESTAMPTZ DEFAULT now(),
|
||||
UNIQUE(item_id, location_id)
|
||||
);
|
||||
|
||||
-- Sequence counters for part number generation
|
||||
CREATE TABLE sequences (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
schema_id UUID REFERENCES schemas(id),
|
||||
scope TEXT NOT NULL, -- scope key (e.g., project code, type code)
|
||||
current_value INTEGER NOT NULL DEFAULT 0,
|
||||
UNIQUE(schema_id, scope)
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. YAML Configuration System
|
||||
|
||||
### 4.1 Part Numbering Schema
|
||||
|
||||
Schemas define how part numbers are generated. Each schema consists of **segments** that are concatenated with a **separator**.
|
||||
|
||||
```yaml
|
||||
# /etc/silo/schemas/kindred-rd.yaml
|
||||
schema:
|
||||
name: kindred-rd
|
||||
version: 3
|
||||
description: "Kindred Systems R&D part numbering"
|
||||
|
||||
# Separator between segments (default: "-")
|
||||
separator: "-"
|
||||
|
||||
# Uniqueness enforcement
|
||||
uniqueness:
|
||||
scope: global
|
||||
case_sensitive: false
|
||||
|
||||
segments:
|
||||
- name: category
|
||||
type: enum
|
||||
description: "Category code (2-3 characters)"
|
||||
required: true
|
||||
values:
|
||||
F01: "Hex Cap Screw"
|
||||
F02: "Socket Head Cap Screw"
|
||||
# ... 70+ categories across:
|
||||
# F01-F18: Fasteners
|
||||
# C01-C17: Fluid Fittings
|
||||
# R01-R44: Motion Components
|
||||
# S01-S17: Structural Materials
|
||||
# E01-E27: Electrical Components
|
||||
# M01-M18: Mechanical Components
|
||||
# T01-T08: Tooling and Fixtures
|
||||
# A01-A07: Assemblies
|
||||
# P01-P05: Purchased/Off-the-Shelf
|
||||
# X01-X08: Custom Fabricated Parts
|
||||
|
||||
- name: sequence
|
||||
type: serial
|
||||
length: 4
|
||||
padding: "0"
|
||||
start: 1
|
||||
description: "Sequential number within category"
|
||||
scope: "{category}"
|
||||
|
||||
format: "{category}-{sequence}"
|
||||
|
||||
# Example outputs:
|
||||
# F01-0001 (first hex cap screw)
|
||||
# R27-0001 (first linear rail)
|
||||
# A01-0001 (first assembly)
|
||||
```
|
||||
|
||||
> **Note:** The schema was migrated from a `{project}-{type}-{sequence}` format (v1) to `{category}-{sequence}` (v3). Projects are now managed as many-to-many tags on items rather than embedded in the part number. See `migrations/006_project_tags.sql`.
|
||||
|
||||
### 4.2 Segment Types
|
||||
|
||||
| Type | Description | Options |
|
||||
|------|-------------|---------|
|
||||
| `string` | Fixed or variable length string | `length`, `min_length`, `max_length`, `pattern`, `case` |
|
||||
| `enum` | Predefined set of values | `values` (map of code → description) |
|
||||
| `serial` | Auto-incrementing integer | `length`, `padding`, `start`, `scope` |
|
||||
| `date` | Date-based segment | `format` (strftime-style) |
|
||||
| `constant` | Fixed value | `value` |
|
||||
|
||||
### 4.3 Serial Scope Templates
|
||||
|
||||
The `scope` field in serial segments supports template variables referencing other segments:
|
||||
|
||||
```yaml
|
||||
# Sequence per category (current kindred-rd schema)
|
||||
scope: "{category}"
|
||||
|
||||
# Global sequence (no scope)
|
||||
scope: null
|
||||
```
|
||||
|
||||
### 4.4 Alternative Schema Example (Simple Sequential)
|
||||
|
||||
```yaml
|
||||
# /etc/silo/schemas/simple.yaml
|
||||
schema:
|
||||
name: simple
|
||||
version: 1
|
||||
description: "Simple non-intelligent numbering"
|
||||
|
||||
segments:
|
||||
- name: prefix
|
||||
type: constant
|
||||
value: "P"
|
||||
|
||||
- name: sequence
|
||||
type: serial
|
||||
length: 6
|
||||
padding: "0"
|
||||
scope: null # global counter
|
||||
|
||||
format: "{prefix}{sequence}"
|
||||
separator: ""
|
||||
|
||||
# Output: P000001, P000002, ...
|
||||
```
|
||||
|
||||
### 4.5 Location Hierarchy Schema
|
||||
|
||||
```yaml
|
||||
# /etc/silo/schemas/locations.yaml
|
||||
location_schema:
|
||||
name: kindred-lab
|
||||
version: 1
|
||||
|
||||
hierarchy:
|
||||
- level: 0
|
||||
type: facility
|
||||
name_pattern: "^[a-z-]+$"
|
||||
|
||||
- level: 1
|
||||
type: area
|
||||
name_pattern: "^[a-z-]+$"
|
||||
|
||||
- level: 2
|
||||
type: shelf
|
||||
name_pattern: "^shelf-[a-z]$"
|
||||
|
||||
- level: 3
|
||||
type: bin
|
||||
name_pattern: "^bin-[0-9]+$"
|
||||
|
||||
# Path format
|
||||
path_separator: "/"
|
||||
|
||||
# Example paths:
|
||||
# lab/main-area/shelf-a/bin-1
|
||||
# lab/storage/shelf-b/bin-12
|
||||
```
|
||||
|
||||
### 4.6 Assembly Metadata Schema
|
||||
|
||||
Each assembly can define its own relationship tracking behavior:
|
||||
|
||||
```yaml
|
||||
# Stored in item properties or as a linked document
|
||||
assembly_config:
|
||||
# What relationship types this assembly uses
|
||||
relationship_types:
|
||||
- component # standard BOM entry
|
||||
- alternate # interchangeable substitute
|
||||
- reference # related but not part of BOM
|
||||
|
||||
# Whether to track reference designators
|
||||
use_reference_designators: true
|
||||
designator_format: "^[A-Z]+[0-9]+$" # e.g., R1, C3, U12
|
||||
|
||||
# Revision linking behavior
|
||||
child_revision_tracking: specific # or "latest"
|
||||
|
||||
# Custom properties for relationships
|
||||
relationship_properties:
|
||||
- name: mounting_orientation
|
||||
type: enum
|
||||
values: [top, bottom, left, right, front, back]
|
||||
- name: notes
|
||||
type: text
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Client Integration
|
||||
|
||||
CAD workbench and spreadsheet extension implementations are maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The Silo server provides the REST API endpoints consumed by those clients.
|
||||
|
||||
### 5.1 File Storage Strategy
|
||||
|
||||
Files are stored as whole objects in MinIO with versioning enabled. Storage path convention: `items/{partNumber}/rev{N}.ext`. SHA-256 checksums are captured on upload for integrity verification.
|
||||
|
||||
Future option: exploded storage (unpack ZIP-based CAD archives for better diffing).
|
||||
|
||||
### 5.2 Checkout Locking (Future)
|
||||
|
||||
Future multi-user support will need a server-side locking strategy:
|
||||
|
||||
- **Pessimistic locking**: Checkout acquires exclusive lock
|
||||
- **Optimistic locking**: Allow concurrent edits, handle conflicts on commit
|
||||
|
||||
Recommendation: Pessimistic locking for CAD files (merge is impractical).
|
||||
|
||||
---
|
||||
|
||||
## 6. Web Interface
|
||||
|
||||
### 6.1 Architecture
|
||||
|
||||
The web UI is a React single-page application served at `/` by the Go server. The SPA communicates with the backend exclusively via the JSON REST API at `/api/*`.
|
||||
|
||||
- **Stack**: React 19, React Router 7, Vite 6, TypeScript 5.7
|
||||
- **Theme**: Catppuccin Mocha (dark) via CSS custom properties
|
||||
- **Styling**: Inline `React.CSSProperties` — no CSS modules, no Tailwind
|
||||
- **State**: Local `useState` + custom hooks (no Redux/Zustand)
|
||||
- **SPA serving**: `web/dist/` served by Go's `NotFound` handler with `index.html` fallback for client-side routing
|
||||
|
||||
### 6.2 Pages
|
||||
|
||||
| Page | Route | Description |
|
||||
|------|-------|-------------|
|
||||
| Items | `/` | Master-detail layout with resizable split panel, sortable table, 5-tab detail view (Main, Properties, Revisions, BOM, Where Used), in-pane CRUD forms |
|
||||
| Projects | `/projects` | Project CRUD with sortable table, in-pane forms |
|
||||
| Schemas | `/schemas` | Schema browser with collapsible segments, enum value CRUD |
|
||||
| Settings | `/settings` | Account info, API token management |
|
||||
| Audit | `/audit` | Item completeness scoring |
|
||||
| Login | `/login` | Username/password form, conditional OIDC button |
|
||||
|
||||
### 6.3 Design Patterns
|
||||
|
||||
- **In-pane forms**: Create/Edit/Delete forms render in the detail pane area (Infor ERP-style), not as modal overlays
|
||||
- **Permission checks**: Write actions conditionally rendered based on user role
|
||||
- **Fuzzy search**: Debounced search with scope toggle (All/PN/Description)
|
||||
- **Persistence**: `useLocalStorage` hook for user preferences (layout mode, column visibility)
|
||||
|
||||
### 6.4 URI Handler
|
||||
|
||||
Register `silo://` protocol handler:
|
||||
|
||||
```
|
||||
silo://open/PROTO-AS-0001 # Open latest revision
|
||||
silo://open/PROTO-AS-0001?rev=3 # Open specific revision
|
||||
```
|
||||
|
||||
See [frontend-spec.md](../frontend-spec.md) for full component specifications.
|
||||
|
||||
---
|
||||
|
||||
## 7. Revision Tracking
|
||||
|
||||
### 7.1 Append-Only Model
|
||||
|
||||
Every property change creates a new revision record. The current state is always the latest revision, but any historical state can be reconstructed.
|
||||
|
||||
```
|
||||
Item: PROTO-AS-0001
|
||||
|
||||
Revision 1 (2026-01-15): Initial creation
|
||||
- description: "Main chassis assembly"
|
||||
- material: null
|
||||
- weight: null
|
||||
|
||||
Revision 2 (2026-01-20): Updated properties
|
||||
- description: "Main chassis assembly"
|
||||
- material: "6061-T6 Aluminum"
|
||||
- weight: 2.5
|
||||
|
||||
Revision 3 (2026-02-01): Design change
|
||||
- description: "Main chassis assembly v2"
|
||||
- material: "6061-T6 Aluminum"
|
||||
- weight: 2.3
|
||||
```
|
||||
|
||||
### 7.2 Revision Creation
|
||||
|
||||
Revisions are created explicitly by user action (not automatic):
|
||||
|
||||
- `silo commit` from FreeCAD
|
||||
- "Save Revision" button in web UI
|
||||
- API call with explicit revision flag
|
||||
|
||||
### 7.3 Revision vs. File Version
|
||||
|
||||
- **Revision**: Silo metadata revision (tracked in PostgreSQL)
|
||||
- **File Version**: MinIO object version (automatic on upload)
|
||||
|
||||
A single Silo revision may span multiple file uploads during editing. Only committed revisions create formal revision records.
|
||||
|
||||
---
|
||||
|
||||
## 8. Relationships and BOM
|
||||
|
||||
### 8.1 Relationship Types
|
||||
|
||||
| Type | Description | Use Case |
|
||||
|------|-------------|----------|
|
||||
| `component` | Part is used in assembly | Standard BOM entry |
|
||||
| `alternate` | Interchangeable substitute | Alternative sourcing |
|
||||
| `reference` | Related item, not in BOM | Drawings, specs, tools |
|
||||
|
||||
### 8.2 Reference Designators
|
||||
|
||||
For assemblies that require them (electronics, complex mechanisms):
|
||||
|
||||
```yaml
|
||||
# Relationship record
|
||||
parent: PROTO-AS-0001
|
||||
child: PROTO-PT-0042
|
||||
type: component
|
||||
quantity: 4
|
||||
reference_designators: ["R1", "R2", "R3", "R4"]
|
||||
```
|
||||
|
||||
### 8.3 Revision-Specific Relationships
|
||||
|
||||
Relationships can link to specific child revisions or track latest:
|
||||
|
||||
```yaml
|
||||
# Locked to specific revision
|
||||
child: PROTO-PT-0042
|
||||
child_revision: 3
|
||||
|
||||
# Always use latest (default for R&D)
|
||||
child: PROTO-PT-0042
|
||||
child_revision: null # means "latest"
|
||||
```
|
||||
|
||||
Assembly metadata YAML controls default behavior per assembly.
|
||||
|
||||
### 8.4 Flat BOM and Assembly Costing
|
||||
|
||||
Two endpoints provide procurement- and manufacturing-oriented views of the BOM:
|
||||
|
||||
**Flat BOM** (`GET /api/items/{partNumber}/bom/flat`) walks the full assembly tree and returns a consolidated list of **leaf parts only** (parts with no BOM children). Quantities are multiplied through each nesting level and duplicate parts are summed.
|
||||
|
||||
```
|
||||
Assembly A (qty 1)
|
||||
├── Sub-assembly B (qty 2)
|
||||
│ ├── Part X (qty 3) → total 6
|
||||
│ └── Part Y (qty 1) → total 2
|
||||
└── Part Z (qty 4) → total 4
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"part_number": "A",
|
||||
"flat_bom": [
|
||||
{ "part_number": "X", "description": "...", "total_quantity": 6 },
|
||||
{ "part_number": "Y", "description": "...", "total_quantity": 2 },
|
||||
{ "part_number": "Z", "description": "...", "total_quantity": 4 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Assembly Cost** (`GET /api/items/{partNumber}/bom/cost`) builds on the flat BOM and multiplies each leaf's `total_quantity` by its `standard_cost` to produce per-line extended costs and a total assembly cost.
|
||||
|
||||
```json
|
||||
{
|
||||
"part_number": "A",
|
||||
"total_cost": 124.50,
|
||||
"cost_breakdown": [
|
||||
{ "part_number": "X", "total_quantity": 6, "unit_cost": 10.00, "extended_cost": 60.00 },
|
||||
{ "part_number": "Y", "total_quantity": 2, "unit_cost": 7.25, "extended_cost": 14.50 },
|
||||
{ "part_number": "Z", "total_quantity": 4, "unit_cost": 12.50, "extended_cost": 50.00 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Both endpoints detect BOM cycles and return **HTTP 409** with the offending path:
|
||||
|
||||
```json
|
||||
{ "error": "cycle_detected", "detail": "BOM cycle detected: A → B → A" }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Physical Inventory
|
||||
|
||||
### 9.1 Location Management
|
||||
|
||||
Locations are hierarchical, defined by YAML schema. Each item can exist at multiple locations with quantities.
|
||||
|
||||
```
|
||||
Location: lab/main-area/shelf-a/bin-3
|
||||
- PROTO-PT-0001: 15 units
|
||||
- PROTO-PT-0002: 8 units
|
||||
|
||||
Location: lab/storage/shelf-b/bin-1
|
||||
- PROTO-PT-0001: 50 units (spare stock)
|
||||
```
|
||||
|
||||
### 9.2 Inventory Operations
|
||||
|
||||
- **Add**: Increase quantity at location
|
||||
- **Remove**: Decrease quantity at location
|
||||
- **Move**: Transfer between locations
|
||||
- **Adjust**: Set absolute quantity (for cycle counts)
|
||||
|
||||
All operations logged for audit trail (future consideration).
|
||||
|
||||
---
|
||||
|
||||
## 10. Authentication
|
||||
|
||||
Silo supports three authentication backends that can be enabled independently or combined. When authentication is disabled (`auth.enabled: false`), all routes are open and a synthetic dev user with the `admin` role is injected into every request.
|
||||
|
||||
### 10.1 Backends
|
||||
|
||||
| Backend | Use Case | Config Key |
|
||||
|---------|----------|------------|
|
||||
| **Local** | Username/password stored in database (bcrypt cost 12) | `auth.local` |
|
||||
| **LDAP** | FreeIPA / Active Directory via LDAP bind | `auth.ldap` |
|
||||
| **OIDC** | Keycloak or any OpenID Connect provider (redirect flow) | `auth.oidc` |
|
||||
|
||||
### 10.2 Role Model
|
||||
|
||||
Three roles with a strict hierarchy: `admin > editor > viewer`
|
||||
|
||||
| Permission | viewer | editor | admin |
|
||||
|-----------|--------|--------|-------|
|
||||
| Read items, projects, schemas, BOMs | Yes | Yes | Yes |
|
||||
| Create/update items and revisions | No | Yes | Yes |
|
||||
| Upload files, manage BOMs | No | Yes | Yes |
|
||||
| Import CSV/ODS | No | Yes | Yes |
|
||||
| Manage own API tokens | Yes | Yes | Yes |
|
||||
| User management (future) | No | No | Yes |
|
||||
|
||||
### 10.3 API Tokens
|
||||
|
||||
Raw token format: `silo_` + 64 hex characters (32 random bytes from `crypto/rand`). Only the SHA-256 hash is stored in the database. Tokens inherit the owning user's role.
|
||||
|
||||
### 10.4 Sessions
|
||||
|
||||
PostgreSQL-backed sessions via `alexedwards/scs` pgxstore. Cookie: `silo_session`, HttpOnly, SameSite=Lax, 24h lifetime. `Secure` flag is set when `auth.enabled` is true.
|
||||
|
||||
See [AUTH.md](AUTH.md) for full architecture details and [AUTH_USER_GUIDE.md](AUTH_USER_GUIDE.md) for setup instructions.
|
||||
|
||||
---
|
||||
|
||||
## 11. API Design
|
||||
|
||||
### 11.1 REST Endpoints (75 Implemented)
|
||||
|
||||
```
|
||||
# Health (no auth)
|
||||
GET /health # Basic health check
|
||||
GET /ready # Readiness (DB + MinIO)
|
||||
|
||||
# Auth (no auth required)
|
||||
GET /login # Login page
|
||||
POST /login # Login form handler
|
||||
POST /logout # Logout
|
||||
GET /auth/oidc # OIDC login redirect
|
||||
GET /auth/callback # OIDC callback
|
||||
|
||||
# Public API (no auth required)
|
||||
GET /api/auth/config # Auth backend configuration (for login UI)
|
||||
|
||||
# Auth API (require auth)
|
||||
GET /api/auth/me # Current authenticated user
|
||||
GET /api/auth/tokens # List user's API tokens
|
||||
POST /api/auth/tokens # Create API token
|
||||
DELETE /api/auth/tokens/{id} # Revoke API token
|
||||
|
||||
# Presigned Uploads (editor)
|
||||
POST /api/uploads/presign # Get presigned MinIO upload URL [editor]
|
||||
|
||||
# Schemas (read: viewer, write: editor)
|
||||
GET /api/schemas # List all schemas
|
||||
GET /api/schemas/{name} # Get schema details
|
||||
GET /api/schemas/{name}/properties # Get property schema for category
|
||||
POST /api/schemas/{name}/segments/{segment}/values # Add enum value [editor]
|
||||
PUT /api/schemas/{name}/segments/{segment}/values/{code} # Update enum value [editor]
|
||||
DELETE /api/schemas/{name}/segments/{segment}/values/{code} # Delete enum value [editor]
|
||||
|
||||
# Projects (read: viewer, write: editor)
|
||||
GET /api/projects # List projects
|
||||
GET /api/projects/{code} # Get project
|
||||
GET /api/projects/{code}/items # Get project items
|
||||
GET /api/projects/{code}/sheet.ods # Export project sheet as ODS
|
||||
POST /api/projects # Create project [editor]
|
||||
PUT /api/projects/{code} # Update project [editor]
|
||||
DELETE /api/projects/{code} # Delete project [editor]
|
||||
|
||||
# Items (read: viewer, write: editor)
|
||||
GET /api/items # List/filter items
|
||||
GET /api/items/search # Fuzzy search
|
||||
GET /api/items/export.csv # Export items to CSV
|
||||
GET /api/items/template.csv # CSV import template
|
||||
GET /api/items/export.ods # Export items to ODS
|
||||
GET /api/items/template.ods # ODS import template
|
||||
POST /api/items # Create item [editor]
|
||||
POST /api/items/import # Import items from CSV [editor]
|
||||
POST /api/items/import.ods # Import items from ODS [editor]
|
||||
|
||||
# Item Detail
|
||||
GET /api/items/{partNumber} # Get item details
|
||||
PUT /api/items/{partNumber} # Update item [editor]
|
||||
DELETE /api/items/{partNumber} # Archive item [editor]
|
||||
|
||||
# Item-Project Tags
|
||||
GET /api/items/{partNumber}/projects # Get item's projects
|
||||
POST /api/items/{partNumber}/projects # Add project tags [editor]
|
||||
DELETE /api/items/{partNumber}/projects/{code} # Remove project tag [editor]
|
||||
|
||||
# Revisions
|
||||
GET /api/items/{partNumber}/revisions # List revisions
|
||||
GET /api/items/{partNumber}/revisions/compare # Compare two revisions
|
||||
GET /api/items/{partNumber}/revisions/{revision} # Get specific revision
|
||||
POST /api/items/{partNumber}/revisions # Create revision [editor]
|
||||
PATCH /api/items/{partNumber}/revisions/{revision} # Update status/labels [editor]
|
||||
POST /api/items/{partNumber}/revisions/{revision}/rollback # Rollback to revision [editor]
|
||||
|
||||
# Files
|
||||
GET /api/items/{partNumber}/files # List item file attachments
|
||||
GET /api/items/{partNumber}/file # Download latest file
|
||||
GET /api/items/{partNumber}/file/{revision} # Download file at revision
|
||||
POST /api/items/{partNumber}/file # Upload file [editor]
|
||||
POST /api/items/{partNumber}/files # Associate uploaded file with item [editor]
|
||||
DELETE /api/items/{partNumber}/files/{fileId} # Remove file association [editor]
|
||||
PUT /api/items/{partNumber}/thumbnail # Set item thumbnail [editor]
|
||||
|
||||
# BOM
|
||||
GET /api/items/{partNumber}/bom # List direct children
|
||||
GET /api/items/{partNumber}/bom/expanded # Multi-level BOM (recursive)
|
||||
GET /api/items/{partNumber}/bom/flat # Flattened BOM (leaf parts, rolled-up quantities)
|
||||
GET /api/items/{partNumber}/bom/cost # Assembly cost roll-up
|
||||
GET /api/items/{partNumber}/bom/where-used # Where-used (parent lookup)
|
||||
GET /api/items/{partNumber}/bom/export.csv # Export BOM as CSV
|
||||
GET /api/items/{partNumber}/bom/export.ods # Export BOM as ODS
|
||||
POST /api/items/{partNumber}/bom # Add BOM entry [editor]
|
||||
POST /api/items/{partNumber}/bom/import # Import BOM from CSV [editor]
|
||||
PUT /api/items/{partNumber}/bom/{childPartNumber} # Update BOM entry [editor]
|
||||
DELETE /api/items/{partNumber}/bom/{childPartNumber} # Remove BOM entry [editor]
|
||||
|
||||
# Audit (viewer)
|
||||
GET /api/audit/completeness # Item completeness scores
|
||||
GET /api/audit/completeness/{partNumber} # Item detail breakdown
|
||||
|
||||
# Integrations — Odoo (read: viewer, write: editor)
|
||||
GET /api/integrations/odoo/config # Get Odoo configuration
|
||||
GET /api/integrations/odoo/sync-log # Get sync history
|
||||
PUT /api/integrations/odoo/config # Update Odoo config [editor]
|
||||
POST /api/integrations/odoo/test-connection # Test connection [editor] (stub)
|
||||
POST /api/integrations/odoo/sync/push/{partNumber} # Push to Odoo [editor] (stub)
|
||||
POST /api/integrations/odoo/sync/pull/{odooId} # Pull from Odoo [editor] (stub)
|
||||
|
||||
# Sheets (editor)
|
||||
POST /api/sheets/diff # Diff ODS sheet against DB [editor]
|
||||
|
||||
# Part Number Generation (editor)
|
||||
POST /api/generate-part-number # Generate without creating item [editor]
|
||||
```
|
||||
|
||||
### 11.2 Not Yet Implemented
|
||||
|
||||
The following endpoints from the original design are not yet implemented:
|
||||
|
||||
```
|
||||
# Locations (tables exist, no API handlers)
|
||||
GET /api/locations
|
||||
POST /api/locations
|
||||
GET /api/locations/{path}
|
||||
DELETE /api/locations/{path}
|
||||
|
||||
# Inventory (tables exist, no API handlers)
|
||||
GET /api/inventory/{partNumber}
|
||||
POST /api/inventory/{partNumber}/adjust
|
||||
POST /api/inventory/{partNumber}/move
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. MVP Scope
|
||||
|
||||
### 12.1 Implemented
|
||||
|
||||
- [x] PostgreSQL database schema (11 migrations)
|
||||
- [x] YAML schema parser for part numbering
|
||||
- [x] Part number generation engine
|
||||
- [x] CLI tool (`cmd/silo`)
|
||||
- [x] API server (`cmd/silod`) with 75 endpoints
|
||||
- [x] MinIO integration for file storage with versioning
|
||||
- [x] BOM relationships (component, alternate, reference)
|
||||
- [x] Multi-level BOM (recursive expansion with configurable depth)
|
||||
- [x] Where-used queries (reverse parent lookup)
|
||||
- [x] Flat BOM flattening with quantity roll-up and cycle detection
|
||||
- [x] Assembly cost roll-up using standard_cost
|
||||
- [x] BOM CSV and ODS export/import
|
||||
- [x] Reference designator tracking
|
||||
- [x] Revision history (append-only) with rollback and comparison
|
||||
- [x] Revision status and labels
|
||||
- [x] Project management with many-to-many item tagging
|
||||
- [x] CSV import/export with dry-run validation
|
||||
- [x] ODS spreadsheet import/export (items, BOMs, project sheets)
|
||||
- [x] Web UI for items, projects, schemas, audit, settings (React SPA)
|
||||
- [x] File attachments with presigned upload and thumbnail support
|
||||
- [x] Authentication (local, LDAP, OIDC) with role-based access control
|
||||
- [x] API token management (SHA-256 hashed)
|
||||
- [x] Session management (PostgreSQL-backed)
|
||||
- [x] Audit logging and completeness scoring
|
||||
- [x] CSRF protection (nosurf)
|
||||
- [x] Fuzzy search
|
||||
- [x] Property schema versioning framework
|
||||
- [x] Docker Compose deployment (dev and prod)
|
||||
- [x] systemd service and deployment scripts
|
||||
|
||||
### 12.2 Partially Implemented
|
||||
|
||||
- [ ] Location hierarchy (database tables exist, no API endpoints)
|
||||
- [ ] Inventory tracking (database tables exist, no API endpoints)
|
||||
- [ ] Date segment type (schema parser placeholder only)
|
||||
- [ ] Part number format validation on creation
|
||||
- [ ] Odoo ERP integration (config and sync-log functional; push/pull are stubs)
|
||||
|
||||
### 12.3 Not Started
|
||||
|
||||
- [ ] Unit tests (Go server — 9 test files exist, coverage is partial)
|
||||
- [ ] Schema migration tooling
|
||||
- [ ] Checkout locking
|
||||
- [ ] Approval workflows
|
||||
- [ ] Exploded file storage with diffing
|
||||
- [ ] Notifications
|
||||
- [ ] Reporting/analytics
|
||||
|
||||
---
|
||||
|
||||
## 13. Open Questions
|
||||
|
||||
1. **Thumbnail generation**: Generate thumbnails from CAD files on commit? Useful for web UI browsing.
|
||||
|
||||
2. **Search indexing**: PostgreSQL full-text search sufficient, or add dedicated search (Meilisearch, etc.)?
|
||||
|
||||
3. **Checkout locking**: Pessimistic vs optimistic locking strategy for multi-user CAD file editing.
|
||||
|
||||
---
|
||||
|
||||
## 14. References
|
||||
|
||||
### 14.1 Design Influences
|
||||
|
||||
- **CycloneDX BOM specification**: JSON/YAML schema patterns for component identification, relationships, and metadata (https://cyclonedx.org)
|
||||
- **OpenBOM data model**: Reference-instance separation, flexible property schemas
|
||||
- **Ansible inventory YAML**: Hierarchical configuration patterns with variable inheritance
|
||||
|
||||
### 14.2 Related Standards
|
||||
|
||||
- **ISO 10303 (STEP)**: Product data representation
|
||||
- **IPC-2581**: Electronics assembly BOM format
|
||||
- **Package URL (PURL)**: Standardized component identification
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: Example YAML Files
|
||||
|
||||
### A.1 Complete Part Numbering Schema
|
||||
|
||||
See `schemas/kindred-rd.yaml` for the full schema (v3). Summary:
|
||||
|
||||
```yaml
|
||||
# kindred-rd-schema.yaml (abbreviated)
|
||||
schema:
|
||||
name: kindred-rd
|
||||
version: 3
|
||||
description: "Kindred Systems R&D part numbering"
|
||||
|
||||
separator: "-"
|
||||
|
||||
uniqueness:
|
||||
scope: global
|
||||
case_sensitive: false
|
||||
|
||||
segments:
|
||||
- name: category
|
||||
type: enum
|
||||
description: "Category code"
|
||||
required: true
|
||||
values:
|
||||
F01: "Hex Cap Screw"
|
||||
F02: "Socket Head Cap Screw"
|
||||
# ... 70+ categories (see full file)
|
||||
|
||||
- name: sequence
|
||||
type: serial
|
||||
length: 4
|
||||
padding: "0"
|
||||
start: 1
|
||||
description: "Sequential number within category"
|
||||
scope: "{category}"
|
||||
|
||||
format: "{category}-{sequence}"
|
||||
|
||||
# Example outputs: F01-0001, R27-0001, A01-0001
|
||||
```
|
||||
|
||||
### A.2 Complete Location Schema
|
||||
|
||||
```yaml
|
||||
# kindred-locations.yaml
|
||||
location_schema:
|
||||
name: kindred-lab
|
||||
version: 1
|
||||
description: "Kindred Systems lab and storage locations"
|
||||
|
||||
path_separator: "/"
|
||||
|
||||
hierarchy:
|
||||
- level: 0
|
||||
type: facility
|
||||
description: "Building or site"
|
||||
name_pattern: "^[a-z][a-z0-9-]*$"
|
||||
examples: ["lab", "warehouse", "office"]
|
||||
|
||||
- level: 1
|
||||
type: area
|
||||
description: "Room or zone within facility"
|
||||
name_pattern: "^[a-z][a-z0-9-]*$"
|
||||
examples: ["main-lab", "storage", "assembly"]
|
||||
|
||||
- level: 2
|
||||
type: shelf
|
||||
description: "Shelving unit"
|
||||
name_pattern: "^shelf-[a-z]$"
|
||||
examples: ["shelf-a", "shelf-b"]
|
||||
|
||||
- level: 3
|
||||
type: bin
|
||||
description: "Individual container or bin"
|
||||
name_pattern: "^bin-[0-9]{1,3}$"
|
||||
examples: ["bin-1", "bin-42", "bin-100"]
|
||||
|
||||
# Properties tracked per location type
|
||||
properties:
|
||||
facility:
|
||||
- name: address
|
||||
type: text
|
||||
required: false
|
||||
area:
|
||||
- name: climate_controlled
|
||||
type: boolean
|
||||
default: false
|
||||
shelf:
|
||||
- name: max_weight_kg
|
||||
type: number
|
||||
required: false
|
||||
bin:
|
||||
- name: bin_size
|
||||
type: enum
|
||||
values: [small, medium, large]
|
||||
default: medium
|
||||
```
|
||||
|
||||
### A.3 Assembly Configuration
|
||||
|
||||
```yaml
|
||||
# Stored as item property or linked document
|
||||
# Example: assembly PROTO-AS-0001
|
||||
assembly_config:
|
||||
name: "Main Chassis Assembly"
|
||||
|
||||
relationship_types:
|
||||
- component
|
||||
- alternate
|
||||
- reference
|
||||
|
||||
use_reference_designators: false
|
||||
|
||||
child_revision_tracking: latest
|
||||
|
||||
# Assembly-specific BOM properties
|
||||
relationship_properties:
|
||||
- name: installation_notes
|
||||
type: text
|
||||
- name: torque_spec
|
||||
type: text
|
||||
- name: adhesive_required
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
# Validation rules
|
||||
validation:
|
||||
require_quantity: true
|
||||
min_components: 1
|
||||
```
|
||||
96
docs/src/silo-server/STATUS.md
Normal file
96
docs/src/silo-server/STATUS.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Silo Development Status
|
||||
|
||||
**Last Updated:** 2026-02-08
|
||||
|
||||
---
|
||||
|
||||
## Implementation Status
|
||||
|
||||
### Core Systems
|
||||
|
||||
| Component | Status | Notes |
|
||||
|-----------|--------|-------|
|
||||
| PostgreSQL schema | Complete | 11 migrations applied |
|
||||
| YAML schema parser | Complete | Supports enum, serial, constant, string segments |
|
||||
| Part number generator | Complete | Scoped sequences, category-based format |
|
||||
| API server (`silod`) | Complete | 75 REST endpoints via chi/v5 |
|
||||
| CLI tool (`silo`) | Complete | Item registration and management |
|
||||
| MinIO file storage | Complete | Upload, download, versioning, checksums |
|
||||
| Revision control | Complete | Append-only history, rollback, comparison, status/labels |
|
||||
| Project management | Complete | CRUD, many-to-many item tagging |
|
||||
| CSV import/export | Complete | Dry-run validation, template generation |
|
||||
| ODS import/export | Complete | Items, BOMs, project sheets, templates |
|
||||
| Multi-level BOM | Complete | Recursive expansion, where-used, CSV/ODS export/import |
|
||||
| Authentication | Complete | Local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak |
|
||||
| Role-based access control | Complete | admin > editor > viewer hierarchy |
|
||||
| API token management | Complete | SHA-256 hashed, bearer auth |
|
||||
| Session management | Complete | PostgreSQL-backed (pgxstore), 24h lifetime |
|
||||
| Audit logging | Complete | audit_log table, completeness scoring |
|
||||
| CSRF protection | Complete | nosurf on web forms |
|
||||
| Fuzzy search | Complete | sahilm/fuzzy library |
|
||||
| Web UI | Complete | React SPA (Vite + TypeScript), 6 pages, Catppuccin Mocha theme |
|
||||
| File attachments | Complete | Presigned uploads, item file association, thumbnails |
|
||||
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull are stubs |
|
||||
| Docker Compose | Complete | Dev and production configurations |
|
||||
| Deployment scripts | Complete | setup-host, deploy, init-db, setup-ipa-nginx |
|
||||
| systemd service | Complete | Unit file and environment template |
|
||||
|
||||
### Client Integrations
|
||||
|
||||
FreeCAD workbench and LibreOffice Calc extension are maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The server provides the REST API and ODS endpoints consumed by those clients.
|
||||
|
||||
### Not Yet Implemented
|
||||
|
||||
| Feature | Notes |
|
||||
|---------|-------|
|
||||
| Location API endpoints | Database tables exist (`locations`, `inventory`), no REST handlers |
|
||||
| Inventory API endpoints | Database tables exist, no REST handlers |
|
||||
| Date segment type | Schema parser placeholder only |
|
||||
| Part number format validation | API accepts but does not validate format on creation |
|
||||
| Unit tests | 9 Go test files across api, db, ods, partnum, schema packages |
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure
|
||||
|
||||
| Service | Host | Status |
|
||||
|---------|------|--------|
|
||||
| PostgreSQL | psql.kindred.internal:5432 | Running |
|
||||
| MinIO | localhost:9000 (API) / :9001 (console) | Configured |
|
||||
| Silo API | localhost:8080 | Builds successfully |
|
||||
|
||||
---
|
||||
|
||||
## Schema Status
|
||||
|
||||
The part numbering schema (`kindred-rd`) is at **version 3** using the `{category}-{sequence}` format (e.g., `F01-0001`). This replaced the earlier `{project}-{type}-{sequence}` format. Projects are now managed as many-to-many tags rather than being embedded in part numbers.
|
||||
|
||||
The schema defines 170 category codes across 10 groups:
|
||||
- F01-F18: Fasteners
|
||||
- C01-C17: Fluid Fittings
|
||||
- R01-R44: Motion Components
|
||||
- S01-S17: Structural Materials
|
||||
- E01-E27: Electrical Components
|
||||
- M01-M18: Mechanical Components
|
||||
- T01-T08: Tooling and Fixtures
|
||||
- A01-A07: Assemblies
|
||||
- P01-P05: Purchased/Off-the-Shelf
|
||||
- X01-X08: Custom Fabricated Parts
|
||||
|
||||
---
|
||||
|
||||
## Database Migrations
|
||||
|
||||
| Migration | Description |
|
||||
|-----------|-------------|
|
||||
| 001_initial.sql | Core schema (items, revisions, relationships, locations, inventory, sequences) |
|
||||
| 002_sequence_by_name.sql | Sequence naming changes |
|
||||
| 003_remove_material.sql | Schema cleanup |
|
||||
| 004_cad_sync_state.sql | CAD synchronization state |
|
||||
| 005_property_schema_version.sql | Property versioning framework |
|
||||
| 006_project_tags.sql | Many-to-many project-item relationships |
|
||||
| 007_revision_status.sql | Revision status and labels |
|
||||
| 008_odoo_integration.sql | Odoo ERP integration tables (integrations, sync_log) |
|
||||
| 009_auth.sql | Authentication system (users, api_tokens, sessions, audit_log, user tracking columns) |
|
||||
| 010_item_extended_fields.sql | Extended item fields (sourcing_type, sourcing_link, standard_cost, long_description) |
|
||||
| 011_item_files.sql | Item file attachments (item_files table, thumbnail_key column) |
|
||||
728
docs/src/silo-server/frontend-spec.md
Normal file
728
docs/src/silo-server/frontend-spec.md
Normal file
@@ -0,0 +1,728 @@
|
||||
# Silo Frontend Specification
|
||||
|
||||
Current as of 2026-02-08. Documents the React + Vite + TypeScript frontend (migration from Go templates is complete).
|
||||
|
||||
## Overview
|
||||
|
||||
The Silo web UI has been migrated from server-rendered Go templates to a React single-page application. The Go templates (~7,000 lines across 7 files) have been removed. The Go API server serves JSON at `/api/*` and the React SPA at `/`.
|
||||
|
||||
**Stack**: React 19, React Router 7, Vite 6, TypeScript 5.7
|
||||
**Theme**: Catppuccin Mocha (dark) via CSS custom properties
|
||||
**Styling**: Inline React styles using `React.CSSProperties` — no CSS modules, no Tailwind, no styled-components
|
||||
**State**: Local `useState` + custom hooks. No global state library (no Redux, Zustand, etc.)
|
||||
**Dependencies**: Minimal — only `react`, `react-dom`, `react-router-dom`. No axios, no tanstack-query.
|
||||
|
||||
## Migration Status
|
||||
|
||||
| Phase | Issue | Title | Status |
|
||||
|-------|-------|-------|--------|
|
||||
| 1 | #7 | Scaffold React + Vite + TS, shared layout, auth, API client | Code complete |
|
||||
| 2 | #8 | Migrate Items page with UI improvements | Code complete |
|
||||
| 3 | #9 | Migrate Projects, Schemas, Settings, Login pages | Code complete |
|
||||
| 4 | #10 | Remove Go templates, Docker integration, cleanup | Complete |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser
|
||||
└── React SPA (served at /)
|
||||
├── Vite dev server (development) → proxies /api/* to Go backend
|
||||
└── Static files in web/dist/ (production) → served by Go binary
|
||||
|
||||
Go Server (silod)
|
||||
├── /api/* JSON REST API
|
||||
├── /login, /logout Session auth endpoints (form POST)
|
||||
├── /auth/oidc OIDC redirect flow
|
||||
└── /* React SPA (NotFound handler serves index.html for client-side routing)
|
||||
```
|
||||
|
||||
### Auth Flow
|
||||
|
||||
1. React app loads, `AuthProvider` calls `GET /api/auth/me`
|
||||
2. If 401 → render `LoginPage` (React form)
|
||||
3. Login form POSTs `application/x-www-form-urlencoded` to `/login` (Go handler sets session cookie)
|
||||
4. On success, `AuthProvider.refresh()` re-fetches `/api/auth/me`, user state populates, app renders
|
||||
5. OIDC: link to `/auth/oidc` triggers Go-served redirect flow, callback sets session, user returns to app
|
||||
6. API client auto-redirects to `/login` on any 401 response
|
||||
|
||||
### Public API Endpoint
|
||||
|
||||
`GET /api/auth/config` — returns `{ oidc_enabled: bool, local_enabled: bool }` so the login page can conditionally show the OIDC button without hardcoding.
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
web/
|
||||
├── index.html
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
├── tsconfig.node.json
|
||||
├── vite.config.ts
|
||||
└── src/
|
||||
├── main.tsx Entry point, renders AuthProvider + BrowserRouter + App
|
||||
├── App.tsx Route definitions, auth guard
|
||||
├── api/
|
||||
│ ├── client.ts fetch wrapper: get, post, put, del + ApiError class
|
||||
│ └── types.ts All TypeScript interfaces (272 lines)
|
||||
├── context/
|
||||
│ └── AuthContext.tsx AuthProvider with login/logout/refresh methods
|
||||
├── hooks/
|
||||
│ ├── useAuth.ts Context consumer hook
|
||||
│ ├── useItems.ts Items fetching with search, filters, pagination, debounce
|
||||
│ └── useLocalStorage.ts Typed localStorage persistence hook
|
||||
├── styles/
|
||||
│ ├── theme.css Catppuccin Mocha CSS custom properties
|
||||
│ └── global.css Base element styles
|
||||
├── components/
|
||||
│ ├── AppShell.tsx Header nav + user info + <Outlet/>
|
||||
│ ├── ContextMenu.tsx Reusable right-click positioned menu
|
||||
│ └── items/ Items page components (16 files)
|
||||
│ ├── ItemsToolbar.tsx Search, filters, layout toggle, action buttons
|
||||
│ ├── ItemTable.tsx Sortable table, column config, compact rows
|
||||
│ ├── ItemDetail.tsx 5-tab detail panel (Main, Properties, Revisions, BOM, Where Used)
|
||||
│ ├── MainTab.tsx Metadata display, project tags editor, file info
|
||||
│ ├── PropertiesTab.tsx Form/JSON dual-mode property editor
|
||||
│ ├── RevisionsTab.tsx Revision list, compare diff, status, rollback
|
||||
│ ├── BOMTab.tsx BOM table with inline CRUD, cost calculations
|
||||
│ ├── WhereUsedTab.tsx Parent assemblies table
|
||||
│ ├── SplitPanel.tsx Resizable horizontal/vertical layout container
|
||||
│ ├── FooterStats.tsx Fixed bottom bar with item counts
|
||||
│ ├── CreateItemPane.tsx In-pane create form with schema category properties
|
||||
│ ├── EditItemPane.tsx In-pane edit form
|
||||
│ ├── DeleteItemPane.tsx In-pane delete confirmation
|
||||
│ └── ImportItemsPane.tsx CSV upload with dry-run/import flow
|
||||
└── pages/
|
||||
├── LoginPage.tsx Username/password form + OIDC button
|
||||
├── ItemsPage.tsx Orchestrator: toolbar, split panel, table, detail/CRUD panes
|
||||
├── ProjectsPage.tsx Project CRUD with sortable table, in-pane forms
|
||||
├── SchemasPage.tsx Schema browser with collapsible segments, enum value CRUD
|
||||
├── SettingsPage.tsx Account info, API token management
|
||||
└── AuditPage.tsx Audit completeness (placeholder, expanded in Issue #5)
|
||||
```
|
||||
|
||||
**Total**: ~40 source files, ~7,600 lines of TypeScript/TSX.
|
||||
|
||||
## Design System
|
||||
|
||||
### Theme
|
||||
|
||||
Catppuccin Mocha dark theme. All colors referenced via CSS custom properties:
|
||||
|
||||
| Token | Use |
|
||||
|-------|-----|
|
||||
| `--ctp-base` | Page background, input backgrounds |
|
||||
| `--ctp-mantle` | Header background |
|
||||
| `--ctp-surface0` | Card backgrounds, even table rows |
|
||||
| `--ctp-surface1` | Borders, dividers, hover states |
|
||||
| `--ctp-surface2` | Secondary button backgrounds |
|
||||
| `--ctp-text` | Primary text |
|
||||
| `--ctp-subtext0/1` | Secondary/muted text, labels |
|
||||
| `--ctp-overlay0` | Placeholder text |
|
||||
| `--ctp-mauve` | Brand accent, primary buttons, nav active |
|
||||
| `--ctp-blue` | Editor role badge, edit headers |
|
||||
| `--ctp-green` | Success banners, create headers |
|
||||
| `--ctp-red` | Errors, delete actions, danger buttons |
|
||||
| `--ctp-peach` | Part numbers, project codes, token prefixes |
|
||||
| `--ctp-teal` | Viewer role badge |
|
||||
| `--ctp-sapphire` | Links, collapsible toggles |
|
||||
| `--ctp-crust` | Dark text on colored backgrounds |
|
||||
|
||||
### Typography
|
||||
|
||||
- Body: system font stack (Inter, -apple-system, etc.)
|
||||
- Monospace: JetBrains Mono (part numbers, codes, tokens)
|
||||
- Table cells: 0.85rem
|
||||
- Labels: 0.85rem, weight 500
|
||||
- Table headers: 0.8rem, uppercase, letter-spacing 0.05em
|
||||
|
||||
### Component Patterns
|
||||
|
||||
**Tables**: Inline styles, compact rows (28-32px), alternating `base`/`surface0` backgrounds, sortable headers with arrow indicators, right-click column config (Items page).
|
||||
|
||||
**Forms**: In-pane forms (Infor ERP-style) — not modal overlays. Create/Edit/Delete forms render in the detail pane area with a colored header bar (green=create, blue=edit, red=delete). Cancel returns to previous view.
|
||||
|
||||
**Cards**: `surface0` background, `0.75rem` border radius, `1.5rem` padding.
|
||||
|
||||
**Buttons**: Primary (`mauve` bg, `crust` text), secondary (`surface1` bg), danger (`red` bg or translucent red bg with red text).
|
||||
|
||||
**Errors**: Red text with translucent red background banner, `0.4rem` border radius.
|
||||
|
||||
**Role badges**: Colored pill badges — admin=mauve, editor=blue, viewer=teal.
|
||||
|
||||
## Page Specifications
|
||||
|
||||
### Items Page (completed in #8)
|
||||
|
||||
The most complex page. Master-detail layout with resizable split panel.
|
||||
|
||||
**Toolbar**: Debounced search (300ms) with scope toggle (All/PN/Description), type and project filter dropdowns, layout toggle (horizontal/vertical), export/import/create buttons.
|
||||
|
||||
**Table**: 7 configurable columns (part_number, item_type, description, revision, projects, created, actions). Visibility stored per layout mode in localStorage. Right-click header opens ContextMenu with checkboxes. Compact rows, zebra striping, click to select.
|
||||
|
||||
**Detail panel**: 5 tabs — Main (metadata + project tags + file info), Properties (form/JSON editor, save creates revision), Revisions (compare, status management, rollback), BOM (inline CRUD, cost calculations, CSV export), Where Used (parent assemblies).
|
||||
|
||||
**CRUD panes**: In-pane forms for Create (schema category properties, project tags), Edit (basic fields), Delete (confirmation), Import (CSV upload with dry-run).
|
||||
|
||||
**Footer**: Fixed 28px bottom bar showing Total | Parts | Assemblies | Documents counts, reactive to filters.
|
||||
|
||||
**State**: `PaneMode` discriminated union manages which pane is shown. `useItems` hook handles fetching, search, filters, pagination. `useLocalStorage` persists layout and column preferences.
|
||||
|
||||
### Projects Page (completed in #9)
|
||||
|
||||
Sortable table with columns: Code, Name, Description, Items (count fetched per project), Created, Actions.
|
||||
|
||||
**CRUD**: In-pane forms above the table. Create requires code (2-10 chars, auto-uppercase), name, description. Edit allows name and description changes. Delete shows confirmation with project code.
|
||||
|
||||
**Navigation**: Click project code navigates to Items page with `?project=CODE` filter.
|
||||
|
||||
**Permissions**: Create/Edit/Delete buttons only visible to editor/admin roles.
|
||||
|
||||
### Schemas Page (completed in #9)
|
||||
|
||||
Schema cards with collapsible segment details. Each schema shows name, description, format string, version, and example part numbers.
|
||||
|
||||
**Segments**: Expandable list showing segment name, type badge, description. Enum segments include a values table with code and description columns.
|
||||
|
||||
**Enum CRUD**: Inline table operations — add row at bottom, edit replaces the row, delete highlights the row with confirmation. All operations call `POST/PUT/DELETE /api/schemas/{name}/segments/{segment}/values/{code}`.
|
||||
|
||||
### Settings Page (completed in #9)
|
||||
|
||||
Two cards:
|
||||
|
||||
**Account**: Read-only grid showing username, display name, email, auth source, role (with colored badge). Data from `useAuth()` context.
|
||||
|
||||
**API Tokens**: Create form (name input + button), one-time token display in green banner with copy-to-clipboard, token list table (name, prefix, created, last used, expires, revoke). Revoke has inline confirm step. Uses `GET/POST/DELETE /api/auth/tokens`.
|
||||
|
||||
### Login Page (completed in #9)
|
||||
|
||||
Standalone centered card (no AppShell). Username/password form, OIDC button shown conditionally based on `GET /api/auth/config`. Error messages in red banner. Submit calls `AuthContext.login()` which POSTs form data to `/login` then re-fetches the user.
|
||||
|
||||
### Audit Page (placeholder)
|
||||
|
||||
Basic table showing audit completeness data from `GET /api/audit/completeness`. Will be expanded as part of Issue #5 (Component Audit UI with completeness scoring and inline editing).
|
||||
|
||||
## API Client
|
||||
|
||||
`web/src/api/client.ts` — thin wrapper around `fetch`:
|
||||
|
||||
- Always sends `credentials: 'include'` for session cookies
|
||||
- Always sets `Content-Type: application/json`
|
||||
- 401 responses redirect to `/login`
|
||||
- Non-OK responses parsed as `{ error, message }` and thrown as `ApiError`
|
||||
- 204 responses return `undefined`
|
||||
- Exports: `get<T>()`, `post<T>()`, `put<T>()`, `del()`
|
||||
|
||||
## Type Definitions
|
||||
|
||||
`web/src/api/types.ts` — 272 lines covering all API response and request shapes:
|
||||
|
||||
**Core models**: User, Item, Project, Schema, SchemaSegment, Revision, BOMEntry
|
||||
**Audit**: AuditFieldResult, AuditItemResult, AuditSummary, AuditCompletenessResponse
|
||||
**Search**: FuzzyResult (extends Item with score)
|
||||
**BOM**: WhereUsedEntry, AddBOMEntryRequest, UpdateBOMEntryRequest
|
||||
**Items**: CreateItemRequest, UpdateItemRequest, CreateRevisionRequest
|
||||
**Projects**: CreateProjectRequest, UpdateProjectRequest
|
||||
**Schemas**: CreateSchemaValueRequest, UpdateSchemaValueRequest, PropertyDef, PropertySchema
|
||||
**Auth**: AuthConfig, ApiToken, ApiTokenCreated
|
||||
**Revisions**: RevisionComparison
|
||||
**Import**: CSVImportResult, CSVImportError
|
||||
**Errors**: ErrorResponse
|
||||
|
||||
## Completed Work
|
||||
|
||||
### Issue #10: Remove Go Templates + Docker Integration -- COMPLETE
|
||||
|
||||
Completed in commit `50923cf`. All Go templates deleted, `web.go` handler removed, SPA serves at `/` via `NotFound` handler with `index.html` fallback. `build/package/Dockerfile` added.
|
||||
|
||||
### Remaining Work
|
||||
|
||||
### Issue #5: Component Audit UI (future)
|
||||
|
||||
The Audit page will be expanded with completeness scoring, inline editing, tier filtering, and category breakdowns. This will be built natively in React using the patterns established in the migration.
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
cd web && npm install
|
||||
|
||||
# Dev server (proxies /api/* to Go backend on :8080)
|
||||
npm run dev
|
||||
|
||||
# Type check
|
||||
npx tsc --noEmit
|
||||
|
||||
# Production build
|
||||
npm run build
|
||||
```
|
||||
|
||||
Vite dev server runs on port 5173 with proxy config in `vite.config.ts` forwarding `/api/*`, `/login`, `/logout`, `/auth/*` to the Go backend.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **No modals for CRUD** — use in-pane forms (Infor ERP-style pattern)
|
||||
- **No shared component library extraction** until a pattern repeats 3+ times
|
||||
- **Inline styles only** — all styling via `React.CSSProperties` objects, using Catppuccin CSS variables
|
||||
- **No class components** — functional components with hooks only
|
||||
- **Permission checks**: derive `isEditor` from `user.role` in each page, conditionally render write actions
|
||||
- **Error handling**: try/catch with error state, display in red banners inline
|
||||
- **Data fetching**: `useEffect` + API client on mount, loading/error/data states
|
||||
- **Persistence**: `useLocalStorage` hook for user preferences (layout mode, column visibility)
|
||||
|
||||
## New Frontend Tasks
|
||||
|
||||
# CreateItemPane Redesign Specification
|
||||
|
||||
**Date**: 2026-02-06
|
||||
**Scope**: Replace existing `CreateItemPane.tsx` with a two-column layout, multi-stage category picker, file attachment via MinIO, and full use of screen real estate.
|
||||
**Parent**: Items page (`ItemsPage.tsx`) — renders in the detail pane area per existing in-pane CRUD pattern.
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
The pane uses a CSS Grid two-column layout instead of the current single-column form:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────┬──────────────┐
|
||||
│ Header: "New Item" [green bar] Cancel │ Create │ │
|
||||
├──────────────────────────────────────────────────────┤ │
|
||||
│ │ Auto- │
|
||||
│ ── Identity ────────────────────────────────────── │ assigned │
|
||||
│ [Part Number *] [Type * v] │ metadata │
|
||||
│ [Description ] │ │
|
||||
│ Category * [Domain │ Group │ Subtype ] │──────────────│
|
||||
│ Mechanical│ Structural│ Bracket │ │ │
|
||||
│ Electrical│ Bearings │ Plate │ │ Attachments │
|
||||
│ ... │ ... │ ... │ │ ┌─ ─ ─ ─ ┐ │
|
||||
│ ── Sourcing ────────────────────────────────────── │ │ Drop │ │
|
||||
│ [Sourcing Type v] [Standard Cost $ ] │ │ zone │ │
|
||||
│ [Unit of Measure v] [Sourcing Link ] │ └─ ─ ─ ─ ┘ │
|
||||
│ │ file.FCStd │
|
||||
│ ── Details ─────────────────────────────────────── │ drawing.pdf │
|
||||
│ [Long Description ] │ │
|
||||
│ [Projects: [tag][tag] type to search... ] │──────────────│
|
||||
│ │ Thumbnail │
|
||||
│ │ [preview] │
|
||||
└──────────────────────────────────────────────────────┴──────────────┘
|
||||
```
|
||||
|
||||
Grid definition: `grid-template-columns: 1fr 320px`. The left column scrolls independently if content overflows. The right sidebar is a flex column with sections separated by `--ctp-surface1` borders.
|
||||
|
||||
## File Location
|
||||
|
||||
`web/src/components/items/CreateItemPane.tsx` (replaces existing file)
|
||||
|
||||
New supporting files:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `web/src/components/items/CategoryPicker.tsx` | Multi-stage category selector |
|
||||
| `web/src/components/items/FileDropZone.tsx` | Drag-and-drop file upload with MinIO presigned URLs |
|
||||
| `web/src/components/items/TagInput.tsx` | Multi-select tag input for projects |
|
||||
| `web/src/hooks/useCategories.ts` | Fetches category tree from schema data |
|
||||
| `web/src/hooks/useFileUpload.ts` | Manages presigned URL upload flow |
|
||||
|
||||
## Component Breakdown
|
||||
|
||||
### CreateItemPane
|
||||
|
||||
Top-level orchestrator. Manages form state, submission, and layout.
|
||||
|
||||
**Props** (unchanged interface):
|
||||
|
||||
```typescript
|
||||
interface CreateItemPaneProps {
|
||||
onCreated: (item: Item) => void;
|
||||
onCancel: () => void;
|
||||
}
|
||||
```
|
||||
|
||||
**State**:
|
||||
|
||||
```typescript
|
||||
const [form, setForm] = useState<CreateItemForm>({
|
||||
part_number: '',
|
||||
item_type: 'part',
|
||||
description: '',
|
||||
category_path: [], // e.g. ['Mechanical', 'Structural', 'Bracket']
|
||||
sourcing_type: 'manufactured',
|
||||
standard_cost: '',
|
||||
unit_of_measure: 'ea',
|
||||
sourcing_link: '',
|
||||
long_description: '',
|
||||
project_ids: [],
|
||||
});
|
||||
const [attachments, setAttachments] = useState<PendingAttachment[]>([]);
|
||||
const [thumbnail, setThumbnail] = useState<PendingAttachment | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [submitting, setSubmitting] = useState(false);
|
||||
```
|
||||
|
||||
**Submission flow**:
|
||||
|
||||
1. Validate required fields (part_number, item_type, category_path length === 3).
|
||||
2. `POST /api/items` with form data → returns created `Item` with UUID.
|
||||
3. For each attachment in `attachments[]`, call the file association endpoint: `POST /api/items/{id}/files` with the MinIO object key returned from upload.
|
||||
4. If thumbnail exists, `PUT /api/items/{id}/thumbnail` with the object key.
|
||||
5. Call `onCreated(item)`.
|
||||
|
||||
If step 2 fails, show error banner. If file association fails, show warning but still navigate (item was created, files can be re-attached).
|
||||
|
||||
**Header bar**: Green (`--ctp-green` background, `--ctp-crust` text) per existing create-pane convention. "New Item" title on left, Cancel (ghost button) and Create Item (primary button, `--ctp-green` bg) on right.
|
||||
|
||||
### CategoryPicker
|
||||
|
||||
Three-column scrollable list for hierarchical category selection.
|
||||
|
||||
**Props**:
|
||||
|
||||
```typescript
|
||||
interface CategoryPickerProps {
|
||||
value: string[]; // current selection path, e.g. ['Mechanical', 'Structural']
|
||||
onChange: (path: string[]) => void;
|
||||
categories: CategoryNode[]; // top-level nodes
|
||||
}
|
||||
|
||||
interface CategoryNode {
|
||||
name: string;
|
||||
children?: CategoryNode[];
|
||||
}
|
||||
```
|
||||
|
||||
**Rendering**: Three side-by-side `<div>` columns inside a container with `border: 1px solid var(--ctp-surface1)` and `border-radius: 0.4rem`. Each column has:
|
||||
|
||||
- A sticky header row (10px uppercase, `--ctp-overlay0` text, `--ctp-mantle` background) labeling the tier. Labels come from the schema definition if available, otherwise "Level 1", "Level 2", "Level 3".
|
||||
- A scrollable list of options. Each option is a `<div>` row, 28px height, `0.85rem` font. Hover: `--ctp-surface0` background. Selected: translucent mauve background (`rgba(203, 166, 247, 0.12)`), `--ctp-mauve` text, weight 600.
|
||||
- If a node has children, show a `›` chevron on the right side of the row.
|
||||
|
||||
Column 1 always shows all top-level nodes. Column 2 shows children of the selected Column 1 node. Column 3 shows children of the selected Column 2 node. If nothing is selected in a column, the next column shows an empty state with muted text: "Select a [tier name]".
|
||||
|
||||
Below the picker, render a breadcrumb trail: `Mechanical › Structural › Bracket` in `--ctp-mauve` with `›` separators in `--ctp-overlay0`. Only show segments that are selected.
|
||||
|
||||
**Data source**: Categories are derived from schemas. The `useCategories` hook calls `GET /api/schemas` and transforms the response into a `CategoryNode[]` tree. The exact mapping depends on how schemas define category hierarchies — if schemas don't currently support hierarchical categories, this requires a backend addition (see Backend Changes section).
|
||||
|
||||
**Max height**: 180px per column with `overflow-y: auto`.
|
||||
|
||||
### FileDropZone
|
||||
|
||||
Handles drag-and-drop and click-to-browse file uploads with MinIO presigned URL flow.
|
||||
|
||||
**Props**:
|
||||
|
||||
```typescript
|
||||
interface FileDropZoneProps {
|
||||
files: PendingAttachment[];
|
||||
onFilesAdded: (files: PendingAttachment[]) => void;
|
||||
onFileRemoved: (index: number) => void;
|
||||
accept?: string; // e.g. '.FCStd,.step,.stl,.pdf,.png,.jpg'
|
||||
}
|
||||
|
||||
interface PendingAttachment {
|
||||
file: File;
|
||||
objectKey: string; // MinIO key after upload
|
||||
uploadProgress: number; // 0-100
|
||||
uploadStatus: 'pending' | 'uploading' | 'complete' | 'error';
|
||||
error?: string;
|
||||
}
|
||||
```
|
||||
|
||||
**Drop zone UI**: Dashed `2px` border using `--ctp-surface1`, `border-radius: 0.5rem`, centered content with a paperclip icon (Unicode 📎 or inline SVG), "Drop files here or **browse**" text, and accepted formats in `--ctp-overlay0` at 10px.
|
||||
|
||||
States:
|
||||
- **Default**: dashed border `--ctp-surface1`
|
||||
- **Drag over**: dashed border `--ctp-mauve`, background `rgba(203, 166, 247, 0.05)`
|
||||
- **Uploading**: show progress per file in the file list
|
||||
|
||||
Clicking the zone opens a hidden `<input type="file" multiple>`.
|
||||
|
||||
**File list**: Rendered below the drop zone. Each file shows:
|
||||
- Type icon: colored 28×28 rounded square. Color mapping: `.FCStd`/`.step`/`.stl` → `--ctp-blue` ("CAD"), `.pdf` → `--ctp-red` ("PDF"), `.png`/`.jpg` → `--ctp-green` ("IMG"), other → `--ctp-overlay1` ("FILE").
|
||||
- File name (truncated with ellipsis).
|
||||
- File size + type label in `--ctp-overlay0` at 10px.
|
||||
- Upload progress bar (thin 2px bar under the file item, `--ctp-mauve` fill) when uploading.
|
||||
- Remove button (`×`) on the right, `--ctp-overlay0` → `--ctp-red` on hover.
|
||||
|
||||
**Upload flow** (managed by `useFileUpload` hook):
|
||||
|
||||
1. On file selection/drop, immediately request a presigned upload URL: `POST /api/uploads/presign` with `{ filename, content_type, size }`.
|
||||
2. Backend returns `{ object_key, upload_url, expires_at }`.
|
||||
3. `PUT` the file directly to the presigned MinIO URL using `XMLHttpRequest` (for progress tracking).
|
||||
4. On completion, update `PendingAttachment.uploadStatus` to `'complete'` and store the `object_key`.
|
||||
5. The `object_key` is later sent to the item creation endpoint to associate the file.
|
||||
|
||||
If the presigned URL endpoint doesn't exist yet, see Backend Changes.
|
||||
|
||||
### TagInput
|
||||
|
||||
Reusable multi-select input for projects (and potentially other tag-like fields).
|
||||
|
||||
**Props**:
|
||||
|
||||
```typescript
|
||||
interface TagInputProps {
|
||||
value: string[]; // selected project IDs
|
||||
onChange: (ids: string[]) => void;
|
||||
placeholder?: string;
|
||||
searchFn: (query: string) => Promise<{ id: string; label: string }[]>;
|
||||
}
|
||||
```
|
||||
|
||||
**Rendering**: Container styled like a form input (`--ctp-crust` bg, `--ctp-surface1` border, `border-radius: 0.4rem`). Inside:
|
||||
- Selected tags as inline pills: `rgba(203, 166, 247, 0.15)` bg, `--ctp-mauve` text, 11px font, with `×` remove button.
|
||||
- A bare `<input>` (no border/bg) that grows to fill remaining width, `min-width: 80px`.
|
||||
|
||||
**Behavior**: On typing, debounce 200ms, call `searchFn(query)`. Show a dropdown below the input with matching results. Click or Enter selects. Already-selected items are excluded from results. Escape or blur closes the dropdown.
|
||||
|
||||
The dropdown is an absolutely-positioned `<div>` below the input container, `--ctp-crust` background, `--ctp-surface1` border, `border-radius: 0.4rem`, `max-height: 160px`, `overflow-y: auto`. Each row is 28px, hover `--ctp-surface0`.
|
||||
|
||||
**For projects**: `searchFn` calls `GET /api/projects?q={query}` and maps to `{ id: project.id, label: project.code + ' — ' + project.name }`.
|
||||
|
||||
### useCategories Hook
|
||||
|
||||
```typescript
|
||||
function useCategories(): {
|
||||
categories: CategoryNode[];
|
||||
loading: boolean;
|
||||
error: string | null;
|
||||
}
|
||||
```
|
||||
|
||||
Fetches `GET /api/schemas` on mount and transforms into a category tree. Caches in a module-level variable so repeated renders don't refetch. If the API doesn't currently support hierarchical categories, this returns a flat list as a single-tier picker until the backend is extended.
|
||||
|
||||
### useFileUpload Hook
|
||||
|
||||
```typescript
|
||||
function useFileUpload(): {
|
||||
upload: (file: File) => Promise<PendingAttachment>;
|
||||
uploading: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
Encapsulates the presigned URL flow. Returns a function that takes a `File`, gets a presigned URL, uploads via XHR with progress tracking, and returns the completed `PendingAttachment`. The component manages the array of attachments in its own state.
|
||||
|
||||
## Styling
|
||||
|
||||
All styling via inline `React.CSSProperties` objects, per project convention. Reference Catppuccin tokens through `var(--ctp-*)` strings. No CSS modules, no Tailwind, no class names.
|
||||
|
||||
Common style patterns to extract as `const` objects at the top of each file:
|
||||
|
||||
```typescript
|
||||
const styles = {
|
||||
container: {
|
||||
display: 'grid',
|
||||
gridTemplateColumns: '1fr 320px',
|
||||
height: '100%',
|
||||
overflow: 'hidden',
|
||||
} as React.CSSProperties,
|
||||
|
||||
formArea: {
|
||||
padding: '1.5rem 2rem',
|
||||
overflowY: 'auto',
|
||||
} as React.CSSProperties,
|
||||
|
||||
formGrid: {
|
||||
display: 'grid',
|
||||
gridTemplateColumns: '1fr 1fr',
|
||||
gap: '1.25rem 1.5rem',
|
||||
maxWidth: '800px',
|
||||
} as React.CSSProperties,
|
||||
|
||||
sidebar: {
|
||||
background: 'var(--ctp-mantle)',
|
||||
borderLeft: '1px solid var(--ctp-surface0)',
|
||||
display: 'flex',
|
||||
flexDirection: 'column' as const,
|
||||
overflowY: 'auto',
|
||||
} as React.CSSProperties,
|
||||
|
||||
// ... etc
|
||||
};
|
||||
```
|
||||
|
||||
## Form Sections
|
||||
|
||||
The form is visually divided by section headers. Each header is a flex row containing a label (11px uppercase, `--ctp-overlay0`) and a `flex: 1` horizontal line (`1px solid --ctp-surface0`). Sections span `grid-column: 1 / -1`.
|
||||
|
||||
| Section | Fields |
|
||||
|---------|--------|
|
||||
| Identity | Part Number*, Type*, Description, Category* |
|
||||
| Sourcing | Sourcing Type, Standard Cost, Unit of Measure, Sourcing Link |
|
||||
| Details | Long Description, Projects |
|
||||
|
||||
## Sidebar Sections
|
||||
|
||||
The right sidebar is divided into three sections with `borderBottom: 1px solid var(--ctp-surface0)`:
|
||||
|
||||
**Auto-assigned metadata**: Read-only key-value rows showing:
|
||||
- UUID: "On create" in `--ctp-teal` italic
|
||||
- Revision: "A" (hardcoded initial)
|
||||
- Created By: current user's display name from `useAuth()`
|
||||
|
||||
**Attachments**: `FileDropZone` component. Takes `flex: 1` to fill available space.
|
||||
|
||||
**Thumbnail**: A 4:3 aspect ratio placeholder box (`--ctp-crust` bg, `--ctp-surface0` border) with centered text "Generated from CAD file or upload manually". Clicking opens file picker filtered to images. If a thumbnail is uploaded, show it as an `<img>` with `object-fit: cover`.
|
||||
|
||||
## Backend Changes
|
||||
|
||||
Items 1-3 and 5 below are implemented (migration `011_item_files.sql`, `internal/api/file_handlers.go`). Item 4 (hierarchical categories) remains open.
|
||||
|
||||
### 1. Presigned Upload URL -- IMPLEMENTED
|
||||
|
||||
```
|
||||
POST /api/uploads/presign
|
||||
Request: { "filename": "bracket.FCStd", "content_type": "application/octet-stream", "size": 2400000 }
|
||||
Response: { "object_key": "uploads/tmp/{uuid}/{filename}", "upload_url": "https://minio.../...", "expires_at": "2026-02-06T..." }
|
||||
```
|
||||
|
||||
The Go handler generates a presigned PUT URL via the MinIO SDK. Objects are uploaded to a temporary prefix. On item creation, they're moved/linked to the item's permanent prefix.
|
||||
|
||||
### 2. File Association -- IMPLEMENTED
|
||||
|
||||
```
|
||||
POST /api/items/{id}/files
|
||||
Request: { "object_key": "uploads/tmp/{uuid}/bracket.FCStd", "filename": "bracket.FCStd", "content_type": "...", "size": 2400000 }
|
||||
Response: { "file_id": "uuid", "filename": "...", "size": ..., "created_at": "..." }
|
||||
```
|
||||
|
||||
Moves the object from the temp prefix to `items/{item_id}/files/{file_id}` and creates a row in a new `item_files` table.
|
||||
|
||||
### 3. Thumbnail -- IMPLEMENTED
|
||||
|
||||
```
|
||||
PUT /api/items/{id}/thumbnail
|
||||
Request: { "object_key": "uploads/tmp/{uuid}/thumb.png" }
|
||||
Response: 204
|
||||
```
|
||||
|
||||
Stores the thumbnail at `items/{item_id}/thumbnail.png` in MinIO. Updates `item.thumbnail_key` column.
|
||||
|
||||
### 4. Hierarchical Categories -- NOT IMPLEMENTED
|
||||
|
||||
If schemas don't currently support a hierarchical category tree, one of these approaches:
|
||||
|
||||
**Option A — Schema-driven**: Add a `category_tree` JSON column to the `schemas` table that defines the hierarchy. The `GET /api/schemas` response already returns schemas; the frontend transforms this into the picker tree.
|
||||
|
||||
**Option B — Dedicated table**:
|
||||
|
||||
```sql
|
||||
CREATE TABLE categories (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
name TEXT NOT NULL,
|
||||
parent_id UUID REFERENCES categories(id),
|
||||
sort_order INT NOT NULL DEFAULT 0,
|
||||
UNIQUE(parent_id, name)
|
||||
);
|
||||
```
|
||||
|
||||
With endpoints:
|
||||
```
|
||||
GET /api/categories → flat list with parent_id, frontend builds tree
|
||||
POST /api/categories → { name, parent_id? }
|
||||
PUT /api/categories/{id} → { name, sort_order }
|
||||
DELETE /api/categories/{id} → cascade check
|
||||
```
|
||||
|
||||
**Recommendation**: Option B is more flexible and keeps categories as a first-class entity. The three-tier picker doesn't need to be limited to exactly three levels — it can render as many columns as the deepest category path, but three is the practical default (Domain → Group → Subtype).
|
||||
|
||||
### 5. Database Schema Addition -- IMPLEMENTED
|
||||
|
||||
```sql
|
||||
CREATE TABLE item_files (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE,
|
||||
filename TEXT NOT NULL,
|
||||
content_type TEXT NOT NULL DEFAULT 'application/octet-stream',
|
||||
size BIGINT NOT NULL DEFAULT 0,
|
||||
object_key TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_item_files_item ON item_files(item_id);
|
||||
|
||||
ALTER TABLE items ADD COLUMN thumbnail_key TEXT;
|
||||
ALTER TABLE items ADD COLUMN category_id UUID REFERENCES categories(id);
|
||||
ALTER TABLE items ADD COLUMN sourcing_type TEXT NOT NULL DEFAULT 'manufactured';
|
||||
ALTER TABLE items ADD COLUMN sourcing_link TEXT;
|
||||
ALTER TABLE items ADD COLUMN standard_cost NUMERIC(12,2);
|
||||
ALTER TABLE items ADD COLUMN unit_of_measure TEXT NOT NULL DEFAULT 'ea';
|
||||
ALTER TABLE items ADD COLUMN long_description TEXT;
|
||||
```
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. **TagInput component** — reusable, no backend changes needed, uses existing projects API.
|
||||
2. **CategoryPicker component** — start with flat/mock data, wire to real API after backend adds categories.
|
||||
3. **FileDropZone + useFileUpload** — requires presigned URL backend endpoint first.
|
||||
4. **CreateItemPane rewrite** — compose the above into the two-column layout.
|
||||
5. **Backend: categories table + endpoints** — unblocks real category data.
|
||||
6. **Backend: presigned uploads + item_files** — unblocks file attachments.
|
||||
7. **Backend: items table migration** — adds new columns (sourcing_type, standard_cost, etc.).
|
||||
|
||||
Steps 1-2 can start immediately. Steps 5-7 can run in parallel once specified. Step 4 ties it all together.
|
||||
|
||||
## Types to Add
|
||||
|
||||
Add to `web/src/api/types.ts`:
|
||||
|
||||
```typescript
|
||||
// Categories
|
||||
interface Category {
|
||||
id: string;
|
||||
name: string;
|
||||
parent_id: string | null;
|
||||
sort_order: number;
|
||||
}
|
||||
|
||||
interface CategoryNode {
|
||||
name: string;
|
||||
id: string;
|
||||
children?: CategoryNode[];
|
||||
}
|
||||
|
||||
// File uploads
|
||||
interface PresignRequest {
|
||||
filename: string;
|
||||
content_type: string;
|
||||
size: number;
|
||||
}
|
||||
|
||||
interface PresignResponse {
|
||||
object_key: string;
|
||||
upload_url: string;
|
||||
expires_at: string;
|
||||
}
|
||||
|
||||
interface ItemFile {
|
||||
id: string;
|
||||
item_id: string;
|
||||
filename: string;
|
||||
content_type: string;
|
||||
size: number;
|
||||
object_key: string;
|
||||
created_at: string;
|
||||
}
|
||||
|
||||
// Extended create request
|
||||
interface CreateItemRequest {
|
||||
part_number: string;
|
||||
item_type: 'part' | 'assembly' | 'document';
|
||||
description?: string;
|
||||
category_id?: string;
|
||||
sourcing_type?: 'manufactured' | 'purchased' | 'phantom';
|
||||
standard_cost?: number;
|
||||
unit_of_measure?: string;
|
||||
sourcing_link?: string;
|
||||
long_description?: string;
|
||||
project_ids?: string[];
|
||||
}
|
||||
|
||||
// Pending upload (frontend only, not an API type)
|
||||
interface PendingAttachment {
|
||||
file: File;
|
||||
objectKey: string;
|
||||
uploadProgress: number;
|
||||
uploadStatus: 'pending' | 'uploading' | 'complete' | 'error';
|
||||
error?: string;
|
||||
}
|
||||
```
|
||||
122
docs/src/silo-server/overview.md
Normal file
122
docs/src/silo-server/overview.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# Kindred Silo
|
||||
|
||||
Item database and part management system.
|
||||
|
||||
## Overview
|
||||
|
||||
Kindred Silo is an R&D-oriented item database with:
|
||||
|
||||
- **Configurable part number generation** via YAML schemas
|
||||
- **Revision tracking** with append-only history, rollback, comparison, and status labels
|
||||
- **BOM management** with multi-level expansion, flat BOM flattening, assembly costing, where-used queries, CSV/ODS export
|
||||
- **Authentication** with local (bcrypt), LDAP/FreeIPA, and OIDC/Keycloak backends
|
||||
- **Role-based access control** (admin > editor > viewer) with API tokens and sessions
|
||||
- **ODS import/export** for items, BOMs, and project sheets
|
||||
- **Audit/completeness scoring** with weighted per-category property validation
|
||||
- **Web UI** — React SPA (Vite + TypeScript, Catppuccin Mocha theme) for item browsing, project management, schema editing, and audit
|
||||
- **CAD integration** via REST API ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc))
|
||||
- **Physical inventory** tracking with hierarchical locations (schema ready)
|
||||
|
||||
## Components
|
||||
|
||||
```
|
||||
silo/
|
||||
├── cmd/
|
||||
│ ├── silo/ # CLI tool
|
||||
│ └── silod/ # API server
|
||||
├── internal/
|
||||
│ ├── api/ # HTTP handlers and routes (75 endpoints)
|
||||
│ ├── auth/ # Authentication (local, LDAP, OIDC)
|
||||
│ ├── config/ # Configuration loading
|
||||
│ ├── db/ # PostgreSQL repositories
|
||||
│ ├── migration/ # Property migration utilities
|
||||
│ ├── odoo/ # Odoo ERP integration
|
||||
│ ├── ods/ # ODS spreadsheet library
|
||||
│ ├── partnum/ # Part number generation
|
||||
│ ├── schema/ # YAML schema parsing
|
||||
│ ├── storage/ # MinIO file storage
|
||||
│ └── testutil/ # Test helpers
|
||||
├── web/ # React SPA (Vite + TypeScript)
|
||||
│ └── src/
|
||||
│ ├── api/ # API client and type definitions
|
||||
│ ├── components/ # Reusable UI components
|
||||
│ ├── context/ # Auth context provider
|
||||
│ ├── hooks/ # Custom React hooks
|
||||
│ ├── pages/ # Page components (Items, Projects, Schemas, Settings, Audit, Login)
|
||||
│ └── styles/ # Catppuccin Mocha theme and global styles
|
||||
├── migrations/ # Database migrations (11 files)
|
||||
├── schemas/ # Part numbering schemas (YAML)
|
||||
├── deployments/ # Docker Compose and systemd configs
|
||||
├── scripts/ # Deployment and setup scripts
|
||||
└── docs/ # Documentation
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Docker Compose (quickest)
|
||||
cp config.example.yaml config.yaml
|
||||
# Edit config.yaml with your database, MinIO, and auth settings
|
||||
make docker-up
|
||||
|
||||
# Or manual setup
|
||||
psql -h localhost -U silo -d silo -f migrations/*.sql
|
||||
go run ./cmd/silod -config config.yaml
|
||||
```
|
||||
|
||||
When auth is enabled, a default admin account is created on first startup using the credentials in `config.yaml` under `auth.local.default_admin_username` and `auth.local.default_admin_password`.
|
||||
|
||||
```bash
|
||||
# CLI usage
|
||||
go run ./cmd/silo register --schema kindred-rd --category F01
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
See `config.example.yaml` for all options.
|
||||
|
||||
## Authentication
|
||||
|
||||
Silo supports three authentication backends, configured in `config.yaml`:
|
||||
|
||||
| Backend | Description |
|
||||
|---------|-------------|
|
||||
| **Local** | Built-in accounts with bcrypt passwords |
|
||||
| **LDAP** | FreeIPA / Active Directory integration |
|
||||
| **OIDC** | Keycloak / OpenID Connect providers |
|
||||
|
||||
Roles: **admin** (full access) > **editor** (create/modify items) > **viewer** (read-only).
|
||||
|
||||
API tokens provide programmatic access for scripts and CAD clients. Set `auth.enabled: false` for development without authentication.
|
||||
|
||||
See [docs/AUTH.md](docs/AUTH.md) for full details.
|
||||
|
||||
## Client Integrations
|
||||
|
||||
CAD and spreadsheet integrations are maintained in separate repositories:
|
||||
|
||||
- **Kindred Create / FreeCAD workbench** -- [silo-mod](https://git.kindred-systems.com/kindred/silo-mod)
|
||||
- **LibreOffice Calc extension** -- [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)
|
||||
|
||||
The server provides the REST API and ODS endpoints consumed by these clients.
|
||||
|
||||
## Documentation
|
||||
|
||||
| Document | Description |
|
||||
|----------|-------------|
|
||||
| [docs/SPECIFICATION.md](docs/SPECIFICATION.md) | Full design specification and API reference |
|
||||
| [docs/STATUS.md](docs/STATUS.md) | Implementation status |
|
||||
| [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) | Production deployment guide |
|
||||
| [docs/CONFIGURATION.md](docs/CONFIGURATION.md) | Configuration reference (all `config.yaml` options) |
|
||||
| [docs/AUTH.md](docs/AUTH.md) | Authentication system design |
|
||||
| [docs/AUTH_USER_GUIDE.md](docs/AUTH_USER_GUIDE.md) | User guide for login, tokens, and roles |
|
||||
| [docs/GAP_ANALYSIS.md](docs/GAP_ANALYSIS.md) | Gap analysis and revision control roadmap |
|
||||
| [docs/COMPONENT_AUDIT.md](docs/COMPONENT_AUDIT.md) | Component audit tool design |
|
||||
| [ROADMAP.md](ROADMAP.md) | Feature roadmap and SOLIDWORKS PDM comparison |
|
||||
| [frontend-spec.md](frontend-spec.md) | React SPA frontend specification |
|
||||
|
||||
## License
|
||||
|
||||
MIT License - Copyright (c) 2026 Kindred Systems LLC
|
||||
|
||||
See [LICENSE](LICENSE) for details.
|
||||
Reference in New Issue
Block a user