kindred/silo
1
1
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update stale documentation to reflect current state

Browse Source 
Closes #1 — Bring documentation in line with implemented features.

GAP_ANALYSIS.md:
- Mark auth system and audit log gaps as Implemented
- Replace FreeCAD Integration section with Client Integration (silo-mod)
- Update Phase 2 sections: auth and audit marked COMPLETE
- Update Appendix A file structure and Appendix B endpoints

STATUS.md:
- Update client integrations to reference silo-mod and silo-calc repos
- Update unit tests row to remove pkg/calc/tests reference

ROADMAP.md:
- Update executive summary with links to silo-mod and silo-calc
- Update unit tests row, CAD gap section references

SPECIFICATION.md:
- Update architecture overview to reference silo-mod and silo-calc
- Update Section 5 Client Integration with both repos

REPOSITORY_STATUS.md:
- Remove Python/FreeCAD row from language stats, update totals
This commit is contained in:
Forbes
2026-02-06 16:18:58 -06:00
parent 31586755b7
commit 004dc9aef0
5 changed files with 356 additions and 310 deletions
Download Patch File Download Diff File Expand all files Collapse all files

289
docs/SPECIFICATION.md
View File

@@ -1,14 +1,14 @@
# Silo: Item Database and Part Management System for FreeCAD
# Silo: Item Database and Part Management System
**Version:** 0.1 Draft
**Date:** January 2026
**Version:** 0.2
**Date:** February 2026
**Author:** Kindred Systems LLC
---
## 1. Overview
Silo is an item database with configurable part number generation, designed for R&D-oriented workflows. It integrates with FreeCAD 1.0+ to provide git-like object management, revision tracking, and physical inventory location management.
Silo is an item database with configurable part number generation, designed for R&D-oriented workflows. It provides revision tracking, BOM management, file versioning, and physical inventory location management through a REST API and web UI. CAD integration (FreeCAD workbench, LibreOffice Calc extension) is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)).
### 1.1 Core Philosophy
@@ -29,53 +29,45 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb
```
┌─────────────────────────────────────────────────────────────┐
│ FreeCAD 1.0+ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Silo Workbench (Python) │ │
│ │ - silo checkout / commit / status / log │ │
│ │ - Part number generation │ │
│ │ - Property sync with FreeCAD objects │ │
│ └─────────────────────────────────────────────────────┘ │
│ CAD Clients (silo-mod, silo-calc) │
│ FreeCAD Workbench · LibreOffice Calc Extension │
│ (maintained in separate repositories) │
└─────────────────────────────────────────────────────────────┘
│
│ REST API
▼
┌─────────────────────────────────────────────────────────────┐
│ Silo Core (CLI/Library) │
│ Silo Server (silod) │
│ - REST API (74 endpoints) │
│ - Authentication (local, LDAP, OIDC) │
│ - Schema parsing and validation │
│ - Part number generation engine │
│ - Revision management │
│ - Relationship graph │
│ - Relationship graph / BOM │
│ - Web UI (htmx) │
└─────────────────────────────────────────────────────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────────┐
│ PostgreSQL │ │ MinIO │
│ (psql.kindred.internal)│ │ - .FCStd file storage │
│ (psql.kindred.internal)│ │ - File storage │
│ - Item metadata │ │ - Versioned objects │
│ - Relationships │ │ - Thumbnails │
│ - Revision history │ │ │
│ - Location hierarchy │ │ │
│ - Auth / Sessions │ │ │
│ - Audit log │ │ │
└─────────────────────────┘ └─────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Web UI (Browse/Search) │
│ - Item browser with hierarchy navigation │
│ - Search and filtering │
│ - "Open in FreeCAD" links (freecad:// URI handler) │
│ - BOM viewer │
└─────────────────────────────────────────────────────────────┘
```
### 2.2 Technology Stack
| Component | Technology | Notes |
|-----------|------------|-------|
| Database | PostgreSQL | Existing instance at psql.kindred.internal |
| Database | PostgreSQL 16 | Existing instance at psql.kindred.internal |
| File Storage | MinIO | S3-compatible, versioning enabled |
| FreeCAD Integration | Python workbench | Macro-style commands |
| CLI & API Server | Go (1.23) | chi/v5 router, pgx/v5 driver, zerolog |
| CLI & API Server | Go (1.24) | chi/v5 router, pgx/v5 driver, zerolog |
| Authentication | Multi-backend | Local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak |
| Sessions | PostgreSQL pgxstore | alexedwards/scs, 24h lifetime |
| Web UI | Go html/template + htmx | Lightweight, minimal JS |
---
@@ -347,52 +339,24 @@ assembly_config:
---
## 5. FreeCAD Integration
## 5. Client Integration
### 5.1 Workbench Commands
CAD workbench and spreadsheet extension implementations are maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The Silo server provides the REST API endpoints consumed by those clients.
The Silo workbench provides toolbar commands in FreeCAD:
### 5.1 File Storage Strategy
| Command | Description | Status |
|---------|-------------|--------|
| `Silo_Save` | Auto-save document and upload to MinIO | Implemented |
| `Silo_Commit` | Save with revision comment | Implemented |
| `Silo_Pull` | Download item by part number / create new | Implemented |
| `Silo_Push` | Batch upload modified files | Implemented |
| `Silo_Info` | View revision history for current item | Implemented |
| `Silo_Register` | Generate part number for current document | Implemented |
| `Silo_Open` | Open item from Silo by part number | Implemented |
| `Silo_Browse` | Browse items in a list dialog | Implemented |
Files are stored as whole objects in MinIO with versioning enabled. Storage path convention: `items/{partNumber}/rev{N}.ext`. SHA-256 checksums are captured on upload for integrity verification.
### 5.2 Property Synchronization
Future option: exploded storage (unpack ZIP-based CAD archives for better diffing).
Silo properties map to FreeCAD custom properties:
### 5.2 Checkout Locking (Future)
```python
# FreeCAD object properties (synced from Silo)
obj.addProperty("App::PropertyString", "SiloPartNumber", "Silo", "Part number")
obj.addProperty("App::PropertyString", "SiloRevision", "Silo", "Current revision")
obj.addProperty("App::PropertyString", "SiloDescription", "Silo", "Item description")
# ... additional properties as defined in schema
```
### 5.3 File Storage Strategy
FreeCAD `.FCStd` files are ZIP archives. Storage options:
1. **Whole file storage** (MVP): Store complete .FCStd in MinIO with versioning
2. **Exploded storage** (future): Unpack and store components separately for better diffing
For MVP, whole file storage is simpler and MinIO versioning handles history.
### 5.4 Checkout Locking (Future)
MVP operates as single-user. Future multi-user support will need locking strategy:
Future multi-user support will need a server-side locking strategy:
- **Pessimistic locking**: Checkout acquires exclusive lock
- **Optimistic locking**: Allow concurrent edits, handle conflicts on commit
Recommendation for future: Pessimistic locking for CAD files (merge is impractical).
Recommendation: Pessimistic locking for CAD files (merge is impractical).
---
@@ -533,34 +497,40 @@ All operations logged for audit trail (future consideration).
---
## 10. Authentication (Future)
## 10. Authentication
### 10.1 Current State (MVP)
Silo supports three authentication backends that can be enabled independently or combined. When authentication is disabled (`auth.enabled: false`), all routes are open and a synthetic dev user with the `admin` role is injected into every request.
Single-user, no authentication required.
### 10.1 Backends
### 10.2 Future: LDAPS Integration
| Backend | Use Case | Config Key |
|---------|----------|------------|
| **Local** | Username/password stored in database (bcrypt cost 12) | `auth.local` |
| **LDAP** | FreeIPA / Active Directory via LDAP bind | `auth.ldap` |
| **OIDC** | Keycloak or any OpenID Connect provider (redirect flow) | `auth.oidc` |
Plan for FreeIPA integration:
### 10.2 Role Model
```yaml
# /etc/silo/auth.yaml
auth:
provider: ldap
server: ldaps://ipa.kindred.internal
base_dn: "dc=kindred,dc=internal"
user_dn_template: "uid={username},cn=users,cn=accounts,dc=kindred,dc=internal"
group_base: "cn=groups,cn=accounts,dc=kindred,dc=internal"
# Role mapping
roles:
admin:
groups: ["silo-admins"]
editor:
groups: ["silo-users", "engineers"]
viewer:
groups: ["silo-viewers"]
```
Three roles with a strict hierarchy: `admin > editor > viewer`
| Permission | viewer | editor | admin |
|-----------|--------|--------|-------|
| Read items, projects, schemas, BOMs | Yes | Yes | Yes |
| Create/update items and revisions | No | Yes | Yes |
| Upload files, manage BOMs | No | Yes | Yes |
| Import CSV/ODS | No | Yes | Yes |
| Manage own API tokens | Yes | Yes | Yes |
| User management (future) | No | No | Yes |
### 10.3 API Tokens
Raw token format: `silo_` + 64 hex characters (32 random bytes from `crypto/rand`). Only the SHA-256 hash is stored in the database. Tokens inherit the owning user's role.
### 10.4 Sessions
PostgreSQL-backed sessions via `alexedwards/scs` pgxstore. Cookie: `silo_session`, HttpOnly, SameSite=Lax, 24h lifetime. `Secure` flag is set when `auth.enabled` is true.
See [AUTH.md](AUTH.md) for full architecture details and [AUTH_USER_GUIDE.md](AUTH_USER_GUIDE.md) for setup instructions.
---
@@ -569,60 +539,111 @@ auth:
### 11.1 REST Endpoints (Implemented)
```
# Health
# Health (no auth)
GET /health # Basic health check
GET /ready # Readiness (DB + MinIO)
# Web UI
GET / # Items page
GET /schemas # Schemas page
# Auth (no auth required)
GET /login # Login page
POST /login # Login form handler
POST /logout # Logout
GET /auth/oidc # OIDC login redirect
GET /auth/callback # OIDC callback
# Schemas
# Web UI (auth + CSRF)
GET / # Items page
GET /projects # Projects page
GET /schemas # Schemas page
GET /audit # Audit/completeness page
GET /settings # User settings / token management
POST /settings/tokens # Create API token (web)
POST /settings/tokens/{id}/revoke # Revoke API token (web)
# Auth API
GET /api/auth/me # Current authenticated user
GET /api/auth/tokens # List user's API tokens
POST /api/auth/tokens # Create API token
DELETE /api/auth/tokens/{id} # Revoke API token
# Schemas (read: viewer, write: editor)
GET /api/schemas # List all schemas
GET /api/schemas/{name} # Get schema details
GET /api/schemas/{name}/properties # Get property schema for category
POST /api/schemas/{name}/segments/{segment}/values # Add enum value
PUT /api/schemas/{name}/segments/{segment}/values/{code} # Update enum value
DELETE /api/schemas/{name}/segments/{segment}/values/{code} # Delete enum value
POST /api/schemas/{name}/segments/{segment}/values # Add enum value [editor]
PUT /api/schemas/{name}/segments/{segment}/values/{code} # Update enum value [editor]
DELETE /api/schemas/{name}/segments/{segment}/values/{code} # Delete enum value [editor]
# Projects
# Projects (read: viewer, write: editor)
GET /api/projects # List projects
POST /api/projects # Create project
GET /api/projects/{code} # Get project
PUT /api/projects/{code} # Update project
DELETE /api/projects/{code} # Delete project
GET /api/projects/{code}/items # Get project items
GET /api/projects/{code}/sheet.ods # Export project sheet as ODS
POST /api/projects # Create project [editor]
PUT /api/projects/{code} # Update project [editor]
DELETE /api/projects/{code} # Delete project [editor]
# Items
GET /api/items # List/search items
POST /api/items # Create item
# Items (read: viewer, write: editor)
GET /api/items # List/filter items
GET /api/items/search # Fuzzy search
GET /api/items/export.csv # Export items to CSV
POST /api/items/import # Import items from CSV
GET /api/items/template.csv # Get CSV import template
GET /api/items/template.csv # CSV import template
GET /api/items/export.ods # Export items to ODS
GET /api/items/template.ods # ODS import template
POST /api/items # Create item [editor]
POST /api/items/import # Import items from CSV [editor]
POST /api/items/import.ods # Import items from ODS [editor]
# Item Detail
GET /api/items/{partNumber} # Get item details
PUT /api/items/{partNumber} # Update item
DELETE /api/items/{partNumber} # Archive item
PUT /api/items/{partNumber} # Update item [editor]
DELETE /api/items/{partNumber} # Archive item [editor]
# Item-Project Tags
GET /api/items/{partNumber}/projects # Get item's projects
POST /api/items/{partNumber}/projects # Add project tags
DELETE /api/items/{partNumber}/projects/{code} # Remove project tag
POST /api/items/{partNumber}/projects # Add project tags [editor]
DELETE /api/items/{partNumber}/projects/{code} # Remove project tag [editor]
# Revisions
GET /api/items/{partNumber}/revisions # List revisions
POST /api/items/{partNumber}/revisions # Create revision
GET /api/items/{partNumber}/revisions/compare # Compare two revisions
GET /api/items/{partNumber}/revisions/{revision} # Get specific revision
PATCH /api/items/{partNumber}/revisions/{revision} # Update status/labels
POST /api/items/{partNumber}/revisions/{revision}/rollback # Rollback to revision
POST /api/items/{partNumber}/revisions # Create revision [editor]
PATCH /api/items/{partNumber}/revisions/{revision} # Update status/labels [editor]
POST /api/items/{partNumber}/revisions/{revision}/rollback # Rollback to revision [editor]
# Files
POST /api/items/{partNumber}/file # Upload file
GET /api/items/{partNumber}/file # Download latest file
GET /api/items/{partNumber}/file/{revision} # Download file at revision
POST /api/items/{partNumber}/file # Upload file [editor]
# Part Number Generation
POST /api/generate-part-number # Generate without creating item
# BOM
GET /api/items/{partNumber}/bom # List direct children
GET /api/items/{partNumber}/bom/expanded # Multi-level BOM (recursive)
GET /api/items/{partNumber}/bom/where-used # Where-used (parent lookup)
GET /api/items/{partNumber}/bom/export.csv # Export BOM as CSV
GET /api/items/{partNumber}/bom/export.ods # Export BOM as ODS
POST /api/items/{partNumber}/bom # Add BOM entry [editor]
POST /api/items/{partNumber}/bom/import # Import BOM from CSV [editor]
PUT /api/items/{partNumber}/bom/{childPartNumber} # Update BOM entry [editor]
DELETE /api/items/{partNumber}/bom/{childPartNumber} # Remove BOM entry [editor]
# Audit (viewer)
GET /api/audit/completeness # Item completeness scores
GET /api/audit/completeness/{partNumber} # Item detail breakdown
# Integrations — Odoo (read: viewer, write: editor)
GET /api/integrations/odoo/config # Get Odoo configuration
GET /api/integrations/odoo/sync-log # Get sync history
PUT /api/integrations/odoo/config # Update Odoo config [editor]
POST /api/integrations/odoo/test-connection # Test connection [editor] (stub)
POST /api/integrations/odoo/sync/push/{partNumber} # Push to Odoo [editor] (stub)
POST /api/integrations/odoo/sync/pull/{odooId} # Pull from Odoo [editor] (stub)
# Sheets (editor)
POST /api/sheets/diff # Diff ODS sheet against DB [editor]
# Part Number Generation (editor)
POST /api/generate-part-number # Generate without creating item [editor]
```
### 11.2 Not Yet Implemented
@@ -630,14 +651,16 @@ POST /api/generate-part-number # Generate without c
The following endpoints from the original design are not yet implemented:
```
# Locations (tables exist, no API)
# Locations (tables exist, no API handlers)
GET /api/locations
POST /api/locations
GET /api/locations/{path}
DELETE /api/locations/{path}
# Inventory (tables exist, no API)
# Inventory (tables exist, no API handlers)
GET /api/inventory/{partNumber}
POST /api/inventory/{partNumber}/adjust
POST /api/inventory/{partNumber}/move
```
---
@@ -646,20 +669,29 @@ POST /api/inventory/{partNumber}/adjust
### 12.1 Implemented
- [x] PostgreSQL database schema (7 migrations)
- [x] PostgreSQL database schema (10 migrations)
- [x] YAML schema parser for part numbering
- [x] Part number generation engine
- [x] CLI tool (`cmd/silo`)
- [x] API server (`cmd/silod`) with 35+ endpoints
- [x] FreeCAD workbench (save, commit, pull, push, info, register, open, browse)
- [x] API server (`cmd/silod`) with 74 endpoints
- [x] MinIO integration for file storage with versioning
- [x] BOM relationships (component, alternate, reference)
- [x] Multi-level BOM (recursive expansion with configurable depth)
- [x] Where-used queries (reverse parent lookup)
- [x] BOM CSV and ODS export/import
- [x] Reference designator tracking
- [x] Revision history (append-only) with rollback and comparison
- [x] Revision status and labels
- [x] Project management with many-to-many item tagging
- [x] CSV import/export with dry-run validation
- [x] Web UI for items and schemas (htmx)
- [x] ODS spreadsheet import/export (items, BOMs, project sheets)
- [x] Web UI for items, projects, schemas, audit (htmx)
- [x] Authentication (local, LDAP, OIDC) with role-based access control
- [x] API token management (SHA-256 hashed)
- [x] Session management (PostgreSQL-backed)
- [x] Audit logging and completeness scoring
- [x] CSRF protection (nosurf)
- [x] Fuzzy search
- [x] Property schema versioning framework
- [x] Docker Compose deployment (dev and prod)
- [x] systemd service and deployment scripts
@@ -670,17 +702,15 @@ POST /api/inventory/{partNumber}/adjust
- [ ] Inventory tracking (database tables exist, no API endpoints)
- [ ] Date segment type (schema parser placeholder only)
- [ ] Part number format validation on creation
- [ ] Odoo ERP integration (config and sync-log functional; push/pull are stubs)
### 12.3 Not Started
- [ ] Unit tests
- [ ] Unit tests (Go server — minimal coverage exists)
- [ ] Schema migration tooling
- [ ] Multi-user authentication (FreeIPA/LDAP planned)
- [ ] Checkout locking
- [ ] Approval workflows
- [ ] External system integrations (ERP, purchasing)
- [ ] Exploded file storage with diffing
- [ ] Audit logging
- [ ] Notifications
- [ ] Reporting/analytics
@@ -688,15 +718,11 @@ POST /api/inventory/{partNumber}/adjust
## 13. Open Questions
1. ~~**CLI language**: Go for consistency with web UI, or Python for FreeCAD ecosystem alignment?~~ **Resolved:** Go was chosen for both CLI and API server.
1. **Thumbnail generation**: Generate thumbnails from CAD files on commit? Useful for web UI browsing.
2. **Property schema**: Should item properties be schema-defined (like part numbers) or freeform? Recommendation: Support both—schema defines expected properties, but allow ad-hoc additions.
2. **Search indexing**: PostgreSQL full-text search sufficient, or add dedicated search (Meilisearch, etc.)?
3. **Thumbnail generation**: Generate thumbnails from .FCStd on commit? Useful for web UI browsing.
4. **Search indexing**: PostgreSQL full-text search sufficient, or add dedicated search (Meilisearch, etc.)?
5. **Offline operation**: Should FreeCAD workbench support offline mode with sync? Adds significant complexity.
3. **Checkout locking**: Pessimistic vs optimistic locking strategy for multi-user CAD file editing.
---
@@ -706,7 +732,6 @@ POST /api/inventory/{partNumber}/adjust
- **CycloneDX BOM specification**: JSON/YAML schema patterns for component identification, relationships, and metadata (https://cyclonedx.org)
- **OpenBOM data model**: Reference-instance separation, flexible property schemas
- **FreeCAD DynamicData workbench**: Custom property patterns in FreeCAD
- **Ansible inventory YAML**: Hierarchical configuration patterns with variable inheritance
### 14.2 Related Standards