# Silo Gap Analysis **Date:** 2026-02-13 **Status:** Analysis Complete (Updated) --- ## Executive Summary This document analyzes the current state of the Silo project against its specification and against SOLIDWORKS PDM (the industry-leading product data management solution). It identifies documentation gaps, feature gaps, and outlines a roadmap for enhanced revision control capabilities. See [ROADMAP.md](ROADMAP.md) for the platform roadmap and dependency tier structure. --- ## 1. Documentation Gap Analysis ### 1.1 Current Documentation | Document | Coverage | Status | |----------|----------|--------| | `README.md` | Quick start, overview, component map | Current | | `docs/SPECIFICATION.md` | Design specification, API reference | Current | | `docs/STATUS.md` | Implementation status summary | Current | | `docs/DEPLOYMENT.md` | Production deployment guide | Current | | `docs/CONFIGURATION.md` | Configuration reference (all config.yaml options) | Current | | `docs/AUTH.md` | Authentication system design | Current | | `docs/AUTH_USER_GUIDE.md` | User guide for login, tokens, and roles | Current | | `docs/GAP_ANALYSIS.md` | Revision control roadmap | Current | | `docs/ROADMAP.md` | Platform roadmap and dependency tiers | Current | | `frontend-spec.md` | React SPA frontend specification | Current | ### 1.2 Documentation Gaps (Priority Order) #### High Priority | Gap | Impact | Effort | Status | |-----|--------|--------|--------| | **API Reference** | Users cannot integrate programmatically | Medium | Addressed in SPECIFICATION.md Section 11 | | **Deployment Guide** | Cannot deploy to production | Medium | Complete (`docs/DEPLOYMENT.md`) | | **Database Schema Guide** | Migration troubleshooting difficult | Low | Open | | **Configuration Reference** | config.yaml options undocumented | Low | Complete (`docs/CONFIGURATION.md`) | #### Medium Priority | Gap | Impact | Effort | |-----|--------|--------| | **User Workflows** | Users lack step-by-step guidance | Medium | | **Troubleshooting Guide** | Support burden increases | Medium | | **Developer Setup Guide** | Onboarding friction | Low | #### Lower Priority | Gap | Impact | Effort | |-----|--------|--------| | **CHANGELOG.md** | Version history unclear | Low | | **Architecture Decision Records** | Design rationale lost | Medium | | **Integration Guide** | Third-party integration unclear | High | ### 1.3 Recommended Actions 1. ~~**Consolidate specs**: Remove `silo-spec.md` duplicate~~ Done (deleted) 2. ~~**Create API reference**: Full REST endpoint documentation~~ Addressed in SPECIFICATION.md 3. ~~**Create `docs/DEPLOYMENT.md`**: Production deployment guide~~ Done 4. ~~**Create configuration reference**: Document all `config.yaml` options~~ Done (`docs/CONFIGURATION.md`) 5. **Create database schema guide**: Document migrations and troubleshooting --- ## 2. Current Revision Control Implementation ### 2.1 What's Implemented | Feature | Status | Location | |---------|--------|----------| | Append-only revision history | Complete | `internal/db/items.go` | | Sequential revision numbering | Complete | Database trigger | | Property snapshots (JSONB) | Complete | `revisions.properties` | | File versioning (MinIO) | Complete | `internal/storage/` | | SHA256 checksums | Complete | Captured on upload | | Revision comments | Complete | `revisions.comment` | | User attribution | Complete | `revisions.created_by` | | Property schema versioning | Complete | Migration 005 | | BOM revision pinning | Complete | `relationships.child_revision` | ### 2.2 Database Schema ```sql -- Current revision schema (migrations/001_initial.sql) CREATE TABLE revisions ( id UUID PRIMARY KEY, item_id UUID REFERENCES items(id) ON DELETE CASCADE, revision_number INTEGER NOT NULL, properties JSONB NOT NULL DEFAULT '{}', file_key TEXT, file_version TEXT, -- MinIO version ID file_checksum TEXT, -- SHA256 file_size BIGINT, thumbnail_key TEXT, created_at TIMESTAMPTZ DEFAULT now(), created_by TEXT, comment TEXT, property_schema_version INTEGER DEFAULT 1, UNIQUE(item_id, revision_number) ); ``` ### 2.3 API Endpoints | Endpoint | Method | Status | |----------|--------|--------| | `/api/items/{pn}/revisions` | GET | Implemented | | `/api/items/{pn}/revisions` | POST | Implemented | | `/api/items/{pn}/revisions/{rev}` | GET | Implemented | | `/api/items/{pn}/file` | POST | Implemented | | `/api/items/{pn}/file` | GET | Implemented (latest) | | `/api/items/{pn}/file/{rev}` | GET | Implemented | ### 2.4 Client Integration FreeCAD workbench maintained in separate [silo-mod](https://git.kindred-systems.com/kindred/silo-mod) repository. The server provides the revision and file endpoints consumed by the workbench. --- ## 3. Revision Control Gaps ### 3.1 Critical Gaps | Gap | Description | Impact | Status | |-----|-------------|--------|--------| | ~~**No rollback**~~ | ~~Cannot revert to previous revision~~ | ~~Data recovery difficult~~ | **Implemented** | | ~~**No comparison**~~ | ~~Cannot diff between revisions~~ | ~~Change tracking manual~~ | **Implemented** | | **No locking** | No concurrent edit protection | Multi-user unsafe | Open | | **No approval workflow** | No release/sign-off process | Quality control gap | Open | ### 3.2 Important Gaps | Gap | Description | Impact | Status | |-----|-------------|--------|--------| | **No branching** | Linear history only | No experimental variants | Open | | ~~**No tagging**~~ | ~~No named milestones~~ | ~~Release tracking manual~~ | **Implemented** (revision labels) | | ~~**No audit log**~~ | ~~Actions not logged separately~~ | ~~Compliance gap~~ | **Implemented** (migration 009, `audit_log` table + completeness scoring) | | **Thumbnail missing** | Schema exists, not populated | No visual preview | Open | ### 3.3 Nice-to-Have Gaps | Gap | Description | Impact | |-----|-------------|--------| | **No search** | Cannot search revision comments | Discovery limited | | **No retention policy** | Revisions never expire | Storage grows unbounded | | **No delta storage** | Full file per revision | Storage inefficient | | **No notifications** | No change alerts | Manual monitoring required | --- ## 4. Revision Control Roadmap ### Phase 1: Foundation -- COMPLETE **Goal:** Enable safe single-user revision management All Phase 1 items have been implemented: - **Rollback**: `POST /api/items/{pn}/revisions/{rev}/rollback` - creates new revision from old - **Revision Comparison**: `GET /api/items/{pn}/revisions/compare?from={rev1}&to={rev2}` - property and file diffs - **Revision Labels/Status**: `PATCH /api/items/{pn}/revisions/{rev}` - status (draft/review/released/obsolete) and arbitrary labels via migration 007 --- ### Phase 2: Multi-User Support **Goal:** Enable safe concurrent editing #### 2.1 Pessimistic Locking ``` Effort: High | Priority: High | Risk: Medium | Status: Not Started ``` **Database Migration:** ```sql CREATE TABLE item_locks ( id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE, locked_by TEXT NOT NULL, locked_at TIMESTAMPTZ NOT NULL DEFAULT now(), expires_at TIMESTAMPTZ NOT NULL, lock_type TEXT NOT NULL DEFAULT 'exclusive', comment TEXT, UNIQUE(item_id) ); CREATE INDEX idx_locks_expires ON item_locks(expires_at); ``` **API Endpoints:** ``` POST /api/items/{pn}/lock # Acquire lock DELETE /api/items/{pn}/lock # Release lock GET /api/items/{pn}/lock # Check lock status ``` #### 2.2 Authentication -- COMPLETE Authentication is fully implemented with three backends (local/bcrypt, LDAP/FreeIPA, OIDC/Keycloak), role-based access control (admin > editor > viewer), API token management, and PostgreSQL-backed sessions. See `docs/AUTH.md` for full details. - Migration: `009_auth.sql` - Code: `internal/auth/`, `internal/api/middleware.go` #### 2.3 Audit Logging -- COMPLETE Audit logging is implemented via migration 009 with the `audit_log` table and completeness scoring system. Endpoints: - `GET /api/audit/completeness` — summary of all items - `GET /api/audit/completeness/{partNumber}` — per-item scoring with weighted fields and tier classification Code: `internal/api/audit_handlers.go` --- ### Phase 3: Advanced Features **Goal:** Enhanced revision management capabilities #### 3.1 Branching Support ``` Effort: High | Priority: Low | Risk: High ``` **Concept:** Named revision streams per item **Database Migration:** ```sql CREATE TABLE revision_branches ( id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), item_id UUID NOT NULL REFERENCES items(id) ON DELETE CASCADE, name TEXT NOT NULL, -- 'main', 'experimental', 'customer-variant' base_revision INTEGER, -- Branch point created_at TIMESTAMPTZ DEFAULT now(), created_by TEXT, UNIQUE(item_id, name) ); ALTER TABLE revisions ADD COLUMN branch_id UUID REFERENCES revision_branches(id); ``` **Complexity:** Requires merge logic, conflict resolution UI #### 3.2 Release Management ``` Effort: Medium | Priority: Medium | Risk: Low ``` **Database Migration:** ```sql CREATE TABLE releases ( id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), name TEXT NOT NULL UNIQUE, -- 'v1.0', '2026-Q1' description TEXT, created_at TIMESTAMPTZ DEFAULT now(), created_by TEXT, status TEXT DEFAULT 'draft' -- 'draft', 'released', 'archived' ); CREATE TABLE release_items ( id UUID PRIMARY KEY DEFAULT uuid_generate_v4(), release_id UUID REFERENCES releases(id) ON DELETE CASCADE, item_id UUID REFERENCES items(id) ON DELETE CASCADE, revision_number INTEGER NOT NULL, UNIQUE(release_id, item_id) ); ``` **Use Case:** Snapshot a set of items at specific revisions for a product release #### 3.3 Thumbnail Generation ``` Effort: Medium | Priority: Low | Risk: Low ``` **Implementation Options:** 1. FreeCAD headless rendering (python script) 2. External service (e.g., CAD thumbnail microservice) 3. User-uploaded thumbnails **Changes:** - Add thumbnail generation on file upload - Store in MinIO at `thumbnails/{part_number}/rev{n}.png` - Expose via `GET /api/items/{pn}/thumbnail/{rev}` --- ## 5. Recommended Implementation Order ### Completed 1. ~~**Revision Comparison API**~~ - Implemented 2. ~~**Rollback Support**~~ - Implemented 3. ~~**Revision Labels/Status**~~ - Implemented (migration 007) ### Recently Completed 4. ~~**Authentication**~~ - Implemented (3 backends: local, LDAP, OIDC; RBAC; API tokens; sessions) 5. ~~**Audit Logging**~~ - Implemented (audit_log table, completeness scoring) ### Next (Short-term) 6. **Pessimistic Locking** - Required before multi-user ### Medium-term (3-6 Months) 7. **Release Management** - Product milestone tracking 8. **Thumbnail Generation** - Visual preview capability ### Long-term (Future) 10. **Branching** - Complex, defer until needed 11. **Delta Storage** - Optimization, not critical 12. **Notifications** - Nice-to-have workflow enhancement --- ## 6. Migration Considerations ### Part Number Format Migration (Completed) The recent migration from `XXXXX-CCC-NNNN` to `CCC-NNNN` format has been completed: - Database migration: `migrations/006_project_tags.sql` - Schema update: `schemas/kindred-rd.yaml` v3 - Projects: Now many-to-many tags instead of embedded in part number - File paths: `~/projects/cad/{category}_{name}/{part_number}_{description}.FCStd` ### Future Schema Migrations The property schema versioning framework (`property_schema_version`, `property_migrations` table) is in place but lacks: - Automated migration runners - Rollback capability for failed migrations - Dry-run validation mode **Recommendation:** Build migration tooling before adding complex property schemas. --- ## 7. Open Questions (from Specification) These design decisions remain unresolved: 1. **Property change triggers** - Should editing properties auto-create revision? 2. **Revision metadata editing** - Allow comment updates post-creation? 3. **Soft delete behavior** - Archive or hard delete revisions? 4. **File diff strategy** - Exploded FCStd storage for better diffing? 5. **Retention policy** - How long to keep old revisions? --- ## Appendix A: File Structure Revision endpoints, status, labels, authentication, audit logging, and file attachments are implemented. Current structure: ``` internal/ api/ audit_handlers.go # Audit/completeness endpoints auth_handlers.go # Login, tokens, OIDC bom_handlers.go # Flat BOM, cost roll-up file_handlers.go # Presigned uploads, item files, thumbnails handlers.go # Items, schemas, projects, revisions middleware.go # Auth middleware odoo_handlers.go # Odoo integration endpoints routes.go # Route registration (78 endpoints) search.go # Fuzzy search auth/ auth.go # Auth service: local, LDAP, OIDC db/ items.go # Item and revision repository item_files.go # File attachment repository relationships.go # BOM repository projects.go # Project repository storage/ storage.go # MinIO file storage helpers migrations/ 001_initial.sql # Core schema ... 011_item_files.sql # Item file attachments (latest) ``` Future features would add: ``` internal/ api/ lock_handlers.go # Locking endpoints db/ locks.go # Lock repository releases.go # Release repository migrations/ 012_item_locks.sql # Locking table 013_releases.sql # Release management ``` --- ## Appendix B: API Additions Summary ### Phase 1 Endpoints (Implemented) ``` GET /api/items/{pn}/revisions/compare # Diff two revisions POST /api/items/{pn}/revisions/{rev}/rollback # Create revision from old PATCH /api/items/{pn}/revisions/{rev} # Update status/labels ``` ### Phase 2 Endpoints **Authentication (Implemented):** ``` GET /api/auth/me # Current user info GET /api/auth/tokens # List API tokens POST /api/auth/tokens # Create API token DELETE /api/auth/tokens/{id} # Revoke API token ``` **Audit (Implemented):** ``` GET /api/audit/completeness # All items completeness summary GET /api/audit/completeness/{partNumber} # Per-item scoring ``` **File Attachments (Implemented):** ``` GET /api/auth/config # Auth config (public) POST /api/uploads/presign # Presigned upload URL GET /api/items/{pn}/files # List item files POST /api/items/{pn}/files # Associate file with item DELETE /api/items/{pn}/files/{fileId} # Delete file association PUT /api/items/{pn}/thumbnail # Set item thumbnail GET /api/items/{pn}/bom/flat # Flattened BOM GET /api/items/{pn}/bom/cost # Assembly cost roll-up ``` **Locking (Not Implemented):** ``` POST /api/items/{pn}/lock # Acquire lock DELETE /api/items/{pn}/lock # Release lock GET /api/items/{pn}/lock # Check lock status ``` ### Phase 3 Endpoints (Not Implemented) ``` GET /api/releases # List releases POST /api/releases # Create release GET /api/releases/{name} # Get release details POST /api/releases/{name}/items # Add items to release GET /api/items/{pn}/thumbnail/{rev} # Get thumbnail ``` --- ## Appendix C: SOLIDWORKS PDM Comparison This section compares Silo's capabilities against SOLIDWORKS PDM features. Gaps are categorized by priority and implementation complexity. **Legend:** Silo Status = Full / Partial / None | Priority = Critical / High / Medium / Low | Complexity = Simple / Moderate / Complex ### C.1 Version Control & Revision Management | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | Check-in/check-out | Full pessimistic locking | None | High | Moderate | | Version history | Complete with branching | Full (linear) | - | - | | Revision labels | A, B, C or custom schemes | Full (custom labels) | - | - | | Rollback/restore | Full | Full | - | - | | Compare revisions | Visual + metadata diff | Metadata diff only | Medium | Complex | | Get Latest Revision | One-click retrieval | Partial (API only) | Medium | Simple | Silo lacks pessimistic locking (check-out), which is critical for multi-user CAD environments where file merging is impractical. Visual diff comparison would require FreeCAD integration for CAD file visualization. ### C.2 Workflow Management | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | Custom workflows | Full visual designer | None | Critical | Complex | | State transitions | Configurable with permissions | Basic (status field only) | Critical | Complex | | Parallel approvals | Multiple approvers required | None | High | Complex | | Automatic transitions | Timer/condition-based | None | Medium | Moderate | | Email notifications | On state change | None | High | Moderate | | ECO process | Built-in change management | None | High | Complex | | Child state conditions | Block parent if children invalid | None | Medium | Moderate | Workflow management is the largest functional gap. SOLIDWORKS PDM offers sophisticated state machines with parallel approvals, automatic transitions, and deep integration with engineering change processes. Silo currently has only a simple status field (draft/review/released/obsolete) with no transition rules or approval processes. ### C.3 User Management & Security | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | User authentication | Windows AD, LDAP | Full (local, LDAP, OIDC) | - | - | | Role-based permissions | Granular per folder/state | Partial (3-tier role model) | Medium | Moderate | | Group management | Full | None | Medium | Moderate | | Folder permissions | Read/write/delete per folder | None | Medium | Moderate | | State permissions | Actions allowed per state | None | High | Moderate | | Audit trail | Complete action logging | Full | - | - | | Private files | Pre-check-in visibility control | None | Low | Simple | Authentication is implemented with three backends (local, LDAP/FreeIPA, OIDC/Keycloak) and a 3-tier role model (admin > editor > viewer). Audit logging captures user actions. Remaining gaps: group management, folder-level permissions, and state-based permission rules. ### C.4 Search & Discovery | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | Metadata search | Full with custom cards | Partial (API query params + fuzzy) | High | Moderate | | Full-text content search | iFilters for Office, CAD | None | Medium | Complex | | Quick search | Toolbar with history | Partial (fuzzy search API) | Medium | Simple | | Saved searches | User-defined favorites | None | Medium | Simple | | Advanced operators | AND, OR, NOT, wildcards | None | Medium | Simple | | Multi-variable search | Search across multiple fields | None | Medium | Simple | | Where-used search | Find all assemblies using part | Full | - | - | Silo has API-level filtering, fuzzy search, and where-used queries. Remaining gaps: saved searches, advanced search operators, and a richer search UI. Content search (searching within CAD files) is not planned for the server. ### C.5 BOM Management | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | Single-level BOM | Yes | Full | - | - | | Multi-level BOM | Indented/exploded views | Full (recursive, configurable depth) | - | - | | BOM comparison | Between revisions | None | Medium | Moderate | | BOM export | Excel, XML, ERP formats | Full (CSV, ODS) | - | - | | BOM import | Bulk BOM loading | Full (CSV with upsert) | - | - | | Calculated BOMs | Quantities rolled up | None | Medium | Moderate | | Reference designators | Full support | Full | - | - | | Alternate parts | Substitute tracking | Full | - | - | Multi-level BOM retrieval (recursive CTE with configurable depth) and BOM export (CSV, ODS) are implemented. BOM import supports CSV with upsert and cycle detection. Remaining gap: BOM comparison between revisions. ### C.6 CAD Integration | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | Native CAD add-in | Deep SOLIDWORKS integration | FreeCAD workbench (silo-mod) | Medium | Complex | | Property mapping | Bi-directional sync | Planned (silo-mod) | Medium | Moderate | | Task pane | Embedded in CAD UI | Auth dock panel (silo-mod) | Medium | Complex | | Lightweight components | Handle without full load | N/A | - | - | | Drawing/model linking | Automatic association | Manual | Medium | Moderate | | Multi-CAD support | Third-party formats | FreeCAD only | Low | - | CAD integration is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The Silo server provides the REST API endpoints consumed by those clients. ### C.7 External Integrations | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | ERP integration | SAP, Dynamics, etc. | Partial (Odoo stubs) | Medium | Complex | | API access | Full COM/REST API | Full REST API (78 endpoints) | - | - | | Dispatch scripts | Automation without coding | None | Medium | Moderate | | Task scheduler | Background processing | None | Medium | Moderate | | Email system | SMTP integration | None | High | Simple | | Web portal | Browser access | Full (React SPA + auth) | - | - | Silo has a comprehensive REST API (78 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation. ### C.8 Reporting & Analytics | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | Standard reports | Inventory, usage, activity | None | Medium | Moderate | | Custom reports | User-defined queries | None | Medium | Moderate | | Dashboard | Visual KPIs | None | Low | Moderate | | Export formats | PDF, Excel, CSV | CSV and ODS | Medium | Simple | Reporting capabilities are absent. Basic reports (item counts, revision activity, where-used) would provide immediate value. ### C.9 File Handling | Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity | |---------|---------------|-------------|----------|------------| | File versioning | Automatic | Full (MinIO) | - | - | | File preview | Thumbnails, 3D preview | None | Medium | Complex | | File conversion | PDF, DXF generation | None | Medium | Complex | | Replication | Multi-site sync | None | Low | Complex | | File copy with refs | Copy tree with references | None | Medium | Moderate | File storage works well. Thumbnail generation and file preview would significantly improve the web UI experience. Automatic conversion to PDF/DXF is valuable for sharing with non-CAD users. --- ## Appendix D: Feature Comparison Matrix | Category | Feature | SW PDM Standard | SW PDM Pro | Silo Current | Silo Planned | |----------|---------|-----------------|------------|--------------|--------------| | **Version Control** | Check-in/out | Yes | Yes | No | Tier 1 | | | Version history | Yes | Yes | Yes | - | | | Rollback | Yes | Yes | Yes | - | | | Revision labels/status | Yes | Yes | Yes | - | | | Revision comparison | Yes | Yes | Yes (metadata) | - | | **Workflow** | Custom workflows | Limited | Yes | No | Tier 4 | | | Parallel approval | No | Yes | No | Tier 4 | | | Notifications | No | Yes | No | Tier 1 | | **Security** | User auth | Windows | Windows/LDAP | Yes (local, LDAP, OIDC) | - | | | Permissions | Basic | Granular | Partial (role-based) | Tier 4 | | | Audit trail | Basic | Full | Yes | - | | **Search** | Metadata search | Yes | Yes | Partial (API + fuzzy) | Tier 0 | | | Content search | No | Yes | No | Tier 2 | | | Where-used | Yes | Yes | Yes | - | | **BOM** | Single-level | Yes | Yes | Yes | - | | | Multi-level | Yes | Yes | Yes (recursive) | - | | | BOM export | Yes | Yes | Yes (CSV, ODS) | - | | **Data** | CSV import/export | Yes | Yes | Yes | - | | | ODS import/export | No | No | Yes | - | | | Project management | Yes | Yes | Yes | - | | **Integration** | API | Limited | Full | Full REST (78) | - | | | ERP connectors | No | Yes | Partial (Odoo stubs) | Tier 6 | | | Web access | No | Yes | Yes (React SPA + auth) | - | | **Files** | Versioning | Yes | Yes | Yes | - | | | Preview | Yes | Yes | No | Tier 2 | | | Multi-site | No | Yes | No | Not Planned |