kindred/silo
1
1
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
issue-dedup-sourcing-cost
silo/ROADMAP.md
Forbes adf64d722c docs: update ROADMAP.md for React SPA and current counts 
- Replace all htmx references with React SPA
- Update endpoint count from 74 to 75 (6 occurrences)
- Update migration count from 10 to 11
- Update test file count from 1 to 9
- Fix comparison matrix web access entry

Closes #29
2026-02-08 14:05:08 -06:00

537 lines
23 KiB
Markdown
Raw Permalink Blame History

# Silo Roadmap
**Version:** 1.1
**Date:** February 2026
**Purpose:** Project inventory, SOLIDWORKS PDM gap analysis, and development roadmap
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Current Project Inventory](#current-project-inventory)
3. [SOLIDWORKS PDM Gap Analysis](#solidworks-pdm-gap-analysis)
4. [Feature Roadmap](#feature-roadmap)
5. [Implementation Phases](#implementation-phases)
---
## Executive Summary
Silo is an R&D-oriented item database and part management system. It provides configurable part number generation, revision tracking, BOM management, and file versioning through MinIO storage. CAD integration (FreeCAD workbench, LibreOffice Calc extension) is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)).
This document compares Silo's current capabilities against SOLIDWORKS PDM—the industry-leading product data management solution—to identify gaps and prioritize future development.
### Key Differentiators
| Aspect | Silo | SOLIDWORKS PDM |
|--------|------|----------------|
| **Target CAD** | FreeCAD / Kindred Create (open source) | SOLIDWORKS (proprietary) |
| **Part Numbering** | Schema-as-configuration (YAML) | Fixed format with some customization |
| **Licensing** | Open source / Kindred Proprietary | Commercial ($3,000-$10,000+ per seat) |
| **Storage** | PostgreSQL + MinIO (S3-compatible) | SQL Server + File Archive |
| **Philosophy** | R&D-oriented, lightweight | Enterprise-grade, comprehensive |
---
## Current Project Inventory
### Implemented Features (MVP Complete)
#### Core Database System
- PostgreSQL schema with 11 migrations
- UUID-based identifiers throughout
- Soft delete support via `archived_at` timestamps
- Atomic sequence generation for part numbers
#### Part Number Generation
- YAML schema parser with validation
- Segment types: `string`, `enum`, `serial`, `constant`
- Scope templates for serial counters (e.g., `{category}`, `{project}`)
- Format templates for custom output
#### Item Management
- Full CRUD operations for items
- Item types: part, assembly, drawing, document, tooling, purchased, electrical, software
- Custom properties via JSONB storage
- Project tagging with many-to-many relationships
#### Revision Control
- Append-only revision history
- Revision metadata: properties, file reference, checksum, comment
- Status tracking: draft, review, released, obsolete
- Labels/tags per revision
- Revision comparison (diff)
- Rollback functionality
#### File Management
- MinIO integration with versioning
- File upload/download via REST API
- SHA256 checksums for integrity
- Storage path: `items/{partNumber}/rev{N}.FCStd`
#### Bill of Materials (BOM)
- Relationship types: component, alternate, reference
- Multi-level BOM (recursive expansion with configurable depth)
- Where-used queries (reverse parent lookup)
- BOM CSV and ODS export/import with cycle detection
- Reference designators for electronics
- Quantity tracking with units
- Revision-specific child linking
#### Project Management
- Project CRUD operations
- Unique project codes (2-10 characters)
- Item-to-project tagging
- Project-filtered queries
#### Data Import/Export
- CSV export with configurable properties
- CSV import with dry-run validation
- ODS spreadsheet import/export (items, BOMs, project sheets)
- Template generation for import formatting
#### API & Web Interface
- REST API with 75 endpoints
- Authentication: local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak
- Role-based access control (admin > editor > viewer)
- API token management (SHA-256 hashed)
- Session management (PostgreSQL-backed, 24h lifetime)
- CSRF protection (nosurf on web forms)
- Middleware: logging, CORS, recovery, request ID
- Web UI — React SPA (Vite + TypeScript, Catppuccin Mocha theme)
- Fuzzy search
- Health and readiness probes
#### Audit & Completeness
- Audit logging (database table with user/action/resource tracking)
- Item completeness scoring with weighted fields
- Category-specific property validation
- Tier classification (critical/low/partial/good/complete)
#### Configuration
- YAML configuration with environment variable overrides
- Multi-schema support
- Docker Compose deployment ready
### Partially Implemented
| Feature | Status | Notes |
|---------|--------|-------|
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull sync operations are stubs |
| Date segment type | Not started | Schema parser placeholder exists |
| Part number validation | Not started | API accepts but doesn't validate format |
| Location hierarchy CRUD | Schema only | Tables exist, no API endpoints |
| Inventory tracking | Schema only | Tables exist, no API endpoints |
| Unit tests | Partial | 9 Go test files across api, db, ods, partnum, schema packages |
### Infrastructure Status
| Component | Status |
|-----------|--------|
| PostgreSQL | Running (psql.kindred.internal) |
| MinIO | Configured in Docker Compose |
| Silo API Server | Builds successfully |
| Docker Compose | Complete (dev and production) |
| systemd service | Unit file and env template ready |
| Deployment scripts | setup-host, deploy, init-db, setup-ipa-nginx |
---
## SOLIDWORKS PDM Gap Analysis
This section compares Silo's capabilities against SOLIDWORKS PDM features. Gaps are categorized by priority and implementation complexity.
### Legend
- **Silo Status:** Full / Partial / None
- **Priority:** Critical / High / Medium / Low
- **Complexity:** Simple / Moderate / Complex
---
### 1. Version Control & Revision Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Check-in/check-out | Full pessimistic locking | None | High | Moderate |
| Version history | Complete with branching | Full (linear) | - | - |
| Revision labels | A, B, C or custom schemes | Full (custom labels) | - | - |
| Rollback/restore | Full | Full | - | - |
| Compare revisions | Visual + metadata diff | Metadata diff only | Medium | Complex |
| Get Latest Revision | One-click retrieval | Partial (API only) | Medium | Simple |
**Gap Analysis:**
Silo lacks pessimistic locking (check-out), which is critical for multi-user CAD environments where file merging is impractical. Visual diff comparison would require FreeCAD integration for CAD file visualization.
---
### 2. Workflow Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Custom workflows | Full visual designer | None | Critical | Complex |
| State transitions | Configurable with permissions | Basic (status field only) | Critical | Complex |
| Parallel approvals | Multiple approvers required | None | High | Complex |
| Automatic transitions | Timer/condition-based | None | Medium | Moderate |
| Email notifications | On state change | None | High | Moderate |
| ECO process | Built-in change management | None | High | Complex |
| Child state conditions | Block parent if children invalid | None | Medium | Moderate |
**Gap Analysis:**
Workflow management is the largest functional gap. SOLIDWORKS PDM offers sophisticated state machines with parallel approvals, automatic transitions, and deep integration with engineering change processes. Silo currently has only a simple status field (draft/review/released/obsolete) with no transition rules or approval processes.
---
### 3. User Management & Security
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| User authentication | Windows AD, LDAP | Full (local, LDAP, OIDC) | - | - |
| Role-based permissions | Granular per folder/state | Partial (3-tier role model) | Medium | Moderate |
| Group management | Full | None | Medium | Moderate |
| Folder permissions | Read/write/delete per folder | None | Medium | Moderate |
| State permissions | Actions allowed per state | None | High | Moderate |
| Audit trail | Complete action logging | Full | - | - |
| Private files | Pre-check-in visibility control | None | Low | Simple |
**Gap Analysis:**
Authentication is implemented with three backends (local, LDAP/FreeIPA, OIDC/Keycloak) and a 3-tier role model (admin > editor > viewer). Audit logging captures user actions. Remaining gaps: group management, folder-level permissions, and state-based permission rules.
---
### 4. Search & Discovery
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Metadata search | Full with custom cards | Partial (API query params + fuzzy) | High | Moderate |
| Full-text content search | iFilters for Office, CAD | None | Medium | Complex |
| Quick search | Toolbar with history | Partial (fuzzy search API) | Medium | Simple |
| Saved searches | User-defined favorites | None | Medium | Simple |
| Advanced operators | AND, OR, NOT, wildcards | None | Medium | Simple |
| Multi-variable search | Search across multiple fields | None | Medium | Simple |
| Where-used search | Find all assemblies using part | Full | - | - |
**Gap Analysis:**
Silo has API-level filtering, fuzzy search, and where-used queries. Remaining gaps: saved searches, advanced search operators, and a richer search UI. Content search (searching within CAD files) is not planned for the server.
---
### 5. BOM Management
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Single-level BOM | Yes | Full | - | - |
| Multi-level BOM | Indented/exploded views | Full (recursive, configurable depth) | - | - |
| BOM comparison | Between revisions | None | Medium | Moderate |
| BOM export | Excel, XML, ERP formats | Full (CSV, ODS) | - | - |
| BOM import | Bulk BOM loading | Full (CSV with upsert) | - | - |
| Calculated BOMs | Quantities rolled up | None | Medium | Moderate |
| Reference designators | Full support | Full | - | - |
| Alternate parts | Substitute tracking | Full | - | - |
**Gap Analysis:**
Multi-level BOM retrieval (recursive CTE with configurable depth) and BOM export (CSV, ODS) are implemented. BOM import supports CSV with upsert and cycle detection. Remaining gap: BOM comparison between revisions.
---
### 6. CAD Integration
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Native CAD add-in | Deep SOLIDWORKS integration | FreeCAD workbench (silo-mod) | Medium | Complex |
| Property mapping | Bi-directional sync | Planned (silo-mod) | Medium | Moderate |
| Task pane | Embedded in CAD UI | Auth dock panel (silo-mod) | Medium | Complex |
| Lightweight components | Handle without full load | N/A | - | - |
| Drawing/model linking | Automatic association | Manual | Medium | Moderate |
| Multi-CAD support | Third-party formats | FreeCAD only | Low | - |
**Gap Analysis:**
CAD integration is maintained in separate repositories ([silo-mod](https://git.kindred-systems.com/kindred/silo-mod), [silo-calc](https://git.kindred-systems.com/kindred/silo-calc)). The Silo server provides the REST API endpoints consumed by those clients.
---
### 7. External Integrations
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| ERP integration | SAP, Dynamics, etc. | Partial (Odoo stubs) | Medium | Complex |
| API access | Full COM/REST API | Full REST API (75 endpoints) | - | - |
| Dispatch scripts | Automation without coding | None | Medium | Moderate |
| Task scheduler | Background processing | None | Medium | Moderate |
| Email system | SMTP integration | None | High | Simple |
| Web portal | Browser access | Full (React SPA + auth) | - | - |
**Gap Analysis:**
Silo has a comprehensive REST API (75 endpoints) and a full web UI with authentication. Odoo ERP integration has config/sync-log scaffolding but push/pull operations are stubs. Remaining gaps: email notifications, task scheduler, dispatch automation.
---
### 8. Reporting & Analytics
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| Standard reports | Inventory, usage, activity | None | Medium | Moderate |
| Custom reports | User-defined queries | None | Medium | Moderate |
| Dashboard | Visual KPIs | None | Low | Moderate |
| Export formats | PDF, Excel, CSV | CSV and ODS | Medium | Simple |
**Gap Analysis:**
Reporting capabilities are absent. Basic reports (item counts, revision activity, where-used) would provide immediate value.
---
### 9. File Handling
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|---------|---------------|-------------|----------|------------|
| File versioning | Automatic | Full (MinIO) | - | - |
| File preview | Thumbnails, 3D preview | None | Medium | Complex |
| File conversion | PDF, DXF generation | None | Medium | Complex |
| Replication | Multi-site sync | None | Low | Complex |
| File copy with refs | Copy tree with references | None | Medium | Moderate |
**Gap Analysis:**
File storage works well. Thumbnail generation and file preview would significantly improve the web UI experience. Automatic conversion to PDF/DXF is valuable for sharing with non-CAD users.
---
### Gap Summary by Priority
#### Completed (Previously Critical/High)
1. ~~**User authentication**~~ - Implemented: local, LDAP, OIDC
2. ~~**Role-based permissions**~~ - Implemented: 3-tier role model (admin/editor/viewer)
3. ~~**Audit trail**~~ - Implemented: audit_log table with completeness scoring
4. ~~**Where-used search**~~ - Implemented: reverse parent lookup API
5. ~~**Multi-level BOM API**~~ - Implemented: recursive expansion with configurable depth
6. ~~**BOM export**~~ - Implemented: CSV and ODS formats
#### Critical Gaps (Required for Team Use)
1. **Workflow engine** - State machines with transitions and approvals
2. **Check-out locking** - Pessimistic locking for CAD files
#### High Priority Gaps (Significant Value)
1. **Email notifications** - Alert users on state changes
2. **Web UI search** - Advanced search interface with saved searches
3. **Folder/state permissions** - Granular access control beyond role model
#### Medium Priority Gaps (Nice to Have)
1. **Saved searches** - Frequently used queries
2. **File preview/thumbnails** - Visual browsing
3. **Reporting** - Activity and inventory reports
4. **Scheduled tasks** - Background automation
5. **BOM comparison** - Revision diff for assemblies
---
## Feature Roadmap
### Phase 1: Foundation (Current - Q2 2026)
*Complete MVP and stabilize core functionality*
| Feature | Description | Status |
|---------|-------------|--------|
| MinIO integration | File upload/download with versioning and checksums | Complete |
| Revision control | Rollback, comparison, status/labels | Complete |
| CSV import/export | Dry-run validation, template generation | Complete |
| ODS import/export | Items, BOMs, project sheets, templates | Complete |
| Project management | CRUD, many-to-many item tagging | Complete |
| Multi-level BOM | Recursive expansion, where-used, export | Complete |
| Authentication | Local, LDAP, OIDC with role-based access | Complete |
| Audit logging | Action logging, completeness scoring | Complete |
| Unit tests | Core API and database operations | Not Started |
| Date segment type | Support date-based part number segments | Not Started |
| Part number validation | Validate format on creation | Not Started |
| Location CRUD API | Expose location hierarchy via REST | Not Started |
| Inventory API | Expose inventory operations via REST | Not Started |
### Phase 2: Multi-User (Q2-Q3 2026)
*Enable team collaboration*
| Feature | Description | Status |
|---------|-------------|--------|
| LDAP authentication | Integrate with FreeIPA/Active Directory | **Complete** |
| OIDC authentication | Keycloak / OpenID Connect | **Complete** |
| Audit logging | Record all user actions with timestamps | **Complete** |
| Session management | Token-based and session-based API authentication | **Complete** |
| User/group management | Create, assign, manage users and groups | Not Started |
| Folder permissions | Read/write/delete per folder hierarchy | Not Started |
| Check-out locking | Pessimistic locks with timeout | Not Started |
### Phase 3: Workflow Engine (Q3-Q4 2026)
*Implement engineering change processes*
| Feature | Description | Complexity |
|---------|-------------|------------|
| Workflow designer | YAML-defined state machines | Complex |
| State transitions | Configurable transition rules | Complex |
| Transition permissions | Who can execute which transitions | Moderate |
| Single approvals | Basic approval workflow | Moderate |
| Parallel approvals | Multi-approver gates | Complex |
| Automatic transitions | Timer and condition-based | Complex |
| Email notifications | SMTP integration for alerts | Simple |
| Child state conditions | Block parent transitions | Moderate |
### Phase 4: Search & Discovery (Q4 2026 - Q1 2027)
*Improve findability and navigation*
| Feature | Description | Status |
|---------|-------------|--------|
| Where-used queries | Find parent assemblies | **Complete** |
| Fuzzy search | Quick search across items | **Complete** |
| Advanced search UI | Web interface with filters | Not Started |
| Search operators | AND, OR, NOT, wildcards | Not Started |
| Saved searches | User favorites | Not Started |
| Content search | Search within file content | Not Started |
### Phase 5: BOM & Reporting (Q1-Q2 2027)
*Enhanced BOM management and analytics*
| Feature | Description | Status |
|---------|-------------|--------|
| Multi-level BOM API | Recursive assembly retrieval | **Complete** |
| BOM export | CSV and ODS formats | **Complete** |
| BOM import | CSV with upsert and cycle detection | **Complete** |
| BOM comparison | Diff between revisions | Not Started |
| Standard reports | Activity, inventory, usage | Not Started |
| Custom queries | User-defined report builder | Not Started |
| Dashboard | Visual KPIs and metrics | Not Started |
### Phase 6: Advanced Features (Q2-Q4 2027)
*Enterprise capabilities*
| Feature | Description | Complexity |
|---------|-------------|------------|
| File preview | Thumbnail generation | Complex |
| File conversion | Auto-generate PDF/DXF | Complex |
| ERP integration | Adapter framework | Complex |
| Task scheduler | Background job processing | Moderate |
| Webhooks | Event notifications to external systems | Moderate |
| API rate limiting | Protect against abuse | Simple |
---
## Implementation Phases
### Phase 1 Detailed Tasks
#### 1.1 MinIO Integration -- COMPLETE
- [x] MinIO service configured in Docker Compose
- [x] File upload via REST API
- [x] File download via REST API (latest and by revision)
- [x] SHA256 checksums on upload
#### 1.2 Authentication & Authorization -- COMPLETE
- [x] Local authentication (bcrypt)
- [x] LDAP/FreeIPA authentication
- [x] OIDC/Keycloak authentication
- [x] Role-based access control (admin/editor/viewer)
- [x] API token management (SHA-256 hashed)
- [x] Session management (PostgreSQL-backed)
- [x] CSRF protection (nosurf)
- [x] Audit logging (database table)
#### 1.3 Multi-level BOM & Export -- COMPLETE
- [x] Recursive BOM expansion with configurable depth
- [x] Where-used reverse lookup
- [x] BOM CSV export/import with cycle detection
- [x] BOM ODS export
- [x] ODS item export/import/template
#### 1.4 Unit Test Suite
- [ ] Database connection and transaction tests
- [ ] Item CRUD operation tests
- [ ] Revision creation and retrieval tests
- [ ] Part number generation tests
- [ ] File upload/download tests
- [ ] CSV import/export tests
- [ ] API endpoint tests
#### 1.5 Missing Segment Types
- [ ] Implement date segment type
- [ ] Add strftime-style format support
#### 1.6 Location & Inventory APIs
- [ ] `GET /api/locations` - List locations
- [ ] `POST /api/locations` - Create location
- [ ] `GET /api/locations/{path}` - Get location
- [ ] `DELETE /api/locations/{path}` - Delete location
- [ ] `GET /api/inventory/{partNumber}` - Get inventory
- [ ] `POST /api/inventory/{partNumber}/adjust` - Adjust quantity
- [ ] `POST /api/inventory/{partNumber}/move` - Move between locations
---
## Success Metrics
### Phase 1 (Foundation)
- All existing tests pass
- File upload/download works end-to-end
- FreeCAD users can checkout, modify, commit parts
### Phase 2 (Multi-User)
- 5+ concurrent users supported
- No data corruption under concurrent access
- Audit log captures all modifications
### Phase 3 (Workflow)
- Engineering change process completable in Silo
- Email notifications delivered reliably
- Workflow state visible in web UI
### Phase 4+ (Advanced)
- Search returns results in <2 seconds
- Where-used queries complete in <5 seconds
- BOM export matches assembly structure
---
## References
### SOLIDWORKS PDM Documentation
- [SOLIDWORKS PDM Product Page](https://www.solidworks.com/product/solidworks-pdm)
- [What's New in SOLIDWORKS PDM 2025](https://blogs.solidworks.com/solidworksblog/2024/10/whats-new-in-solidworks-pdm-2025.html)
- [Top 5 Enhancements in SOLIDWORKS PDM 2024](https://blogs.solidworks.com/solidworksblog/2023/10/top-5-enhancements-in-solidworks-pdm-2024.html)
- [SOLIDWORKS PDM Workflow Transitions](https://help.solidworks.com/2023/english/EnterprisePDM/Admin/c_workflow_transition.htm)
- [Ultimate Guide to SOLIDWORKS PDM Permissions](https://www.goengineer.com/blog/ultimate-guide-to-solidworks-pdm-permissions)
- [Searching in SOLIDWORKS PDM](https://help.solidworks.com/2021/english/EnterprisePDM/fileexplorer/c_searches.htm)
- [SOLIDWORKS PDM API Getting Started](https://3dswym.3dexperience.3ds.com/wiki/solidworks-news-info/getting-started-with-the-solidworks-pdm-api-solidpractices_gBCYaM75RgORBcpSO1m_Mw)
### Silo Documentation
- [Specification](docs/SPECIFICATION.md)
- [Development Status](docs/STATUS.md)
- [Deployment Guide](docs/DEPLOYMENT.md)
- [Gap Analysis](docs/GAP_ANALYSIS.md)
---
## Appendix: Feature Comparison Matrix
| Category | Feature | SW PDM Standard | SW PDM Pro | Silo Current | Silo Planned |
|----------|---------|-----------------|------------|--------------|--------------|
| **Version Control** | Check-in/out | Yes | Yes | No | Phase 2 |
| | Version history | Yes | Yes | Yes | - |
| | Rollback | Yes | Yes | Yes | - |
| | Revision labels/status | Yes | Yes | Yes | - |
| | Revision comparison | Yes | Yes | Yes (metadata) | - |
| **Workflow** | Custom workflows | Limited | Yes | No | Phase 3 |
| | Parallel approval | No | Yes | No | Phase 3 |
| | Notifications | No | Yes | No | Phase 3 |
| **Security** | User auth | Windows | Windows/LDAP | Yes (local, LDAP, OIDC) | - |
| | Permissions | Basic | Granular | Partial (role-based) | Phase 2 |
| | Audit trail | Basic | Full | Yes | - |
| **Search** | Metadata search | Yes | Yes | Partial (API + fuzzy) | Phase 4 |
| | Content search | No | Yes | No | Phase 4 |
| | Where-used | Yes | Yes | Yes | - |
| **BOM** | Single-level | Yes | Yes | Yes | - |
| | Multi-level | Yes | Yes | Yes (recursive) | - |
| | BOM export | Yes | Yes | Yes (CSV, ODS) | - |
| **Data** | CSV import/export | Yes | Yes | Yes | - |
| | ODS import/export | No | No | Yes | - |
| | Project management | Yes | Yes | Yes | - |
| **Integration** | API | Limited | Full | Full REST (75) | - |
| | ERP connectors | No | Yes | Partial (Odoo stubs) | Phase 6 |
| | Web access | No | Yes | Yes (React SPA + auth) | - |
| **Files** | Versioning | Yes | Yes | Yes | - |
| | Preview | Yes | Yes | No | Phase 6 |
| | Multi-site | No | Yes | No | Not Planned |
Reference in New Issue View Git Blame Copy Permalink