docs: replace MinIO with filesystem storage throughout #139
@@ -10,8 +10,6 @@
|
|||||||
#
|
#
|
||||||
# Credentials via environment variables (set in /etc/silo/silod.env):
|
# Credentials via environment variables (set in /etc/silo/silod.env):
|
||||||
# SILO_DB_PASSWORD
|
# SILO_DB_PASSWORD
|
||||||
# SILO_MINIO_ACCESS_KEY
|
|
||||||
# SILO_MINIO_SECRET_KEY
|
|
||||||
# SILO_SESSION_SECRET
|
# SILO_SESSION_SECRET
|
||||||
# SILO_ADMIN_PASSWORD
|
# SILO_ADMIN_PASSWORD
|
||||||
|
|
||||||
@@ -30,12 +28,9 @@ database:
|
|||||||
max_connections: 20
|
max_connections: 20
|
||||||
|
|
||||||
storage:
|
storage:
|
||||||
endpoint: "minio.example.internal:9000"
|
backend: "filesystem"
|
||||||
access_key: "" # Set via SILO_MINIO_ACCESS_KEY
|
filesystem:
|
||||||
secret_key: "" # Set via SILO_MINIO_SECRET_KEY
|
root_dir: "/opt/silo/data"
|
||||||
bucket: "silo-files"
|
|
||||||
use_ssl: true
|
|
||||||
region: "us-east-1"
|
|
||||||
|
|
||||||
schemas:
|
schemas:
|
||||||
directory: "/opt/silo/schemas"
|
directory: "/opt/silo/schemas"
|
||||||
|
|||||||
@@ -6,10 +6,6 @@
|
|||||||
# Database: silo, User: silo
|
# Database: silo, User: silo
|
||||||
SILO_DB_PASSWORD=
|
SILO_DB_PASSWORD=
|
||||||
|
|
||||||
# MinIO credentials (minio.example.internal)
|
|
||||||
# User: silouser
|
|
||||||
SILO_MINIO_ACCESS_KEY=silouser
|
|
||||||
SILO_MINIO_SECRET_KEY=
|
|
||||||
|
|
||||||
# Authentication
|
# Authentication
|
||||||
# Session secret (required when auth is enabled)
|
# Session secret (required when auth is enabled)
|
||||||
|
|||||||
@@ -73,25 +73,27 @@ database:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Storage (MinIO/S3)
|
## Storage (Filesystem)
|
||||||
|
|
||||||
| Key | Type | Default | Env Override | Description |
|
Files are stored on the local filesystem under a configurable root directory.
|
||||||
|-----|------|---------|-------------|-------------|
|
|
||||||
| `storage.endpoint` | string | — | `SILO_MINIO_ENDPOINT` | MinIO/S3 endpoint (`host:port`) |
|
| Key | Type | Default | Description |
|
||||||
| `storage.access_key` | string | — | `SILO_MINIO_ACCESS_KEY` | Access key |
|
|-----|------|---------|-------------|
|
||||||
| `storage.secret_key` | string | — | `SILO_MINIO_SECRET_KEY` | Secret key |
|
| `storage.backend` | string | `"filesystem"` | Storage backend (`filesystem`) |
|
||||||
| `storage.bucket` | string | — | — | S3 bucket name (created automatically if missing) |
|
| `storage.filesystem.root_dir` | string | — | Root directory for file storage (required) |
|
||||||
| `storage.use_ssl` | bool | `false` | — | Use HTTPS for MinIO connections |
|
|
||||||
| `storage.region` | string | `"us-east-1"` | — | S3 region |
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
storage:
|
storage:
|
||||||
endpoint: "localhost:9000"
|
backend: "filesystem"
|
||||||
access_key: "" # use SILO_MINIO_ACCESS_KEY env var
|
filesystem:
|
||||||
secret_key: "" # use SILO_MINIO_SECRET_KEY env var
|
root_dir: "/opt/silo/data"
|
||||||
bucket: "silo-files"
|
```
|
||||||
use_ssl: false
|
|
||||||
region: "us-east-1"
|
Ensure the directory exists and is writable by the `silo` user:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo mkdir -p /opt/silo/data
|
||||||
|
sudo chown silo:silo /opt/silo/data
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -264,9 +266,6 @@ All environment variable overrides. These take precedence over values in `config
|
|||||||
| `SILO_DB_NAME` | `database.name` | PostgreSQL database name |
|
| `SILO_DB_NAME` | `database.name` | PostgreSQL database name |
|
||||||
| `SILO_DB_USER` | `database.user` | PostgreSQL user |
|
| `SILO_DB_USER` | `database.user` | PostgreSQL user |
|
||||||
| `SILO_DB_PASSWORD` | `database.password` | PostgreSQL password |
|
| `SILO_DB_PASSWORD` | `database.password` | PostgreSQL password |
|
||||||
| `SILO_MINIO_ENDPOINT` | `storage.endpoint` | MinIO endpoint |
|
|
||||||
| `SILO_MINIO_ACCESS_KEY` | `storage.access_key` | MinIO access key |
|
|
||||||
| `SILO_MINIO_SECRET_KEY` | `storage.secret_key` | MinIO secret key |
|
|
||||||
| `SILO_SESSION_SECRET` | `auth.session_secret` | Session cookie signing secret |
|
| `SILO_SESSION_SECRET` | `auth.session_secret` | Session cookie signing secret |
|
||||||
| `SILO_ADMIN_USERNAME` | `auth.local.default_admin_username` | Default admin username |
|
| `SILO_ADMIN_USERNAME` | `auth.local.default_admin_username` | Default admin username |
|
||||||
| `SILO_ADMIN_PASSWORD` | `auth.local.default_admin_password` | Default admin password |
|
| `SILO_ADMIN_PASSWORD` | `auth.local.default_admin_password` | Default admin password |
|
||||||
@@ -296,11 +295,9 @@ database:
|
|||||||
sslmode: "disable"
|
sslmode: "disable"
|
||||||
|
|
||||||
storage:
|
storage:
|
||||||
endpoint: "localhost:9000"
|
backend: "filesystem"
|
||||||
access_key: "minioadmin"
|
filesystem:
|
||||||
secret_key: "minioadmin"
|
root_dir: "./data"
|
||||||
bucket: "silo-files"
|
|
||||||
use_ssl: false
|
|
||||||
|
|
||||||
schemas:
|
schemas:
|
||||||
directory: "./schemas"
|
directory: "./schemas"
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
> instructions. This document covers ongoing maintenance and operations for an
|
> instructions. This document covers ongoing maintenance and operations for an
|
||||||
> existing deployment.
|
> existing deployment.
|
||||||
|
|
||||||
This guide covers deploying Silo to a dedicated VM using external PostgreSQL and MinIO services.
|
This guide covers deploying Silo to a dedicated VM using external PostgreSQL and local filesystem storage.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
@@ -26,28 +26,25 @@ This guide covers deploying Silo to a dedicated VM using external PostgreSQL and
|
|||||||
│ │ silod │ │
|
│ │ silod │ │
|
||||||
│ │ (Silo API Server) │ │
|
│ │ (Silo API Server) │ │
|
||||||
│ │ :8080 │ │
|
│ │ :8080 │ │
|
||||||
|
│ │ Files: /opt/silo/data │ │
|
||||||
│ └───────────────────────────────────────────────────────────┘ │
|
│ └───────────────────────────────────────────────────────────┘ │
|
||||||
└─────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
│ │
|
│
|
||||||
▼ ▼
|
▼
|
||||||
┌─────────────────────────┐ ┌─────────────────────────────────┐
|
┌─────────────────────────┐
|
||||||
│ psql.example.internal │ │ minio.example.internal │
|
│ psql.example.internal │
|
||||||
│ PostgreSQL 16 │ │ MinIO S3 │
|
│ PostgreSQL 16 │
|
||||||
│ :5432 │ │ :9000 (API) │
|
│ :5432 │
|
||||||
│ │ │ :9001 (Console) │
|
└─────────────────────────┘
|
||||||
└─────────────────────────┘ └─────────────────────────────────┘
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## External Services
|
## External Services
|
||||||
|
|
||||||
The following external services are already configured:
|
| Service | Host | Database | User |
|
||||||
|
|---------|------|----------|------|
|
||||||
| Service | Host | Database/Bucket | User |
|
|
||||||
|---------|------|-----------------|------|
|
|
||||||
| PostgreSQL | psql.example.internal:5432 | silo | silo |
|
| PostgreSQL | psql.example.internal:5432 | silo | silo |
|
||||||
| MinIO | minio.example.internal:9000 | silo-files | silouser |
|
|
||||||
|
|
||||||
Migrations have been applied to the database.
|
Files are stored on the local filesystem at `/opt/silo/data`. Migrations have been applied to the database.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -107,21 +104,15 @@ Fill in the values:
|
|||||||
# Database credentials (psql.example.internal)
|
# Database credentials (psql.example.internal)
|
||||||
SILO_DB_PASSWORD=your-database-password
|
SILO_DB_PASSWORD=your-database-password
|
||||||
|
|
||||||
# MinIO credentials (minio.example.internal)
|
|
||||||
SILO_MINIO_ACCESS_KEY=silouser
|
|
||||||
SILO_MINIO_SECRET_KEY=your-minio-secret-key
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Verify External Services
|
### Verify External Services
|
||||||
|
|
||||||
Before deploying, verify connectivity to external services:
|
Before deploying, verify connectivity to PostgreSQL:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Test PostgreSQL
|
|
||||||
psql -h psql.example.internal -U silo -d silo -c 'SELECT 1'
|
psql -h psql.example.internal -U silo -d silo -c 'SELECT 1'
|
||||||
|
|
||||||
# Test MinIO
|
|
||||||
curl -I http://minio.example.internal:9000/minio/health/live
|
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -183,6 +174,7 @@ sudo -E /opt/silo/src/scripts/deploy.sh
|
|||||||
| File | Purpose |
|
| File | Purpose |
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| `/opt/silo/bin/silod` | Server binary |
|
| `/opt/silo/bin/silod` | Server binary |
|
||||||
|
| `/opt/silo/data/` | File storage root |
|
||||||
| `/opt/silo/src/` | Git repository checkout |
|
| `/opt/silo/src/` | Git repository checkout |
|
||||||
| `/etc/silo/config.yaml` | Server configuration |
|
| `/etc/silo/config.yaml` | Server configuration |
|
||||||
| `/etc/silo/silod.env` | Environment variables (secrets) |
|
| `/etc/silo/silod.env` | Environment variables (secrets) |
|
||||||
@@ -242,7 +234,7 @@ sudo journalctl -u silod --since "2024-01-15 10:00:00"
|
|||||||
# Basic health check
|
# Basic health check
|
||||||
curl http://localhost:8080/health
|
curl http://localhost:8080/health
|
||||||
|
|
||||||
# Full readiness check (includes DB and MinIO)
|
# Full readiness check (includes DB)
|
||||||
curl http://localhost:8080/ready
|
curl http://localhost:8080/ready
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -318,24 +310,6 @@ psql -h psql.example.internal -U silo -d silo -f /opt/silo/src/migrations/008_ne
|
|||||||
|
|
||||||
3. Check `pg_hba.conf` on PostgreSQL server allows connections from this host.
|
3. Check `pg_hba.conf` on PostgreSQL server allows connections from this host.
|
||||||
|
|
||||||
### Connection Refused to MinIO
|
|
||||||
|
|
||||||
1. Test network connectivity:
|
|
||||||
```bash
|
|
||||||
nc -zv minio.example.internal 9000
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Test with curl:
|
|
||||||
```bash
|
|
||||||
curl -I http://minio.example.internal:9000/minio/health/live
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Check SSL settings in config match MinIO setup:
|
|
||||||
```yaml
|
|
||||||
storage:
|
|
||||||
use_ssl: true # or false
|
|
||||||
```
|
|
||||||
|
|
||||||
### Health Check Fails
|
### Health Check Fails
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -345,7 +319,9 @@ curl -v http://localhost:8080/ready
|
|||||||
|
|
||||||
# If ready fails but health passes, check external services
|
# If ready fails but health passes, check external services
|
||||||
psql -h psql.example.internal -U silo -d silo -c 'SELECT 1'
|
psql -h psql.example.internal -U silo -d silo -c 'SELECT 1'
|
||||||
curl http://minio.example.internal:9000/minio/health/live
|
|
||||||
|
# Check file storage directory
|
||||||
|
ls -la /opt/silo/data
|
||||||
```
|
```
|
||||||
|
|
||||||
### Build Fails
|
### Build Fails
|
||||||
@@ -460,10 +436,9 @@ sudo systemctl reload nginx
|
|||||||
|
|
||||||
- [ ] `/etc/silo/silod.env` has mode 600 (`chmod 600`)
|
- [ ] `/etc/silo/silod.env` has mode 600 (`chmod 600`)
|
||||||
- [ ] Database password is strong and unique
|
- [ ] Database password is strong and unique
|
||||||
- [ ] MinIO credentials are specific to silo (not admin)
|
|
||||||
- [ ] SSL/TLS enabled for PostgreSQL (`sslmode: require`)
|
- [ ] SSL/TLS enabled for PostgreSQL (`sslmode: require`)
|
||||||
- [ ] SSL/TLS enabled for MinIO (`use_ssl: true`) if available
|
|
||||||
- [ ] HTTPS enabled via nginx reverse proxy
|
- [ ] HTTPS enabled via nginx reverse proxy
|
||||||
|
- [ ] File storage directory (`/opt/silo/data`) owned by `silo` user with mode 750
|
||||||
- [ ] Silod listens on localhost only (`host: 127.0.0.1`)
|
- [ ] Silod listens on localhost only (`host: 127.0.0.1`)
|
||||||
- [ ] Firewall allows only ports 80, 443 (not 8080)
|
- [ ] Firewall allows only ports 80, 443 (not 8080)
|
||||||
- [ ] Service runs as non-root `silo` user
|
- [ ] Service runs as non-root `silo` user
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ See [ROADMAP.md](ROADMAP.md) for the platform roadmap and dependency tier struct
|
|||||||
| Append-only revision history | Complete | `internal/db/items.go` |
|
| Append-only revision history | Complete | `internal/db/items.go` |
|
||||||
| Sequential revision numbering | Complete | Database trigger |
|
| Sequential revision numbering | Complete | Database trigger |
|
||||||
| Property snapshots (JSONB) | Complete | `revisions.properties` |
|
| Property snapshots (JSONB) | Complete | `revisions.properties` |
|
||||||
| File versioning (MinIO) | Complete | `internal/storage/` |
|
| File storage (filesystem) | Complete | `internal/storage/` |
|
||||||
| SHA256 checksums | Complete | Captured on upload |
|
| SHA256 checksums | Complete | Captured on upload |
|
||||||
| Revision comments | Complete | `revisions.comment` |
|
| Revision comments | Complete | `revisions.comment` |
|
||||||
| User attribution | Complete | `revisions.created_by` |
|
| User attribution | Complete | `revisions.created_by` |
|
||||||
@@ -93,7 +93,7 @@ CREATE TABLE revisions (
|
|||||||
revision_number INTEGER NOT NULL,
|
revision_number INTEGER NOT NULL,
|
||||||
properties JSONB NOT NULL DEFAULT '{}',
|
properties JSONB NOT NULL DEFAULT '{}',
|
||||||
file_key TEXT,
|
file_key TEXT,
|
||||||
file_version TEXT, -- MinIO version ID
|
file_version TEXT, -- storage version ID
|
||||||
file_checksum TEXT, -- SHA256
|
file_checksum TEXT, -- SHA256
|
||||||
file_size BIGINT,
|
file_size BIGINT,
|
||||||
thumbnail_key TEXT,
|
thumbnail_key TEXT,
|
||||||
@@ -283,7 +283,7 @@ Effort: Medium | Priority: Low | Risk: Low
|
|||||||
|
|
||||||
**Changes:**
|
**Changes:**
|
||||||
- Add thumbnail generation on file upload
|
- Add thumbnail generation on file upload
|
||||||
- Store in MinIO at `thumbnails/{part_number}/rev{n}.png`
|
- Store at `thumbnails/{part_number}/rev{n}.png`
|
||||||
- Expose via `GET /api/items/{pn}/thumbnail/{rev}`
|
- Expose via `GET /api/items/{pn}/thumbnail/{rev}`
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -377,7 +377,7 @@ internal/
|
|||||||
relationships.go # BOM repository
|
relationships.go # BOM repository
|
||||||
projects.go # Project repository
|
projects.go # Project repository
|
||||||
storage/
|
storage/
|
||||||
storage.go # MinIO file storage helpers
|
storage.go # File storage helpers
|
||||||
migrations/
|
migrations/
|
||||||
001_initial.sql # Core schema
|
001_initial.sql # Core schema
|
||||||
...
|
...
|
||||||
@@ -572,7 +572,7 @@ Reporting capabilities are absent. Basic reports (item counts, revision activity
|
|||||||
|
|
||||||
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
| Feature | SOLIDWORKS PDM | Silo Status | Priority | Complexity |
|
||||||
|---------|---------------|-------------|----------|------------|
|
|---------|---------------|-------------|----------|------------|
|
||||||
| File versioning | Automatic | Full (MinIO) | - | - |
|
| File versioning | Automatic | Full (filesystem) | - | - |
|
||||||
| File preview | Thumbnails, 3D preview | None | Medium | Complex |
|
| File preview | Thumbnails, 3D preview | None | Medium | Complex |
|
||||||
| File conversion | PDF, DXF generation | None | Medium | Complex |
|
| File conversion | PDF, DXF generation | None | Medium | Complex |
|
||||||
| Replication | Multi-site sync | None | Low | Complex |
|
| Replication | Multi-site sync | None | Low | Complex |
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
This guide covers two installation methods:
|
This guide covers two installation methods:
|
||||||
|
|
||||||
- **[Option A: Docker Compose](#option-a-docker-compose)** — self-contained stack with all services. Recommended for evaluation, small teams, and environments where Docker is the standard.
|
- **[Option A: Docker Compose](#option-a-docker-compose)** — self-contained stack with all services. Recommended for evaluation, small teams, and environments where Docker is the standard.
|
||||||
- **[Option B: Daemon Install](#option-b-daemon-install-systemd--external-services)** — systemd service with external PostgreSQL, MinIO, and optional LDAP/nginx. Recommended for production deployments integrated with existing infrastructure.
|
- **[Option B: Daemon Install](#option-b-daemon-install-systemd--external-services)** — systemd service with external PostgreSQL and optional LDAP/nginx. Files are stored on the local filesystem. Recommended for production deployments integrated with existing infrastructure.
|
||||||
|
|
||||||
Both methods produce the same result: a running Silo server with a web UI, REST API, and authentication.
|
Both methods produce the same result: a running Silo server with a web UI, REST API, and authentication.
|
||||||
|
|
||||||
@@ -48,7 +48,7 @@ Regardless of which method you choose:
|
|||||||
|
|
||||||
## Option A: Docker Compose
|
## Option A: Docker Compose
|
||||||
|
|
||||||
A single Docker Compose file runs everything: PostgreSQL, MinIO, OpenLDAP, and Silo. An optional nginx container can be enabled for reverse proxying.
|
A single Docker Compose file runs everything: PostgreSQL, OpenLDAP, and Silo. Files are stored on the local filesystem. An optional nginx container can be enabled for reverse proxying.
|
||||||
|
|
||||||
### A.1 Prerequisites
|
### A.1 Prerequisites
|
||||||
|
|
||||||
@@ -80,7 +80,6 @@ The setup script generates credentials and configuration files:
|
|||||||
It prompts for:
|
It prompts for:
|
||||||
- Server domain (default: `localhost`)
|
- Server domain (default: `localhost`)
|
||||||
- PostgreSQL password (auto-generated if you press Enter)
|
- PostgreSQL password (auto-generated if you press Enter)
|
||||||
- MinIO credentials (auto-generated)
|
|
||||||
- OpenLDAP admin password and initial user (auto-generated)
|
- OpenLDAP admin password and initial user (auto-generated)
|
||||||
- Silo local admin account (fallback when LDAP is unavailable)
|
- Silo local admin account (fallback when LDAP is unavailable)
|
||||||
|
|
||||||
@@ -106,7 +105,7 @@ Wait for all services to become healthy:
|
|||||||
docker compose -f deployments/docker-compose.allinone.yaml ps
|
docker compose -f deployments/docker-compose.allinone.yaml ps
|
||||||
```
|
```
|
||||||
|
|
||||||
You should see `silo-postgres`, `silo-minio`, `silo-openldap`, and `silo-api` all in a healthy state.
|
You should see `silo-postgres`, `silo-openldap`, and `silo-api` all in a healthy state.
|
||||||
|
|
||||||
View logs:
|
View logs:
|
||||||
|
|
||||||
@@ -124,7 +123,7 @@ docker compose -f deployments/docker-compose.allinone.yaml logs -f silo
|
|||||||
# Health check
|
# Health check
|
||||||
curl http://localhost:8080/health
|
curl http://localhost:8080/health
|
||||||
|
|
||||||
# Readiness check (includes database and storage connectivity)
|
# Readiness check (includes database connectivity)
|
||||||
curl http://localhost:8080/ready
|
curl http://localhost:8080/ready
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -226,7 +225,7 @@ The Silo container is rebuilt from the updated source. Database migrations in `m
|
|||||||
|
|
||||||
## Option B: Daemon Install (systemd + External Services)
|
## Option B: Daemon Install (systemd + External Services)
|
||||||
|
|
||||||
This method runs Silo as a systemd service on a dedicated host, connecting to externally managed PostgreSQL, MinIO, and optionally LDAP services.
|
This method runs Silo as a systemd service on a dedicated host, connecting to externally managed PostgreSQL and optionally LDAP services. Files are stored on the local filesystem.
|
||||||
|
|
||||||
### B.1 Architecture Overview
|
### B.1 Architecture Overview
|
||||||
|
|
||||||
@@ -240,21 +239,22 @@ This method runs Silo as a systemd service on a dedicated host, connecting to ex
|
|||||||
│ ┌───────▼────────┐ │
|
│ ┌───────▼────────┐ │
|
||||||
│ │ silod │ │
|
│ │ silod │ │
|
||||||
│ │ (API server) │ │
|
│ │ (API server) │ │
|
||||||
│ └──┬─────────┬───┘ │
|
│ │ Files: /opt/ │ │
|
||||||
└─────┼─────────┼──────┘
|
│ │ silo/data │ │
|
||||||
│ │
|
│ └──────┬─────────┘ │
|
||||||
┌───────────▼──┐ ┌───▼──────────────┐
|
└─────────┼────────────┘
|
||||||
│ PostgreSQL 16│ │ MinIO (S3) │
|
│
|
||||||
│ :5432 │ │ :9000 API │
|
┌───────────▼──┐
|
||||||
└──────────────┘ │ :9001 Console │
|
│ PostgreSQL 16│
|
||||||
└──────────────────┘
|
│ :5432 │
|
||||||
|
└──────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
### B.2 Prerequisites
|
### B.2 Prerequisites
|
||||||
|
|
||||||
- Linux host (Debian/Ubuntu or RHEL/Fedora/AlmaLinux)
|
- Linux host (Debian/Ubuntu or RHEL/Fedora/AlmaLinux)
|
||||||
- Root or sudo access
|
- Root or sudo access
|
||||||
- Network access to your PostgreSQL and MinIO servers
|
- Network access to your PostgreSQL server
|
||||||
|
|
||||||
The setup script installs Go and other build dependencies automatically.
|
The setup script installs Go and other build dependencies automatically.
|
||||||
|
|
||||||
@@ -281,26 +281,6 @@ Verify:
|
|||||||
psql -h YOUR_PG_HOST -U silo -d silo -c 'SELECT 1'
|
psql -h YOUR_PG_HOST -U silo -d silo -c 'SELECT 1'
|
||||||
```
|
```
|
||||||
|
|
||||||
#### MinIO
|
|
||||||
|
|
||||||
Install MinIO and create a bucket and service account:
|
|
||||||
|
|
||||||
- [MinIO quickstart](https://min.io/docs/minio/linux/index.html)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Using the MinIO client (mc):
|
|
||||||
mc alias set local http://YOUR_MINIO_HOST:9000 minioadmin minioadmin
|
|
||||||
mc mb local/silo-files
|
|
||||||
mc admin user add local silouser YOUR_MINIO_SECRET
|
|
||||||
mc admin policy attach local readwrite --user silouser
|
|
||||||
```
|
|
||||||
|
|
||||||
Verify:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -I http://YOUR_MINIO_HOST:9000/minio/health/live
|
|
||||||
```
|
|
||||||
|
|
||||||
#### LDAP / FreeIPA (Optional)
|
#### LDAP / FreeIPA (Optional)
|
||||||
|
|
||||||
For LDAP authentication, you need an LDAP server with user and group entries. Options:
|
For LDAP authentication, you need an LDAP server with user and group entries. Options:
|
||||||
@@ -339,10 +319,10 @@ The script:
|
|||||||
4. Clones the repository
|
4. Clones the repository
|
||||||
5. Creates the environment file template
|
5. Creates the environment file template
|
||||||
|
|
||||||
To override the default service hostnames:
|
To override the default database hostname:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
SILO_DB_HOST=db.example.com SILO_MINIO_HOST=s3.example.com sudo -E bash scripts/setup-host.sh
|
SILO_DB_HOST=db.example.com sudo -E bash scripts/setup-host.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
### B.5 Configure Credentials
|
### B.5 Configure Credentials
|
||||||
@@ -357,10 +337,6 @@ sudo nano /etc/silo/silod.env
|
|||||||
# Database
|
# Database
|
||||||
SILO_DB_PASSWORD=your-database-password
|
SILO_DB_PASSWORD=your-database-password
|
||||||
|
|
||||||
# MinIO
|
|
||||||
SILO_MINIO_ACCESS_KEY=silouser
|
|
||||||
SILO_MINIO_SECRET_KEY=your-minio-secret
|
|
||||||
|
|
||||||
# Authentication
|
# Authentication
|
||||||
SILO_SESSION_SECRET=generate-a-long-random-string
|
SILO_SESSION_SECRET=generate-a-long-random-string
|
||||||
SILO_ADMIN_USERNAME=admin
|
SILO_ADMIN_USERNAME=admin
|
||||||
@@ -379,7 +355,7 @@ Review the server configuration:
|
|||||||
sudo nano /etc/silo/config.yaml
|
sudo nano /etc/silo/config.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
Update `database.host`, `storage.endpoint`, `server.base_url`, and authentication settings for your environment. See [CONFIGURATION.md](CONFIGURATION.md) for all options.
|
Update `database.host`, `storage.filesystem.root_dir`, `server.base_url`, and authentication settings for your environment. See [CONFIGURATION.md](CONFIGURATION.md) for all options.
|
||||||
|
|
||||||
### B.6 Deploy
|
### B.6 Deploy
|
||||||
|
|
||||||
@@ -412,10 +388,10 @@ sudo /opt/silo/src/scripts/deploy.sh --restart-only
|
|||||||
sudo /opt/silo/src/scripts/deploy.sh --status
|
sudo /opt/silo/src/scripts/deploy.sh --status
|
||||||
```
|
```
|
||||||
|
|
||||||
To override the target host or database host:
|
To override the target host:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
SILO_DEPLOY_TARGET=silo.example.com SILO_DB_HOST=db.example.com sudo -E scripts/deploy.sh
|
SILO_DEPLOY_TARGET=silo.example.com sudo -E scripts/deploy.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
### B.7 Set Up Nginx and TLS
|
### B.7 Set Up Nginx and TLS
|
||||||
|
|||||||
@@ -313,7 +313,7 @@ For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_
|
|||||||
- Rollback functionality
|
- Rollback functionality
|
||||||
|
|
||||||
#### File Management
|
#### File Management
|
||||||
- MinIO integration with versioning
|
- Filesystem-based file storage
|
||||||
- File upload/download via REST API
|
- File upload/download via REST API
|
||||||
- SHA256 checksums for integrity
|
- SHA256 checksums for integrity
|
||||||
- Storage path: `items/{partNumber}/rev{N}.FCStd`
|
- Storage path: `items/{partNumber}/rev{N}.FCStd`
|
||||||
@@ -377,8 +377,8 @@ For full SOLIDWORKS PDM comparison tables, see [GAP_ANALYSIS.md Appendix C](GAP_
|
|||||||
|
|
||||||
## Appendix B: Phase 1 Detailed Tasks
|
## Appendix B: Phase 1 Detailed Tasks
|
||||||
|
|
||||||
### 1.1 MinIO Integration -- COMPLETE
|
### 1.1 File Storage -- COMPLETE
|
||||||
- [x] MinIO service configured in Docker Compose
|
- [x] Filesystem storage backend
|
||||||
- [x] File upload via REST API
|
- [x] File upload via REST API
|
||||||
- [x] File download via REST API (latest and by revision)
|
- [x] File download via REST API (latest and by revision)
|
||||||
- [x] SHA256 checksums on upload
|
- [x] SHA256 checksums on upload
|
||||||
|
|||||||
@@ -49,9 +49,9 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb
|
|||||||
┌───────────────┴───────────────┐
|
┌───────────────┴───────────────┐
|
||||||
▼ ▼
|
▼ ▼
|
||||||
┌─────────────────────────┐ ┌─────────────────────────────┐
|
┌─────────────────────────┐ ┌─────────────────────────────┐
|
||||||
│ PostgreSQL │ │ MinIO │
|
│ PostgreSQL │ │ Local Filesystem │
|
||||||
│ (psql.example.internal)│ │ - File storage │
|
│ (psql.example.internal)│ │ - File storage │
|
||||||
│ - Item metadata │ │ - Versioned objects │
|
│ - Item metadata │ │ - Revision files │
|
||||||
│ - Relationships │ │ - Thumbnails │
|
│ - Relationships │ │ - Thumbnails │
|
||||||
│ - Revision history │ │ │
|
│ - Revision history │ │ │
|
||||||
│ - Auth / Sessions │ │ │
|
│ - Auth / Sessions │ │ │
|
||||||
@@ -64,7 +64,7 @@ Silo treats **part numbering schemas as configuration, not code**. Multiple numb
|
|||||||
| Component | Technology | Notes |
|
| Component | Technology | Notes |
|
||||||
|-----------|------------|-------|
|
|-----------|------------|-------|
|
||||||
| Database | PostgreSQL 16 | Existing instance at psql.example.internal |
|
| Database | PostgreSQL 16 | Existing instance at psql.example.internal |
|
||||||
| File Storage | MinIO | S3-compatible, versioning enabled |
|
| File Storage | Local filesystem | Files stored under configurable root directory |
|
||||||
| CLI & API Server | Go (1.24) | chi/v5 router, pgx/v5 driver, zerolog |
|
| CLI & API Server | Go (1.24) | chi/v5 router, pgx/v5 driver, zerolog |
|
||||||
| Authentication | Multi-backend | Local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak |
|
| Authentication | Multi-backend | Local (bcrypt), LDAP/FreeIPA, OIDC/Keycloak |
|
||||||
| Sessions | PostgreSQL pgxstore | alexedwards/scs, 24h lifetime |
|
| Sessions | PostgreSQL pgxstore | alexedwards/scs, 24h lifetime |
|
||||||
@@ -83,7 +83,7 @@ An **item** is the fundamental entity. Items have:
|
|||||||
- **Properties** (key-value pairs, schema-defined and custom)
|
- **Properties** (key-value pairs, schema-defined and custom)
|
||||||
- **Relationships** to other items
|
- **Relationships** to other items
|
||||||
- **Revisions** (append-only history)
|
- **Revisions** (append-only history)
|
||||||
- **Files** (optional, stored in MinIO)
|
- **Files** (optional, stored on the local filesystem)
|
||||||
- **Location** (optional physical inventory location)
|
- **Location** (optional physical inventory location)
|
||||||
|
|
||||||
### 3.2 Database Schema (Conceptual)
|
### 3.2 Database Schema (Conceptual)
|
||||||
@@ -115,7 +115,7 @@ CREATE TABLE revisions (
|
|||||||
item_id UUID REFERENCES items(id) NOT NULL,
|
item_id UUID REFERENCES items(id) NOT NULL,
|
||||||
revision_number INTEGER NOT NULL,
|
revision_number INTEGER NOT NULL,
|
||||||
properties JSONB NOT NULL, -- all properties at this revision
|
properties JSONB NOT NULL, -- all properties at this revision
|
||||||
file_version TEXT, -- MinIO version ID if applicable
|
file_version TEXT, -- storage version ID if applicable
|
||||||
created_at TIMESTAMPTZ DEFAULT now(),
|
created_at TIMESTAMPTZ DEFAULT now(),
|
||||||
created_by TEXT, -- user identifier (future: LDAP DN)
|
created_by TEXT, -- user identifier (future: LDAP DN)
|
||||||
comment TEXT,
|
comment TEXT,
|
||||||
@@ -345,7 +345,7 @@ CAD workbench and spreadsheet extension implementations are maintained in separa
|
|||||||
|
|
||||||
### 5.1 File Storage Strategy
|
### 5.1 File Storage Strategy
|
||||||
|
|
||||||
Files are stored as whole objects in MinIO with versioning enabled. Storage path convention: `items/{partNumber}/rev{N}.ext`. SHA-256 checksums are captured on upload for integrity verification.
|
Files are stored on the local filesystem under a configurable root directory. Storage path convention: `items/{partNumber}/rev{N}.ext`. SHA-256 checksums are captured on upload for integrity verification.
|
||||||
|
|
||||||
Future option: exploded storage (unpack ZIP-based CAD archives for better diffing).
|
Future option: exploded storage (unpack ZIP-based CAD archives for better diffing).
|
||||||
|
|
||||||
@@ -439,7 +439,7 @@ Revisions are created explicitly by user action (not automatic):
|
|||||||
### 7.3 Revision vs. File Version
|
### 7.3 Revision vs. File Version
|
||||||
|
|
||||||
- **Revision**: Silo metadata revision (tracked in PostgreSQL)
|
- **Revision**: Silo metadata revision (tracked in PostgreSQL)
|
||||||
- **File Version**: MinIO object version (automatic on upload)
|
- **File Version**: File on disk corresponding to a revision
|
||||||
|
|
||||||
A single Silo revision may span multiple file uploads during editing. Only committed revisions create formal revision records.
|
A single Silo revision may span multiple file uploads during editing. Only committed revisions create formal revision records.
|
||||||
|
|
||||||
@@ -603,7 +603,7 @@ See [AUTH.md](AUTH.md) for full architecture details and [AUTH_USER_GUIDE.md](AU
|
|||||||
```
|
```
|
||||||
# Health (no auth)
|
# Health (no auth)
|
||||||
GET /health # Basic health check
|
GET /health # Basic health check
|
||||||
GET /ready # Readiness (DB + MinIO)
|
GET /ready # Readiness (DB)
|
||||||
|
|
||||||
# Auth (no auth required)
|
# Auth (no auth required)
|
||||||
GET /login # Login page
|
GET /login # Login page
|
||||||
@@ -624,8 +624,8 @@ GET /api/auth/tokens # List user's API to
|
|||||||
POST /api/auth/tokens # Create API token
|
POST /api/auth/tokens # Create API token
|
||||||
DELETE /api/auth/tokens/{id} # Revoke API token
|
DELETE /api/auth/tokens/{id} # Revoke API token
|
||||||
|
|
||||||
# Presigned Uploads (editor)
|
# Direct Uploads (editor)
|
||||||
POST /api/uploads/presign # Get presigned MinIO upload URL [editor]
|
POST /api/uploads/presign # Get upload URL [editor]
|
||||||
|
|
||||||
# Schemas (read: viewer, write: editor)
|
# Schemas (read: viewer, write: editor)
|
||||||
GET /api/schemas # List all schemas
|
GET /api/schemas # List all schemas
|
||||||
@@ -744,7 +744,7 @@ POST /api/inventory/{partNumber}/move
|
|||||||
- [x] Part number generation engine
|
- [x] Part number generation engine
|
||||||
- [x] CLI tool (`cmd/silo`)
|
- [x] CLI tool (`cmd/silo`)
|
||||||
- [x] API server (`cmd/silod`) with 78 endpoints
|
- [x] API server (`cmd/silod`) with 78 endpoints
|
||||||
- [x] MinIO integration for file storage with versioning
|
- [x] Filesystem-based file storage
|
||||||
- [x] BOM relationships (component, alternate, reference)
|
- [x] BOM relationships (component, alternate, reference)
|
||||||
- [x] Multi-level BOM (recursive expansion with configurable depth)
|
- [x] Multi-level BOM (recursive expansion with configurable depth)
|
||||||
- [x] Where-used queries (reverse parent lookup)
|
- [x] Where-used queries (reverse parent lookup)
|
||||||
|
|||||||
@@ -15,7 +15,7 @@
|
|||||||
| Part number generator | Complete | Scoped sequences, category-based format |
|
| Part number generator | Complete | Scoped sequences, category-based format |
|
||||||
| API server (`silod`) | Complete | 78 REST endpoints via chi/v5 |
|
| API server (`silod`) | Complete | 78 REST endpoints via chi/v5 |
|
||||||
| CLI tool (`silo`) | Complete | Item registration and management |
|
| CLI tool (`silo`) | Complete | Item registration and management |
|
||||||
| MinIO file storage | Complete | Upload, download, versioning, checksums |
|
| Filesystem file storage | Complete | Upload, download, checksums |
|
||||||
| Revision control | Complete | Append-only history, rollback, comparison, status/labels |
|
| Revision control | Complete | Append-only history, rollback, comparison, status/labels |
|
||||||
| Project management | Complete | CRUD, many-to-many item tagging |
|
| Project management | Complete | CRUD, many-to-many item tagging |
|
||||||
| CSV import/export | Complete | Dry-run validation, template generation |
|
| CSV import/export | Complete | Dry-run validation, template generation |
|
||||||
@@ -29,7 +29,7 @@
|
|||||||
| CSRF protection | Complete | nosurf on web forms |
|
| CSRF protection | Complete | nosurf on web forms |
|
||||||
| Fuzzy search | Complete | sahilm/fuzzy library |
|
| Fuzzy search | Complete | sahilm/fuzzy library |
|
||||||
| Web UI | Complete | React SPA (Vite + TypeScript), 6 pages, Catppuccin Mocha theme |
|
| Web UI | Complete | React SPA (Vite + TypeScript), 6 pages, Catppuccin Mocha theme |
|
||||||
| File attachments | Complete | Presigned uploads, item file association, thumbnails |
|
| File attachments | Complete | Direct uploads, item file association, thumbnails |
|
||||||
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull are stubs |
|
| Odoo ERP integration | Partial | Config and sync-log CRUD functional; push/pull are stubs |
|
||||||
| Docker Compose | Complete | Dev and production configurations |
|
| Docker Compose | Complete | Dev and production configurations |
|
||||||
| Deployment scripts | Complete | setup-host, deploy, init-db, setup-ipa-nginx |
|
| Deployment scripts | Complete | setup-host, deploy, init-db, setup-ipa-nginx |
|
||||||
@@ -56,7 +56,7 @@ FreeCAD workbench and LibreOffice Calc extension are maintained in separate repo
|
|||||||
| Service | Host | Status |
|
| Service | Host | Status |
|
||||||
|---------|------|--------|
|
|---------|------|--------|
|
||||||
| PostgreSQL | psql.example.internal:5432 | Running |
|
| PostgreSQL | psql.example.internal:5432 | Running |
|
||||||
| MinIO | localhost:9000 (API) / :9001 (console) | Configured |
|
| File Storage | /opt/silo/data (filesystem) | Configured |
|
||||||
| Silo API | localhost:8080 | Builds successfully |
|
| Silo API | localhost:8080 | Builds successfully |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
Reference in New Issue
Block a user