kindred/silo
1
1
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity

chore(storage): remove MinIO dependency and clean up #133

New Issue
Closed
opened 2026-02-17 16:13:12 +00:00 by forbes · 0 comments
forbes commented 2026-02-17 16:13:12 +00:00
Owner
Copy Link

Summary

Remove the MinIO SDK dependency and all MinIO-specific code after the filesystem migration is verified in production.

Context

Once all files have been migrated to the filesystem backend (#130) and the new upload/download flows are working (#129), the MinIO code becomes dead weight. This cleanup issue should only be executed after production verification.

Prerequisites — verify before starting

  • All files migrated to filesystem (confirmed via migration script report)
  • No storage_backend = 'minio' rows remaining in item_files or revisions tables
  • Application running successfully with storage.backend: "filesystem" in production
  • At least one full backup cycle completed with filesystem backend

Requirements

1. Remove MinIO Go SDK

Remove from go.mod:

github.com/minio/minio-go/v7 v7.0.66

And transitive dependencies that are only needed by MinIO:

github.com/minio/md5-simd v1.1.2
github.com/minio/sha256-simd v1.0.1

Run go mod tidy to clean up.

2. Remove MinIO implementation

  • Delete or gut internal/storage/storage.go (the MinIO Storage struct and all its methods: Connect, Put, Get, GetVersion, Delete, Ping, Bucket, PresignPut, Copy)
  • Keep internal/storage/interface.go (FileStore interface) and internal/storage/filesystem.go
  • Keep FileKey() and ThumbnailKey() standalone helpers if still used
  • Remove the Config struct fields specific to MinIO (Endpoint, AccessKey, SecretKey, Bucket, UseSSL, Region) from internal/config/config.go — or deprecate them

3. Remove MinIO from Docker Compose

  • deployments/docker-compose.yaml: Remove minio service and minio_data volume
  • deployments/docker-compose.allinone.yaml: Remove minio service and minio_data volume
  • Remove SILO_MINIO_ENDPOINT, SILO_MINIO_ACCESS_KEY, SILO_MINIO_SECRET_KEY env vars from silo service definitions
  • Update deployments/config.dev.yaml to remove MinIO config

4. Remove MinIO environment variable handling

In internal/config/config.go, the Load() function has env var overrides:

if v := os.Getenv("SILO_MINIO_ENDPOINT"); v != "" { cfg.Storage.Endpoint = v }
if v := os.Getenv("SILO_MINIO_ACCESS_KEY"); v != "" { cfg.Storage.AccessKey = v }
if v := os.Getenv("SILO_MINIO_SECRET_KEY"); v != "" { cfg.Storage.SecretKey = v }

Remove these.

5. Simplify startup wiring

In cmd/silod/main.go (lines 68-86), the storage initialization currently:

  1. Checks if cfg.Storage.Endpoint is non-empty
  2. Calls storage.Connect() to create MinIO client
  3. Falls back to nil on failure

Replace with:

  1. Check cfg.Storage.Backend (or cfg.Storage.Filesystem.Root)
  2. Call storage.NewFilesystemStore(root)
  3. Fail hard if filesystem root is invalid (no graceful nil fallback — storage is required)

6. Remove presigned upload endpoint

If not already removed in #129:

  • Remove HandlePresignUpload handler from internal/api/file_handlers.go
  • Remove POST /api/uploads/presign route from internal/api/routes.go
  • Remove frontend presigned upload code from web/src/hooks/useFileUpload.ts if fully replaced

7. Remove migration script

Archive or remove cmd/silod/migrate.go (or cmd/silo-migrate/) — no longer needed after migration is complete.

8. Remove storage_backend columns (optional)

If all rows are 'filesystem', the storage_backend and file_storage_backend columns are unnecessary. Consider a migration to drop them, or leave them as a safety net.

9. Clean up config example

Update config.example.yaml to show only filesystem config (remove MinIO example or move to comments).

Files to modify/delete

  • internal/storage/storage.go — delete (MinIO implementation)
  • internal/config/config.go — remove MinIO config fields and env vars
  • cmd/silod/main.go — simplify storage init
  • internal/api/file_handlers.go — remove HandlePresignUpload
  • internal/api/routes.go — remove presign route
  • deployments/docker-compose.yaml — remove MinIO service
  • deployments/docker-compose.allinone.yaml — remove MinIO service
  • deployments/config.dev.yaml — remove MinIO config
  • config.example.yaml — simplify
  • go.mod, go.sum — remove MinIO SDK
  • web/src/hooks/useFileUpload.ts — remove presigned upload code
  • cmd/silod/migrate.go — archive/remove

Acceptance criteria

  • go build ./... passes with no MinIO references
  • go mod tidy shows no MinIO packages
  • Docker Compose starts without MinIO service
  • All file operations work with filesystem backend
  • No SILO_MINIO_* env vars referenced anywhere
  • Frontend upload works without presigned flow

Priority

P2 — execute only after production migration is verified

Depends on

  • #126 (FileStore interface)
  • #127 (filesystem backend)
  • #128 (metadata columns)
  • #129 (direct upload endpoint)
  • #130 (data migration complete)
  • #131 (docs updated)
  • #132 (thumbnails verified)

Part of

Storage Migration: MinIO → PostgreSQL + Filesystem

## Summary Remove the MinIO SDK dependency and all MinIO-specific code after the filesystem migration is verified in production. ## Context Once all files have been migrated to the filesystem backend (#130) and the new upload/download flows are working (#129), the MinIO code becomes dead weight. This cleanup issue should only be executed after production verification. ## Prerequisites — verify before starting - [ ] All files migrated to filesystem (confirmed via migration script report) - [ ] No `storage_backend = 'minio'` rows remaining in `item_files` or `revisions` tables - [ ] Application running successfully with `storage.backend: "filesystem"` in production - [ ] At least one full backup cycle completed with filesystem backend ## Requirements ### 1. Remove MinIO Go SDK Remove from `go.mod`: ``` github.com/minio/minio-go/v7 v7.0.66 ``` And transitive dependencies that are only needed by MinIO: ``` github.com/minio/md5-simd v1.1.2 github.com/minio/sha256-simd v1.0.1 ``` Run `go mod tidy` to clean up. ### 2. Remove MinIO implementation - Delete or gut `internal/storage/storage.go` (the MinIO `Storage` struct and all its methods: `Connect`, `Put`, `Get`, `GetVersion`, `Delete`, `Ping`, `Bucket`, `PresignPut`, `Copy`) - Keep `internal/storage/interface.go` (`FileStore` interface) and `internal/storage/filesystem.go` - Keep `FileKey()` and `ThumbnailKey()` standalone helpers if still used - Remove the `Config` struct fields specific to MinIO (`Endpoint`, `AccessKey`, `SecretKey`, `Bucket`, `UseSSL`, `Region`) from `internal/config/config.go` — or deprecate them ### 3. Remove MinIO from Docker Compose - `deployments/docker-compose.yaml`: Remove `minio` service and `minio_data` volume - `deployments/docker-compose.allinone.yaml`: Remove `minio` service and `minio_data` volume - Remove `SILO_MINIO_ENDPOINT`, `SILO_MINIO_ACCESS_KEY`, `SILO_MINIO_SECRET_KEY` env vars from silo service definitions - Update `deployments/config.dev.yaml` to remove MinIO config ### 4. Remove MinIO environment variable handling In `internal/config/config.go`, the `Load()` function has env var overrides: ```go if v := os.Getenv("SILO_MINIO_ENDPOINT"); v != "" { cfg.Storage.Endpoint = v } if v := os.Getenv("SILO_MINIO_ACCESS_KEY"); v != "" { cfg.Storage.AccessKey = v } if v := os.Getenv("SILO_MINIO_SECRET_KEY"); v != "" { cfg.Storage.SecretKey = v } ``` Remove these. ### 5. Simplify startup wiring In `cmd/silod/main.go` (lines 68-86), the storage initialization currently: 1. Checks if `cfg.Storage.Endpoint` is non-empty 2. Calls `storage.Connect()` to create MinIO client 3. Falls back to nil on failure Replace with: 1. Check `cfg.Storage.Backend` (or `cfg.Storage.Filesystem.Root`) 2. Call `storage.NewFilesystemStore(root)` 3. Fail hard if filesystem root is invalid (no graceful nil fallback — storage is required) ### 6. Remove presigned upload endpoint If not already removed in #129: - Remove `HandlePresignUpload` handler from `internal/api/file_handlers.go` - Remove `POST /api/uploads/presign` route from `internal/api/routes.go` - Remove frontend presigned upload code from `web/src/hooks/useFileUpload.ts` if fully replaced ### 7. Remove migration script Archive or remove `cmd/silod/migrate.go` (or `cmd/silo-migrate/`) — no longer needed after migration is complete. ### 8. Remove `storage_backend` columns (optional) If all rows are `'filesystem'`, the `storage_backend` and `file_storage_backend` columns are unnecessary. Consider a migration to drop them, or leave them as a safety net. ### 9. Clean up config example Update `config.example.yaml` to show only filesystem config (remove MinIO example or move to comments). ## Files to modify/delete - `internal/storage/storage.go` — delete (MinIO implementation) - `internal/config/config.go` — remove MinIO config fields and env vars - `cmd/silod/main.go` — simplify storage init - `internal/api/file_handlers.go` — remove `HandlePresignUpload` - `internal/api/routes.go` — remove presign route - `deployments/docker-compose.yaml` — remove MinIO service - `deployments/docker-compose.allinone.yaml` — remove MinIO service - `deployments/config.dev.yaml` — remove MinIO config - `config.example.yaml` — simplify - `go.mod`, `go.sum` — remove MinIO SDK - `web/src/hooks/useFileUpload.ts` — remove presigned upload code - `cmd/silod/migrate.go` — archive/remove ## Acceptance criteria - [ ] `go build ./...` passes with no MinIO references - [ ] `go mod tidy` shows no MinIO packages - [ ] Docker Compose starts without MinIO service - [ ] All file operations work with filesystem backend - [ ] No `SILO_MINIO_*` env vars referenced anywhere - [ ] Frontend upload works without presigned flow ## Priority P2 — execute only after production migration is verified ## Depends on - #126 (FileStore interface) - #127 (filesystem backend) - #128 (metadata columns) - #129 (direct upload endpoint) - #130 (data migration complete) - #131 (docs updated) - #132 (thumbnails verified) ## Part of Storage Migration: MinIO → PostgreSQL + Filesystem
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
forbes
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: kindred/silo#133
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?