kindred/silo
1
1
Fork 0
Code Issues 30 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(storage): FileStore interface abstraction + filesystem backend #134

Merged
forbes merged 2 commits from feat-storage-interface-filesystem into main 2026-02-17 17:55:10 +00:00
Conversation 0 Commits 2 Files Changed 9 +545 -28
forbes commented 2026-02-17 17:50:12 +00:00
Owner
Copy Link

Summary

Introduces a FileStore interface to decouple the API layer from MinIO, and implements a local filesystem backend as an alternative storage option.

Two commits, two issues:

Commit 1 — FileStore interface (#126)

Extracts the storage contract into a FileStore interface so handlers work with any backend:

  • FileStore interface in internal/storage/interface.go with 8 methods: Put, Get, GetVersion, Delete, Exists, Copy, PresignPut, Ping
  • Added Exists method to the MinIO Storage struct (via StatObject)
  • Server.storage and ServerState.storage now hold storage.FileStore instead of *storage.Storage
  • StorageConfig gains Backend ("minio" / "filesystem") and Filesystem.RootDir fields
  • main.go uses a backend selection switch — defaults to "minio" for backward compatibility
  • Nil interface safety: store is declared as storage.FileStore so it stays a true nil when unconfigured (avoids the typed-nil-pointer-in-interface gotcha)

Commit 2 — Filesystem backend (#127)

FilesystemStore implementing FileStore for local disk:

  • Atomic writes via temp file + os.Rename — no partial files on crash/failure
  • SHA-256 checksum computed during Put via io.MultiWriter
  • Get/GetVersion return *os.File (GetVersion ignores versionID — no versioning on filesystem)
  • Delete is idempotent (no error if file already gone)
  • Copy uses same atomic write pattern
  • PresignPut returns ErrPresignNotSupported (handled by #129)
  • Ping verifies root directory is writable
  • Wired into main.go backend switch
  • 14 unit tests covering all methods including atomicity, missing files, overwrite, bad root

Config

storage:
  backend: "filesystem"
  filesystem:
    root_dir: "/var/lib/silo/objects"

Or keep using MinIO (the default when backend is omitted):

storage:
  endpoint: "localhost:9000"
  # ...existing MinIO config...

Files changed

File Change
internal/storage/interface.go New — FileStore interface
internal/storage/filesystem.go New — FilesystemStore implementation
internal/storage/filesystem_test.go New — 14 unit tests
internal/storage/storage.go Added Exists method + compile-time check
internal/config/config.go Added Backend, FilesystemConfig
internal/api/handlers.go Server.storage → FileStore interface
internal/api/servermode.go ServerState.storage → FileStore interface
cmd/silod/main.go Backend selection switch, filesystem wiring
config.example.yaml Added backend field + filesystem section

Testing

=== RUN   TestNewFilesystemStore    --- PASS
=== RUN   TestPut                   --- PASS
=== RUN   TestPutAtomicity          --- PASS
=== RUN   TestGet                   --- PASS
=== RUN   TestGetMissing            --- PASS
=== RUN   TestGetVersion            --- PASS
=== RUN   TestDelete                --- PASS
=== RUN   TestDeleteMissing         --- PASS
=== RUN   TestExists                --- PASS
=== RUN   TestCopy                  --- PASS
=== RUN   TestPresignPut            --- PASS
=== RUN   TestPing                  --- PASS
=== RUN   TestPingBadRoot           --- PASS
=== RUN   TestPutOverwrite          --- PASS
PASS

go build ./... and go vet ./... pass cleanly.

Closes

Part of

Storage Migration: MinIO → PostgreSQL + Filesystem

## Summary Introduces a `FileStore` interface to decouple the API layer from MinIO, and implements a local filesystem backend as an alternative storage option. **Two commits, two issues:** ### Commit 1 — FileStore interface (#126) Extracts the storage contract into a `FileStore` interface so handlers work with any backend: - `FileStore` interface in `internal/storage/interface.go` with 8 methods: `Put`, `Get`, `GetVersion`, `Delete`, `Exists`, `Copy`, `PresignPut`, `Ping` - Added `Exists` method to the MinIO `Storage` struct (via `StatObject`) - `Server.storage` and `ServerState.storage` now hold `storage.FileStore` instead of `*storage.Storage` - `StorageConfig` gains `Backend` (`"minio"` / `"filesystem"`) and `Filesystem.RootDir` fields - `main.go` uses a backend selection switch — defaults to `"minio"` for backward compatibility - **Nil interface safety**: `store` is declared as `storage.FileStore` so it stays a true nil when unconfigured (avoids the typed-nil-pointer-in-interface gotcha) ### Commit 2 — Filesystem backend (#127) `FilesystemStore` implementing `FileStore` for local disk: - **Atomic writes** via temp file + `os.Rename` — no partial files on crash/failure - **SHA-256 checksum** computed during `Put` via `io.MultiWriter` - `Get`/`GetVersion` return `*os.File` (`GetVersion` ignores versionID — no versioning on filesystem) - `Delete` is idempotent (no error if file already gone) - `Copy` uses same atomic write pattern - `PresignPut` returns `ErrPresignNotSupported` (handled by #129) - `Ping` verifies root directory is writable - Wired into `main.go` backend switch - **14 unit tests** covering all methods including atomicity, missing files, overwrite, bad root ## Config ```yaml storage: backend: "filesystem" filesystem: root_dir: "/var/lib/silo/objects" ``` Or keep using MinIO (the default when `backend` is omitted): ```yaml storage: endpoint: "localhost:9000" # ...existing MinIO config... ``` ## Files changed | File | Change | |------|--------| | `internal/storage/interface.go` | **New** — `FileStore` interface | | `internal/storage/filesystem.go` | **New** — `FilesystemStore` implementation | | `internal/storage/filesystem_test.go` | **New** — 14 unit tests | | `internal/storage/storage.go` | Added `Exists` method + compile-time check | | `internal/config/config.go` | Added `Backend`, `FilesystemConfig` | | `internal/api/handlers.go` | `Server.storage` → `FileStore` interface | | `internal/api/servermode.go` | `ServerState.storage` → `FileStore` interface | | `cmd/silod/main.go` | Backend selection switch, filesystem wiring | | `config.example.yaml` | Added `backend` field + filesystem section | ## Testing ``` === RUN TestNewFilesystemStore --- PASS === RUN TestPut --- PASS === RUN TestPutAtomicity --- PASS === RUN TestGet --- PASS === RUN TestGetMissing --- PASS === RUN TestGetVersion --- PASS === RUN TestDelete --- PASS === RUN TestDeleteMissing --- PASS === RUN TestExists --- PASS === RUN TestCopy --- PASS === RUN TestPresignPut --- PASS === RUN TestPing --- PASS === RUN TestPingBadRoot --- PASS === RUN TestPutOverwrite --- PASS PASS ``` `go build ./...` and `go vet ./...` pass cleanly. ## Closes - Closes #126 - Closes #127 ## Part of Storage Migration: MinIO → PostgreSQL + Filesystem
forbes added 2 commits 2026-02-17 17:50:13 +00:00
feat(storage): define FileStore interface and refactor to use it b531617e39 
Extract a FileStore interface from the concrete *storage.Storage MinIO
wrapper so the API layer is storage-backend agnostic.

- Define FileStore interface in internal/storage/interface.go
- Add Exists method to MinIO Storage (via StatObject)
- Add compile-time interface satisfaction check
- Change Server.storage and ServerState.storage to FileStore interface
- Update NewServer and NewServerState signatures
- Add Backend and FilesystemConfig fields to StorageConfig
- Add backend selection switch in main.go (minio/filesystem/unknown)
- Update config.example.yaml with backend field

The nil-interface pattern is preserved: when storage is unconfigured,
store remains a true nil FileStore (not a typed nil pointer), so all
existing if s.storage == nil checks continue to work correctly.

Closes #126
feat(storage): implement filesystem backend 9f347e7898 
Implement FilesystemStore satisfying the FileStore interface for local
filesystem storage, replacing MinIO for simpler deployments.

- Atomic writes via temp file + os.Rename (no partial files)
- SHA-256 checksum computed on Put via io.MultiWriter
- Get/GetVersion return os.File (GetVersion ignores versionID)
- Delete is idempotent (no error if file missing)
- Copy uses same atomic write pattern
- PresignPut returns ErrPresignNotSupported
- Ping verifies root directory is writable
- Wire NewFilesystemStore in main.go backend switch
- 14 unit tests covering all methods including atomicity

Closes #127
forbes merged commit 7a9dd057a5 into main 2026-02-17 17:55:10 +00:00
forbes deleted branch feat-storage-interface-filesystem 2026-02-17 17:55:10 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
forbes
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: kindred/silo#134
Delete Branch "feat-storage-interface-filesystem"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?