Storage model

LoonFS is architected around S3-compatible storage as the durable store, with a fast SSD and memory cache for acceleration. In embedded mode, LoonFS coordinates transactions directly with the object store, and compaction and indexing must be explicitly triggered. In server mode, the LoonFS server serializes multiple writers into a single batched commit stream and runs background compaction and indexing.

Metadata and content

LoonFS is natively built on top of object storage, which stores both content and metadata. By default, both use the same object-store bucket.

Each namespace head.json points to a content store. File revisions refer to already-durable immutable objects by a random content ID, along with the byte length and integrity checksums.

Namespace metadata and content objects use separate layouts:

namespaces/{namespace_id}/
├── wal/
│   ├── head.json                 # WAL visibility head; updated with CAS
│   ├── floor.json                # WAL retention floor; updated with CAS
│   └── segments/
│       └── {start_seq:020}-{suffix}.wal.zst
├── metadata/
│   ├── root.json                 # metadata root; updated with CAS
│   ├── manifests/
│   │   └── {manifest_object_id}.manifest.json
│   └── tables/
│       └── {table_id}.sst.zst
├── checkpoints/
│   └── {checkpoint_id}.json
└── uploads/
  └── {upload_id}.json

content-stores/{content_store_id}/
└── objects/
  └── {content_id[4:5]}/       # "con_" occupies content_id[0:3]; so content_id[4:5] are the first two random characters in the ID
      └── {content_id[6:7]}/   # content_id[6:7] are the next two random characters in the ID
          └── {content_id}

The two random shard levels spread content across 65,536 leaf prefixes to avoid same-prefix rate limits in hot namespaces.