Write-ahead log & changefeed
How LoonFS orders atomic commits in the WAL and exposes their semantic filesystem events through the change feed.
Understanding the write-ahead log
Section titled “Understanding the write-ahead log”The write-ahead log (WAL) is the ordered record of committed changes in a namespace. Each logical commit receives one monotonically increasing sequence number. LoonFS may group several logical commits into one physical WAL segment to improve throughput, but the commit order is always maintained.
┌──────────────────────────────────────────────────────────────────────┐ │ Write-ahead log (WAL) │ │ logical commits, ordered by seq │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ seq 417 │ │ seq 418 │ │ seq 419 │ │ seq 420 │ │ seq 421 │ │ │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ │ │ └──────────────────────────────────────────────────────────────────────┘
Commits
Section titled “Commits”Commits are the unit of atomic namespace change. A request supplies a commit_id, optional message and preconditions, and an ordered list of operations, with attribution in the Loonfs-Actor header. The response and change feed record that actor ID as committed_by. Every operation in the list commits transactionally (all-or-nothing).
┌──────────────────────────────────────────────────────────────────────┐
│ Write-ahead log (WAL) │
│ │
│ ┌──────────┐ ┌──────────┐ ┏━━━━━━━━━━┓ ┌──────────┐ ┌──────────┐ │
│ │ seq 417 │ │ seq 418 │ ┃ seq 419 ┃ │ seq 420 │ │ seq 421 │ │
│ └──────────┘ └──────────┘ ┗━━━━━━━━━━┛ └──────────┘ └──────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────┐
│ Commit · seq 419 │
│ │
│ ╔════════════════════════════════════════════════════════════════╗ │
│ ║ Operation 0 · create_directory ║ │
│ ║ path /docs ║ │
│ ╠════════════════════════════════════════════════════════════════╣ │
│ ║ Operation 1 · put_file ║ │
│ ║ path /docs/report.txt ║ │
│ ╚════════════════════════════════════════════════════════════════╝ │
│ │
│ │ │
│ ▼ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ Change-feed events │ │
│ │ 0 · directory_created · ino_41 under ino_1 · "docs" │ │
│ │ 1 · file_created · ino_42 under ino_41 · "report.txt" │ │
│ └────────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────┘
In this example, the second operation (operation 1) can operate on the directory created by the first (operation 0). If either operation fails, the full transaction fails and nothing is committed.
Filesystem operations
Section titled “Filesystem operations”Operations are the client-facing semantic mutations inside a commit. Path-oriented operations include create_directory, put_file, delete_path, move_path, copy_path, undelete, restore_revision, update_attributes, and update_access. Identity-oriented clients can also use create_directory_by_inode, create_file_by_inode, put_file_revision_by_inode, delete_by_inode, and move_by_inode.
Request-level preconditions check the state before any operation runs. Operation-level guards check the state at that operation, including earlier changes in the same commit. For example, replacing a file can require both expected_inode_id and expected_revision_no; inode-addressed moves and deletes require expected_binding_version. The server also checks destination availability and ancestor visibility, and rejects the whole commit if a precondition or operation fails.
See also Transactions.
Internally, LoonFS materializes applied operations into lower-level, replayable WAL deltas. Those deltas are part of the durable storage format, not the commit API or change-feed contract: clients submit operations and consume semantic events.
Changefeed
Section titled “Changefeed”Because the WAL is an append-only record of committed changes, it doubles as the source for the namespace change feed. A client can read it using List changes after a sequence. The CLI also exposes the changefeed via loonfs changes.
The changefeed is composed of the following event types:
| Event | Details |
|---|---|
directory_created | inode_id, parent_inode_id, display_name, and the new binding_version |
file_created | inode_id, parent_inode_id, display_name, new binding_version, initial content_ref, and revision_no |
content_changed | inode_id, content_ref, revision_no — a put over an existing file, or a revision restore |
moved | inode_id, source and destination parent inode IDs and display names, and the new binding_version |
deleted | inode_id and the removed deleted_binding |
undeleted | inode_id, parent_inode_id, display_name, and the new binding_version |
attributes_changed | inode_id, the whole attributes map after the update, and attributes_revision_no |
access_changed | inode_id, boundary, the complete direct grants, and access_revision_no |
Changefeed usage
Section titled “Changefeed usage”Common use cases for the changefeed include maintaining a local filesystem mirror, maintaining a derivative index, and triggering processing pipelines when files are updated.
Pass a live snapshot_id to List changes to limit the feed to that snapshot’s captured sequence. This provides consumers a fixed (non-infinite) endpoint even as new commits stream in. See Snapshots.
Changefeed consumers can process only the event kinds relevant to their projection. For example, the native search index (“grep”) reacts to file_created and content_changed events.