Swift bindings for fjall — a log-structured (LSM-tree), embeddable key-value storage engine written in Rust.
The API mirrors the fjall 3.x Rust API (Database, Keyspace, WriteBatch, Snapshot,
iterators, …) while staying idiomatic Swift: throws instead of Result, Data/String
keys and values, Sendable handles, and Swift naming conventions.
- Embedded key-value storage — a database is just a directory, no server
- Multiple keyspaces (like RocksDB column families), each an isolated LSM-tree
- Atomic cross-keyspace write batches
- Transactions: single-writer (serialized) and optimistic (SSI with conflict detection)
- Consistent point-in-time snapshots (MVCC)
- Range & prefix iteration, forwards and backwards
- LZ4 compression, optional key-value separation for large values
- Thread-safe:
Database,Keyspace,WriteBatchandSnapshotareSendable
Add the package to your Package.swift:
dependencies: [
.package(url: "https://github.com/lachenmayer/fjall-swift", from: "0.1.0")
]and depend on the Fjall product. On Apple platforms the Rust core ships as a prebuilt
XCFramework attached to each GitHub release — no Rust toolchain needed.
Note: until the first release is tagged, the package uses a locally built framework: run
scripts/build-xcframework.shfirst (requires a Rust toolchain).
On Linux, build the Rust static library and point the linker at it:
cargo build --manifest-path rust/Cargo.toml --release
swift build -Xlinker -L"$PWD/rust/target/release"import Fjall
// A database is a single directory on disk.
let db = try Database(path: ".fjall_data")
// A keyspace is an isolated collection of key-value pairs (its own LSM-tree).
let items = try db.keyspace("items")
// Keys and values are bytes (Data); String overloads encode UTF-8.
try items.insert("a", "hello")
let value = try items.getString("a") // "hello"
// Ranges and prefixes iterate in key order — forwards or backwards.
let iter = items.range(from: .included("a"), to: .excluded("f"))
while let pair = try iter.next() {
print(pair.keyString, pair.valueString)
}
// Atomic cross-keyspace batches.
let batch = db.batch()
try batch.insert("x", "1", into: items)
try batch.remove("a", from: items)
try batch.commit()
// Consistent point-in-time snapshots.
let snapshot = db.snapshot()
try items.insert("later", "...")
try snapshot.containsKey("later", in: items) // false
// Control durability explicitly.
try db.persist(.syncAll)fjall orders keys by raw byte comparison, so integer keys must be encoded
fixed-width and big-endian to iterate in numeric order — and signed integers
additionally need their sign bit flipped (otherwise negative values, whose high
bit is set, would sort after positive ones). Data(orderPreservingKey:) does
all of this for any FixedWidthInteger:
// Sequential IDs as keys:
try items.insert(Data(orderPreservingKey: UInt64(42)), payload)
// Iterates in numeric order: 1, 5, 42, 256, 1000, ...
while let pair = try items.iter().next() {
let id = UInt64(orderPreservingKey: pair.key)!
...
}
// Range scans take integer bounds directly:
let recent = try items.range(from: .included(UInt64(1_000))).collect()Stick to one integer type per keyspace: keys of different widths (e.g. a
UInt32 next to a UInt64) do not sort meaningfully against each other.
The last inserted ID can be recovered after reopening with
UInt64(orderPreservingKey: try items.last!.key).
For cross-keyspace transactions with read-your-own-writes semantics, open the database
as a TxDatabase (single-writer: write transactions are serialized) or an
OptimisticTxDatabase (optimistic concurrency: transactions run in parallel and commits
fail with FjallError.conflict when they collide):
let db = try TxDatabase(path: ".fjall_data")
let items = try db.keyspace("items")
// Scoped: commits on return, rolls back on throw.
try db.write { tx in
let value = try tx.getString("counter", in: items) ?? "0"
try tx.insert("counter", String(Int(value)! + 1), into: items)
}
// Or explicit:
let tx = try db.writeTransaction()
try tx.insert("a", "1", into: items)
try tx.commit() // or tx.rollback()
// Read-only transactions are snapshots.
let read = db.readTransaction()With OptimisticTxDatabase, conflicting transactions can be retried automatically:
let db = try OptimisticTxDatabase(path: ".fjall_data")
let items = try db.keyspace("items")
try db.write(attempts: 5) { tx in
let value = try tx.getString("counter", in: items) ?? "0"
try tx.insert("counter", String(Int(value)! + 1), into: items)
}Both give you serializable, cross-keyspace transactions with read-your-own-writes; they differ in how concurrent writers are handled.
TxDatabase (single-writer) |
OptimisticTxDatabase (SSI) |
|
|---|---|---|
| Concurrency model | Pessimistic: one write transaction at a time; starting a second blocks until the first finishes | Optimistic: write transactions run concurrently; conflicts are detected at commit |
| Commit | Always succeeds (barring I/O errors) | Throws FjallError.conflict if another transaction touched the same keys — retry the whole transaction (write(attempts:) does this for you) |
| Best for | Low write concurrency; workloads where retry logic is unwelcome; transactions that must not fail spuriously | Many concurrent writers that mostly touch different keys; read-heavy transactions |
| Worst case | A long transaction stalls every other writer (readers are never blocked) | High contention on the same keys wastes work: transactions repeatedly execute, conflict, and retry |
| Read transactions | Never block, in either mode (MVCC snapshots) | Same |
| Wrapper overhead | Each write transaction runs on a dedicated Rust worker thread, and every operation is a synchronous message to it (~µs). This is how the wrapper bridges fjall's lock-holding transaction type across the FFI boundary; it preserves fjall's exact blocking semantics, including rollback + lock release when a transaction is discarded uncommitted | None beyond the FFI call itself — the transaction object crosses the boundary directly |
Rules of thumb:
- Writers rarely overlap, or you want commits that never fail? →
TxDatabase. - Many writers hitting mostly-disjoint keys? →
OptimisticTxDatabasewithwrite(attempts:). - Under heavy same-key contention neither shines — single-writer serializes everything, optimistic burns retries — but single-writer at least guarantees forward progress for each transaction, so prefer it there.
- No transactions needed at all? Plain
DatabasewithWriteBatchis the fastest option: batches are atomic on commit, they just don't let you read within the batch.
One fjall-level caveat applies to both: transactions (and snapshots) pin old data — the garbage collector cannot reclaim versions a live transaction might still read — so keep them short-lived.
let db = try Database(
path: ".fjall_data",
options: .init(
cacheSize: 64 * 1_024 * 1_024, // 64 MiB block cache
temporary: false,
workerThreads: 4
)
)
let blobs = try db.keyspace(
"blobs",
options: .init(
maxMemtableSize: 32 * 1_024 * 1_024,
// Store large values out of line (recommended for blobs).
kvSeparation: KvSeparationOptions(separationThreshold: 4_096)
)
)All fallible operations use typed throws (throws(FjallError)), so the error type is
part of the signature and catch can be exhaustive. FjallError mirrors fjall::Error:
do {
try items.insert("a", "b")
} catch FjallError.locked {
// another process has the database open
} catch {
// .io, .storage, .poisoned, ... — `error` is a FjallError, not `any Error`
}The only exceptions are APIs that run your closure — db.write { … } and
Iter.forEach — which rethrow whatever the closure throws and are therefore
untyped throws.
┌────────────────────┐
│ Fjall │ hand-written Swift API (this is what you import)
├────────────────────┤
│ FjallFFI │ generated Swift bindings (UniFFI)
├────────────────────┤
│ CFjallFFI │ C symbols: XCFramework (Apple) / static lib (Linux)
├────────────────────┤
│ rust/ (fjall-ffi) │ Rust crate bridging fjall via UniFFI
└────────────────────┘
The Rust crate rust/ wraps fjall 3.x with FFI-friendly types and
exports them via UniFFI. The generated bindings are
committed at Sources/FjallFFI; the Fjall module wraps them in an idiomatic Swift API.
# Rust core tests
cargo test --manifest-path rust/Cargo.toml
# Regenerate Swift bindings after changing rust/src/lib.rs
scripts/generate-bindings.sh
# Swift tests on macOS (builds a native-only XCFramework first)
scripts/build-xcframework.sh --native
FJALL_USE_LOCAL_FRAMEWORK=1 swift test
# Swift tests on Linux
cargo build --manifest-path rust/Cargo.toml --release
swift test -Xlinker -L"$PWD/rust/target/release"Releases are cut with the Release GitHub Actions workflow, which builds the full
XCFramework (macOS + iOS + simulator), rewrites Package.swift with the artifact URL and
checksum, tags the version, and attaches the framework to the GitHub release.
- Bulk ingestion (
start_ingestion) - Compaction strategy / block policy configuration
- Closure-based atomic updates (
fetch_update/update_fetch) — use a transaction instead
Licensed under either of Apache License 2.0 or MIT license at your option — the same terms as fjall itself.
0 comments
log in to comment.