English | 中文

A drop-in replacement for easy_reader that matches CLI throughput.

easy_reader provides bidirectional line navigation for large files — forward, backward, random access. But its forward reads are ~200× slower than wc -l, because each next_line() call issues 2–3 seek+read syscalls per line.

fast_reader keeps the same API surface, adds a persistent 64KB read buffer, and reduces forward-reading syscalls to ~0.003 per line — matching std::io::BufRead and GNU coreutils.

Add to your Cargo.toml:

[dependencies]
fast_reader = "0.1"

use fast_reader::FastReader;
use std::fs::File;

let file = File::open("huge.log")?;
let mut r = FastReader::new(file)?;

// Forward — buffered, ~1 seek per 377 lines
while let Some(line) = r.next_line()? {
    println!("{line}");
}

// Backward — window-based, O(tail) not O(N)
r.eof();
while let Some(line) = r.prev_line()? {
    println!("{line}");
}

// Jump to arbitrary line (O(1), index built incrementally)
r.jump_to_line(500_000)?;

// Random sampling (single seek, O(1))
let line = r.random_line()?;

The root cause: easy_reader has no persistent buffer. Each next_line() → find_start_line() + find_end_line() + read_bytes(), each doing independent seek+read. On a 144MB file, counting lines takes 3.6 seconds via easy_reader — vs 80ms with wc -l.

1M-line JSONL file (~144MB), Windows 11, SSD.

• fast_reader wins on tail, reverse, random sampling (the operations easy_reader was designed for)
• BufRead/coreutils win on sequential full scans (which is expected — they have no bidirectional overhead)
• fast_reader is within 6–9% of BufRead on sequential workloads, while adding O(1) reverse/random

Full easy_reader compatibility plus extras:

This codebase was written with AI assistance — 100% AI-generated (with human review). It is evidence for quality-assured engineering with AI tooling.

MIT