MimeLens

MimeLens is a family of small BERT-style encoders trained to classify binary content from a byte window taken at any offset in a file. The same model can work on streaming bodies, carved fragments without headers, random disk blocks, and packet payloads.

The public project includes 28 released checkpoints and a reference training stack covering byte and binary-BPE models, masked-language-model pretraining, packed-token data, Rust hot paths, run configurations, and tests. The complete mixed-provenance training and evaluation corpora cannot be redistributed; the repository states that limitation and distinguishes the released reproducible surface from an exact data rerun.

The companion paper reports accuracy, robustness, and latency results against Magika, libmagic, and TrID and is available on arXiv.