see benchmark

MOSAIC: A Benchmark for Authentic Language Data

Data annotation providers routinely describe their output as "high quality," "native-level," or "human-verified"  descriptions that carry no measurable definition and cannot be independently checked. This is a particular problem for code-switched and dialect-heavy languages, where the gap between grammatically correct text and authentic, community-real communication is largest. A dataset can be perfectly correct and still fail to represent how people in Bihar, Chennai, or Jakarta actually speak day to day — mixing languages mid-sentence, using regional vocabulary, and leaving in the natural imperfection of real speech. MOSAIC was built to close that gap: to give "authentic language data" an actual, reproducible measurement rather than a marketing description.

MOSAIC (v1.0) is a multi-dimensional scoring framework developed internally at Serideo AI. It evaluates a dataset across six weighted dimensions — Cultural Context, Natural Expression, Code-Switching Authenticity, Human Imperfection, Contextual & Semantic Fidelity, and Regional/Lexical Authenticity — combining independent human review with automated consistency checks into a single composite score, alongside dimension-level detail, reviewer agreement, and statistical confidence. Every reported score is accompanied by its sample size, confidence interval, and inter-rater reliability, so a MOSAIC result can be treated as evidence rather than a claim.

The first published MOSAIC run evaluated a conversational-text dataset in Maithili: a composite score of 76.8/100 (n = 200, 95% CI ± 2.1), with 88.2% inter-rater agreement across 4 independent reviewers. Against an internally constructed, unfiltered baseline in the same language category (70.4/100), the curated dataset scored higher on every dimension — a +6.4 point gap. Contextual & Semantic Fidelity scored highest (82); Human Imperfection scored lowest (72), marking it as the clearest target for the next curation pass.

MOSAIC is a proprietary Serideo research framework, not a third-party certification or an industry-wide standard, and this report reflects a single evaluation run rather than a permanent guarantee — results may vary by language, dialect, task type, and domain. The exact scoring formula, dimension weights, and calibration rubrics are not published. Serideo is extending MOSAIC to additional languages and dataset types as part of ongoing benchmarking work.

‍

Reseacher

serideo team

Date

August 2026