olmOCR-Bench 小页面尺寸样本解析从small_page_size_pg1_repeat1.md理解文档级 OCR 评测数据流【免费下载链接】olmocrToolkit for linearizing PDFs for LLM datasets/training项目地址: https://gitcode.com/GitHub_Trending/ol/olmocr导读本文围绕 olmOCR-Bench 评测框架中的一个具体样本文件——olmocr/bench/sample_data/olmocr_pipeline/small_page_size_pg1_repeat1.md 展开剖析这份 Markdown 输出从生成、命名到被评测消费的完整链路。读者将掌握 olmOCR-Bench 的目录约定、输出文件命名规则、单页 PDF 的转换流程、基准测试的判定机制以及小页面尺寸small page size这一特殊版面在 OCR 评测中被如何对待。一、样本文件在评测框架中的定位olmOCR-Bench 是 olmOCR 仓库中用于自动评估文档级 OCR 效果的基准测试体系其完整说明见 olmocr/bench/README.md。它通过逐页检查 PDF 上机器可判定的客观事实来给 OCR 系统打分评测结果只区分通过/失败刻意回避编辑距离这类软指标——因为对于多篇文章共存的页面语序交换可能依然正确但单字符差异却可能改变公式含义。本文的主角small_page_size_pg1_repeat1.md正是该体系的一个被测输出样例它位于 olmocr/bench/sample_data/olmocr_pipeline/ 目录下。该目录存放的是内置演示数据集中olmocr_pipeline官方 olmOCR 流水线这一候选方法candidate的输出结果与同级目录下的 olmocr/bench/sample_data/dataset.jsonl测试用例定义和pdfs/原始 PDF共同构成一套可直接运行的迷你评测数据。文件名即元数据文件名的每个字段都携带评测所需的信息{pdf文件名}_pg{页码}_repeat{重复次数}.md对照small_page_size_pg1_repeat1.md字段值含义源 PDFsmall_page_size.pdf被测文档见 tests/gnarly_pdfs/small_page_size.pdfpg1第 1 页单页 PDF页码从 1 开始repeat1第 1 次重复同一页面可多次重复推理以评估稳定性该命名规则由两处代码共同约定转换阶段在 olmocr/bench/convert.py 中生成f{base_name}_pg{page_num}_repeat{repeat}.md而评测阶段在 olmocr/bench/benchmark.py 中通过正则^{md_base}_pg\d_repeat\d\.md$匹配这些文件。repeat机制的价值在于单个 OCR 系统存在随机性多次重复取多数通过可以显著提升分数稳定性——olmocr/bench/benchmark.py 中final_passed test_avg 0.5即超过一半重复通过才算通过。二、样本内容逐段解读小页面输出的典型形态该样本是 olmOCR 对small_page_size.pdf第 1 页的 OCR 输出内容为一部 19 世纪农业著作中关于骨粉bones作为肥料使用的章节。完整内容如下any—was very trifling. Since the use of bones has, however, become general, the turnip crop has been, in many instances, ten-fold, and in few less than four or five-fold its former bulk. All the succeeding crops of grain and seeds have been amazingly increased, and, upon the four or five-shift system, there is no doubt the land will go on progressively improving, requiring a less quantity of bones annually, from its increased fertility and power.On light loams, the returns to the Doncaster Committee give bones a preference to farm-yard dung. And we learn that, upon the calcareous soil of the Yorkshire Wolds, heavy crops of turnips have been raised from 16 bushels per acre of bones, while in the same field, and under similar circumstances, but manured from the farm-yard at the rate of from 8 to 10 tons per acre, the turnips have been of the most inferior description.On peat soils, if previously drained and laid dry, their advantages are reported to be so striking, that from fifteen to twenty bushels of dust per acre, drilled, have been also found to very far surpass the ordinary dressing of stable-dung, and even of lime and pigeons-dung.On gravels, the reports are meagre and contradictory, though perhaps reconcilable in principle, as it has been justly observed, that a gravelly soil may embrace every variety of texture and quality, from the light dry sand to the water-logged yellow clay—preserving in each the necessary admixture of stones and grit. To wet gravel, their application has been found decidedly unfavourable.ANALYSIS.An examination of the component parts of soils, and of the power of bones, when applied to them as manure, would go far to explain the irregularity of their different effects upon various kinds of soil. Bone is known to consist of about equal parts of earthy and animal matter; the former chiefly composed of gypsum—which is of so indestructible a nature as to have been termed, by early chemists, the earth of bones—and a small portion of carbonate of lime; from which we may conclude that probably half the weight of bones is in the greater part consumed by plants as direct nourishment in their state of growth, and that the remainder is more gradually absorbed by the soil, as well also as by the plants; for lime, though in small amount, is always present, in greater or less quantity, in all vegetable substances.The quantity of earthy matter varies according to the age of the animal; and, in like manner, the quantity of animal matter varies also in proportion to the condition of the animal. In the best kinds of bones for manure, viz., those from fat young animals, perhaps the following proportions may give an approximation to the relative quantities of each in 100 parts:| Earthy and saline matter | 40 | | Cartilage and jelly | 40 | | Fatty matter | 20 |The soft parts thus form, in the best bone, about sixty, and upon an average, perhaps, amount to fifty per cent., which are almost entirely constituted of the same elements of plants, and all of them, sooner or later, liable to be dissolved and absorbed by the roots. The cartilage, indeed, when the bones have been buried in a dry situation, is very indestructible; but when exposed to the action of air, water, soil, and vegetation, will probably pass into the state of jelly, and be dissolved, or otherwise decomposed,Doncaster Report, p. 8.从这份输出可以观察到一个高质量文档级 OCR 应具备的要素正文自然阅读顺序四段关于不同土壤类型light loams / peat soils / gravels的论述保持原始版式顺序这对应 olmOCR-Bench 的 Natural Reading Order 测试类型Markdown 结构化标记**ANALYSIS.**使用粗体语法呈现小节标题Markdown 表格将原文的每 100 份骨粉成分比例Earthy and saline matter 40 / Cartilage and jelly 40 / Fatty matter 20转换为标准 Markdown 管道表格——这正是 olmOCR-Bench 明确接受的表格形式之一README 规定 Tables can be in either Markdown syntax, or as an htmltable脚注保留末尾的* Doncaster Report, p. 8.是典型的脚注引用对应测试体系中的FootnoteTestolmocr/bench/tests.py。三、输出样本是如何生成的转换链路这份.md文件并非人工撰写而是由转换脚本对small_page_size.pdf执行 OCR 得到。核心入口是python -m olmocr.bench.convert olmocr_pipeline --dir ./olmOCR-bench/bench_data其中olmocr_pipeline是转换器注册表中的方法名olmocr/bench/convert.py 的available_methods字典将其映射到olmocr.bench.runners.run_olmocr_pipeline模块的run_olmocr_pipeline函数该表同时注册了gotocr、marker、mineru、chatgpt、gemini、mistral、docling、paddlepaddle、transformers等十余种外部 OCR 方案意味着同一批 PDF 可以一键对比任意多套系统。转换脚本的关键行为olmocr/bench/convert.py方法参数语法支持方法名[:keyvalue[:key2value2...]]的冒号分隔传参例如mineru:temperature2并可用namefolder_name指定输出子目录名parse_method_arg见 olmocr/bench/convert.py目录约定输出统一写入--dir指定的数据目录下、以方法名命名的子文件夹PDF 需位于该目录的pdfs/子目录中默认目录为sample_data--repeats控制每个页面重复转换的次数直接决定文件名中的repeat编号--force输出文件已存在时默认跳过避免重复计算--parallel并发任务数通过asyncio.Semaphore限流为 0 时串行执行--remove_text先将 PDF 逐页渲染成 PNG 再重新合成 PDF抹去数字文本层会禁用 olmOCR 的文档锚定机制--failfast任一分页转换抛异常立即终止并打印完整堆栈。官方流水线 runner 的底层实现run_olmocr_pipelineolmocr/bench/runners/run_olmocr_pipeline.py以异步方式调用 olmocr/pipeline.py 中的核心函数process_page。其工作方式值得注意通过vllm_server_ready(args)探测http://localhost:30044/v1上是否已有 vLLM 服务若没有则自动拉起一个模型默认allenai/olmOCR-2-7B-1025-FP8max_model_len16384gpu_memory_utilization0.8target_longest_image_dim1288见 olmocr/bench/runners/run_olmocr_pipeline.py服务常驻复用finally中刻意不做关闭便于批处理多页与多候选取page_result.response.natural_text作为输出若发生回退is_fallback或异常则返回None。值得注意的是 olmocr/bench/convert.py当 runner 返回None时转换脚本会写入空文件而非跳过——这样在评测阶段该文件会被识别为存在但内容为空从而被BaselineTest判为失败The text contains no alpha numeric characters而不是静默缺失导致评测中断。四、评测阶段样本如何被打分small_page_size_pg1_repeat1.md产出后由评测器消费python -m olmocr.bench.benchmark --dir ./olmOCR-bench/bench_dataolmocr/bench/benchmark.py 的输入约定为目录下存在pdfs/、若干*.jsonl测试定义以及至少一个非pdfs的候选子目录。流程如下递归扫描pdfs/下所有 PDF得到基准名列表加载每个 JSONL 中的测试用例load_tests见 olmocr/bench/tests.py用线程池并行解析并校验重复 ID对每个 PDF 自动补充一个BaselineTest见 olmocr/bench/benchmark.py校验每个 PDF 的每一页都存在对应候选输出文件否则该候选直接计 0 分并列出错误olmocr/bench/benchmark.py对每个测试按_pg{page}_匹配该页所有repeat文件逐一判定最终得分 各 JSONL 文件通过率的平均值并输出 95% 置信区间bootstrap默认 1000 次采样与两两置换检验结果。本样本可能触发的测试类型结合 olmocr/bench/tests.py 中定义的七种测试类型该样本对应的small_page_size.pdf可被验证的维度包括测试类型类判定逻辑本样本相关特征present/absentTextPresenceTestolmocr/bench/tests.py全文模糊匹配max_diffs推导相似度阈值支持first_n/last_n限定范围正文句子应出现页眉页脚页码应缺席orderTextOrderTestolmocr/bench/tests.py用fuzzysearch.find_near_matches找两个片段验证前者的start 后者的start例如 On light loams 应出现在 On peat soils 之前tableTableTestolmocr/bench/tests.py同时解析 Markdown 与 HTML 表格检查目标单元格及上下左右/表头关系成分表三行数据及top_heading关系baselineBaselineTestolmocr/bench/tests.py非空、无病态重复 n-gramRepeatDetector默认max_repeats30、不含 CJK/Emoji 等禁用字符输出质量底线footnoteFootnoteTestolmocr/bench/tests.py支持[^marker]、sup、Unicode 上标三种标记形态及前后文检查末尾脚注 Doncaster Report所有测试在运行前都会经过normalize_textolmocr/bench/tests.py的统一化处理剥离 Markdown 加粗/斜体符号、压缩空白、统一为 NFC 编码并将弯引号、破折号等花式字符映射为 ASCII 等价物—→-、“”→、‘’→。这正是本样本中**ANALYSIS.**的加粗标记不会干扰文本匹配的原因——评测不在乎 OCR 输出是否带 Markdown 装饰。五、样本背后的 small page size 专项测试small_page_size.pdf并非普通随机样本它在测试套件中承担着专项回归职责。在 tests/test_anchor.py 中testSmallPage专门将该 PDF 送入文档锚定anchor流程对第 1 页调用_pdf_report生成版面报告并调用get_anchor_text提取锚定文本。这说明小页面尺寸是 olmOCR 流水线已知的难点——页面的物理尺寸小于模型默认输入分辨率target_longest_image_dim1288时图像缩放与版面分析需要正确处理否则锚定文本会截断或失真。从样本输出的内容看olmOCR 对该页的处理是成功的保留了完整段落、生成了结构正确的表格且没有将页面信息如页码混入正文。这也与 olmOCR-Bench 的取样原则一致——基准要求文档应能适配标准 A4 纸张且人类可读olmocr/bench/README.md而小尺寸页面恰好是挑战模型鲁棒性的边界情况。六、从样本到基准完整的本地实操路径若要复现从 PDF 到得分的全流程可按以下步骤详细说明见 olmocr/bench/README.md# 1. 安装 bench 依赖与无头浏览器用于数学公式渲染测试 pip install -e .[bench] playwright install chromium # 2. 下载完整基准数据PDF 与 JSONL 标注到本地目录 huggingface-cli download --repo-type dataset --resume-download allenai/olmOCR-bench --local-dir ./olmOCR-bench # 3. 转换对每页 PDF 执行 OCR 并生成 *_pg*_repeat*.md python -m olmocr.bench.convert olmocr_pipeline --dir ./olmOCR-bench/bench_data # 4. 评测对候选输出逐条跑测试并输出汇总与置信区间 python -m olmocr.bench.benchmark --dir ./olmOCR-bench/bench_data评测器还提供若干实用参数--candidate只评测指定方法、--skip_baseline跳过底线检查、--sample N随机抽样 N 条测试、--test_report results.html生成 HTML 报告generate_html_report见 olmocr/bench/report.py、--output_failed failed.jsonl导出全候选均失败的用例便于复盘。此外若先使用python -m olmocr.pipeline在自己的工作区生成结果可借助 olmocr/bench/scripts/workspace_to_bench.py 将工作区 JSONL 结果转换为 bench 目录结构--repeat-index支持把多次工作区运行作为不同 repeat 参与平均。结语small_page_size_pg1_repeat1.md虽然只是评测数据目录中的一个文本文件但沿着它的文件名可以顺藤摸瓜还原出 olmOCR-Bench 的完整设计convert.py负责把 PDF 交给任意 OCR 方案并规范命名benchmark.py负责校验与聚合tests.py的七类测试负责把客观事实翻译成可判定的通过/失败而repeat机制与 bootstrap 置信区间保证了分数对随机性的稳健性。理解这层数据流无论是接入新 OCR 系统还是自建评测集都能直接复用这套已被验证的约定与工具链。【免费下载链接】olmocrToolkit for linearizing PDFs for LLM datasets/training项目地址: https://gitcode.com/GitHub_Trending/ol/olmocr创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考