docparser
Convert research literature into a clean, reproducible Markdown + JSON corpus, with optional vision-language captioning of every embedded figure.
docparser converts research literature — .docx, .xlsx, .pdf, .html,
.pptx, .epub, .txt, .md, .csv — into a clean, reproducible Markdown +
JSON corpus, with optional vision-language captioning of every embedded
figure via OpenRouter, OpenAI, Gemini, a local server, or a fully-local model.
┌────────────────┐
data/raw/*.docx │ docparser │ data/parsed/<slug>/
data/raw/*.xlsx │ - parse_docx │ document.md
data/raw/*.pdf ─────► │ - parse_xlsx │ ────► document.json
data/raw/*.html │ - parse_pdf │
data/raw/*.pptx │ - parse_html │ data/assets/<slug>/
data/raw/*.epub │ - parse_pptx │ img-*.png
data/raw/*.txt|md|csv │ - parse_epub │
│ - VLM caption │
└────────────────┘The package is published on PyPI as
rc-docparser, but the Python
import name is docparser — i.e. pip install rc-docparser then
import docparser.
Why it exists
Building a clean dataset out of mixed research documents is tedious and
error-prone: every format has its own quirks, figures lose their context, and
re-runs aren't reproducible. docparser turns a messy data/raw/ folder into a
predictable Markdown + JSON corpus you can feed into RAG, fine-tuning, or
analysis pipelines.
- One API, nine formats. Office docs, PDFs, web pages, slides, e-books, and
plain text/CSV behind a single
parse_path()/run_all()call. - Markdown and JSON. Human-readable Markdown plus a typed block schema so downstream layers can rely on structure.
- Figures with context. Embedded images are extracted with stable names and their surrounding caption + context, ready for captioning.
- Optional VLM captioning. Send each figure to a vision-language model and get back a strict JSON caption/description — cached by image hash so re-runs are free.
- Lean core, opt-in extras. The base install stays small and MIT; heavier features (PDF backends, OCR, tables, local VLM) are separate install extras.
- Library or CLI. Use it programmatically or drive it from the
docparsercommand.
At a glance
from docparser import WorkspaceLayout, run_all
layout = WorkspaceLayout.under("./project") # data/raw, data/parsed, data/assets, .cache
layout.ensure()
run_all(layout, use_vlm=False) # parse everything in data/rawdocparser is beta (0.2.x) and published on PyPI as rc-docparser. It
requires Python 3.10+. See the roadmap for what's
next.
Where to next
- New here? Start with Installation and the Quickstart.
- Driving it from a terminal? See the CLI.
- Curious what each format yields? Read Supported formats and Output & schema.
- Captioning figures? Read Image captioning (VLM) and Configuration.