vlm-run/vlmrun-hub

Production-ready Pydantic schemas for Visual ETL.

A typed schema catalog that turns images, videos, and documents into validated JSON. Pin a version, compose them, run them with any VLM. Batteries included.

Star on GitHubBrowse Catalog

Skip weeks of schema design.

Every schema is Pydantic-native, validated against real industry use cases, and model-agnostic. Use it with Orion, GPT, Claude, or Gemini without rewriting prompts.

Pydantic v2 native

Type-safe nested models, rich validators, and full mypy/pyright support. Compose schemas the way you compose Python.

Batteries included

Invoices, resumes, ID cards, medical records, receipts, W-2s, and more. Thirty-plus production schemas, ready to import.

Validation at the boundary

Catch bad extractions before they hit your pipeline. Pydantic validators clean, coerce, and reject at the model edge.

Model-agnostic

Works with OpenAI structured outputs, Claude, Gemini, and open-source VLMs via JSON mode. One schema, every backend.

Composable by design

Reuse fragments across schemas. Lift a Address or LineItem block into a larger document without rewriting it.

Open catalog

Browse the public YAML catalog of schemas. Open a PR with yours; we review and merge weekly.

Schema catalog

Thirty schemas, one import away.

Pre-built Pydantic schemas across documents, identity, healthcare, finance, retail, and media. Pin a version and ship.

document.invoice

Customer details, line items, and totals.

finance · work

IMGDOC
document.bank-statement

Account, balances, and transaction history.

finance · banking

IMGDOC
document.resume

Contact, education, work history, and skills.

hr · work

IMGDOC
document.us-passport

Passport number, name, and biometric fields.

identity · travel

IMGDOC
document.business-card

Name, role, company, and contact details.

work

IMGDOC
document.request-for-proposal

Scope, vendor terms, and submission details.

work

IMGDOC
document.india.pan-card

PAN number, name, and DOB fields.

identity

IMGDOC
healthcare.hipaa-release

Patient, recipients, and authorized release scope.

healthcare · hipaa

IMGDOC
accounting.form-w2

Employee, employer, wages, and tax withholding.

finance · hr

IMGDOC
finance.balance-sheet

Assets, liabilities, equity, and totals.

finance

IMGDOC
retail.product-catalog

SKU, attributes, and merchandising fields.

retail

IMG
media.tv-news

Headlines, anchors, and on-screen text.

media · news

IMGVID
media.nfl-game-state

Down, distance, score, and play context.

media · sports

IMGVID
logistics.bill-of-lading

Shipper, consignee, items, and carrier.

logistics

IMGDOC
social.twitter-card

Author, body, engagement, and media.

social · media

IMG
document.invoice

Customer details, line items, and totals.

finance · work

IMGDOC
document.bank-statement

Account, balances, and transaction history.

finance · banking

IMGDOC
document.resume

Contact, education, work history, and skills.

hr · work

IMGDOC
document.us-passport

Passport number, name, and biometric fields.

identity · travel

IMGDOC
document.business-card

Name, role, company, and contact details.

work

IMGDOC
document.request-for-proposal

Scope, vendor terms, and submission details.

work

IMGDOC
document.india.pan-card

PAN number, name, and DOB fields.

identity

IMGDOC
healthcare.hipaa-release

Patient, recipients, and authorized release scope.

healthcare · hipaa

IMGDOC
accounting.form-w2

Employee, employer, wages, and tax withholding.

finance · hr

IMGDOC
finance.balance-sheet

Assets, liabilities, equity, and totals.

finance

IMGDOC
retail.product-catalog

SKU, attributes, and merchandising fields.

retail

IMG
media.tv-news

Headlines, anchors, and on-screen text.

media · news

IMGVID
media.nfl-game-state

Down, distance, score, and play context.

media · sports

IMGVID
logistics.bill-of-lading

Shipper, consignee, items, and carrier.

logistics

IMGDOC
social.twitter-card

Author, body, engagement, and media.

social · media

IMG
document.receipt

Merchant, items, and totals from any receipt.

finance · work

IMGDOC
document.bank-check

Routing, account, payee, and amount.

finance · banking

IMGDOC
document.us-drivers-license

Name, address, DOB, and license details.

identity

IMGDOC
document.utility-bill

Account, billing period, charges, and dues.

energy

IMGDOC
document.insurance-claim

Claimant, policy, incident, and amounts.

insurance

IMGDOC
document.india.aadhaar-card

Aadhaar number, name, address, and DOB.

identity

IMGDOC
healthcare.medical-insurance-card

Member, coverage, group, and provider info.

healthcare · insurance

IMGDOC
healthcare.pathology-report

Specimen, diagnosis, and pathologist findings.

healthcare · medical

IMGDOC
accounting.form-payslip

Earnings, deductions, and YTD totals.

finance · hr

IMGDOC
retail.ecommerce-product-caption

Visual description, attributes, and delivery info.

retail · ecommerce

IMG
food.nutrition-facts-label

Serving size, calories, macros, and ingredients.

retail · food

IMGDOC
media.nba-game-state

Score, quarter, possession, and player stats.

media · sports

IMGVID
aerospace.remote-sensing

Land features, infrastructure, and visibility.

aerospace

IMGVID
real-estate.lease-agreement

Parties, term, rent, and lease clauses.

real-estate

IMGDOC
document.receipt

Merchant, items, and totals from any receipt.

finance · work

IMGDOC
document.bank-check

Routing, account, payee, and amount.

finance · banking

IMGDOC
document.us-drivers-license

Name, address, DOB, and license details.

identity

IMGDOC
document.utility-bill

Account, billing period, charges, and dues.

energy

IMGDOC
document.insurance-claim

Claimant, policy, incident, and amounts.

insurance

IMGDOC
document.india.aadhaar-card

Aadhaar number, name, address, and DOB.

identity

IMGDOC
healthcare.medical-insurance-card

Member, coverage, group, and provider info.

healthcare · insurance

IMGDOC
healthcare.pathology-report

Specimen, diagnosis, and pathologist findings.

healthcare · medical

IMGDOC
accounting.form-payslip

Earnings, deductions, and YTD totals.

finance · hr

IMGDOC
retail.ecommerce-product-caption

Visual description, attributes, and delivery info.

retail · ecommerce

IMG
food.nutrition-facts-label

Serving size, calories, macros, and ingredients.

retail · food

IMGDOC
media.nba-game-state

Score, quarter, possession, and player stats.

media · sports

IMGVID
aerospace.remote-sensing

Land features, infrastructure, and visibility.

aerospace

IMGVID
real-estate.lease-agreement

Parties, term, rent, and lease clauses.

real-estate

IMGDOC
123456789101112131415161718192021
from openai import OpenAI
from vlmrun.hub.schemas.document import Invoice

client = OpenAI(
    base_url="https://api.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

result = client.beta.chat.completions.parse(
    model="vlmrun-orion-1",
    messages=[
        {"role": "user", "content": [
            {"type": "text", "text": "Extract structured data."},
            {"type": "image_url", "image_url": {"url": "https://..."}},
        ]},
    ],
    response_format=Invoice,
)

invoice: Invoice = result.choices[0].message.parsed
print(invoice.total_amount, invoice.vendor_name)

Quick Start

Structured outputs in three lines.

Import a schema, call any VLM, get validated structured data. No prompts to engineer, no regex, no parsing.

  • 30+ pre-built schemas across industries and use cases
  • Drops into OpenAI-compatible clients unchanged
  • Battle-tested schemas in production at scale
  • MIT-licensed: fork, extend, and contribute
View on GitHubTry on the Platform

Bring your own schema, and build with any VLM.

Star on GitHubBook a Demo