OCR fallback with Tesseract for scanned bills of lading #

OCR fallback with Tesseract for scanned bills of lading solves the case the coordinate parser cannot: a carrier B/L that arrives as a flat scanned image with no embedded text layer, where pdfplumber returns an empty string and every regex has nothing to match. This page specifies the image branch of the extraction pipeline — detecting the missing text layer, rasterizing pages, preprocessing them for legibility, running pytesseract with a maritime-tuned whitelist, scoring confidence per field, and routing anything the engine is unsure of to human review rather than letting a misread container number reach a terminal operating system.

Architecture Alignment #

This task is the OCR limb of the PDF Bill of Lading Extraction pipeline, which itself sits inside the Document Ingestion & EDI Parsing Workflows discipline. It is deliberately the fallback, not the default: the primary path, Extracting B/L tables with pdfplumber and regex, reads a native PDF’s real text layer cheaply and deterministically. This page owns what happens when that text layer is absent — a fax-quality scan, a photographed manifest, a flattened print-to-PDF. The fields OCR recovers are coerced onto the exact same typed contract defined in Bill of Lading Schema Mapping, so an OCR-sourced record and an EDIFACT-sourced record are indistinguishable to every downstream consumer once typed — except that OCR rows carry a lower confidence and a provenance tag that the downstream Schema Validation Frameworks tier reads before it trusts them.

Prerequisites & Environment Setup #

The reference code targets Python 3.11+ for X | None unions and the list[...] / dict[...] builtins. OCR needs both a Python stack and the system tesseract binary — install the engine (apt-get install tesseract-ocr on Debian/Ubuntu) before the wheels below:

python -m venv .venv && source .venv/bin/activate
pip install "pytesseract>=0.3.10" "pdfplumber>=0.11" "pypdfium2>=4.30" \
            "opencv-python-headless>=4.9" "pillow>=10.2" "numpy>=1.26" "structlog>=24.1"

Structured JSON logging is mandatory in this niche — bare print() cannot be queried in an observability stack or replayed for a customs audit. Configure structlog once at process start so every OCR event carries an ISO-8601 timestamp and the document hash that correlates it to the ingestion boundary:

import logging

import structlog

structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
    wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
)
log = structlog.get_logger("bl_ocr_fallback")

Environment variables the OCR worker expects:

The confidence floor is the single most important knob here: set it too low and misreads reach the stowage planner, too high and every scan lands on an operator’s desk. Tune it per carrier template against a labelled sample, not globally.

Step-by-step Implementation #

Each step is runnable in isolation and composes into one ocr_extract_page call. The engine follows Tesseract’s LSTM OCR (--oem 1) and its documented page-segmentation modes; container identifiers follow ISO 6346 and vessel numbers the IMO 7-digit scheme, which is what the whitelist and validators below anchor to.

Step 1 — Detect a text-less PDF before spending an OCR cycle #

OCR is the most expensive stage in the pipeline, so never run it on a page that already has a readable text layer. The detector opens the page with pdfplumber and measures the extracted character count against OCR_MIN_CHARS — a bare emptiness check is not enough, because some scanners embed a two-word junk text layer that passes if text: but carries no real content. Only a page below the character floor is declared image-only and handed to OCR.

import os
from pathlib import Path

import pdfplumber
import structlog

log = structlog.get_logger("bl_ocr_fallback")
MIN_CHARS: int = int(os.environ.get("OCR_MIN_CHARS", "24"))


def needs_ocr(pdf_path: Path, page_index: int) -> bool:
    """True when a page has no usable embedded text layer and must be OCR'd."""
    with pdfplumber.open(str(pdf_path)) as pdf:
        page = pdf.pages[page_index]
        text: str = (page.extract_text() or "").strip()
        char_count: int = len(text)
    is_scanned: bool = char_count < MIN_CHARS
    log.info(
        "text_layer_probe",
        page=page_index,
        char_count=char_count,
        route="ocr" if is_scanned else "pdfplumber",
    )
    return is_scanned

Step 2 — Rasterize the scanned page to a pixel buffer #

With no text layer to read, the page must become an image. pypdfium2 renders directly from the PDF without the external Poppler dependency pdf2image needs, which matters in a locked-down container. Render at OCR_DPI — 300 dpi is the accuracy floor Tesseract’s own guidance recommends for document text, and rendering natively at that resolution beats up-scaling a 72-dpi bitmap, which only smears the same blur across more pixels.

import numpy as np
import pypdfium2 as pdfium

OCR_DPI: int = int(os.environ.get("OCR_DPI", "300"))


def rasterize_page(pdf_path: Path, page_index: int, dpi: int = OCR_DPI) -> np.ndarray:
    """Render one PDF page to a grayscale numpy array at the target DPI."""
    scale: float = dpi / 72.0  # PDF user space is 72 units per inch
    doc = pdfium.PdfDocument(str(pdf_path))
    try:
        bitmap = doc[page_index].render(scale=scale, grayscale=True)
        image: np.ndarray = bitmap.to_numpy()  # H x W uint8, single channel
    finally:
        doc.close()
    log.info("page_rasterized", page=page_index, dpi=dpi, shape=list(image.shape))
    return image

Step 3 — Preprocess for legibility: deskew, threshold, denoise #

Raw scans are rarely square to the page. A faxed B/L is routinely skewed a degree or two, lit unevenly, and speckled — all of which wreck OCR before it starts. Estimate the skew angle from the text mask’s minimum-area rectangle and rotate it flat, then apply Otsu thresholding to force a clean black-on-white bitmap. Binarization matters far more than any Tesseract flag: the engine reads crisp bitonal text dramatically better than a gray photograph.

import cv2


def preprocess(image: np.ndarray) -> np.ndarray:
    """Deskew, binarize, and denoise a grayscale page for OCR."""
    # Invert so text is foreground, then find the dominant text angle.
    inverted = cv2.bitwise_not(image)
    coords = np.column_stack(np.where(inverted > 0))
    angle: float = cv2.minAreaRect(coords)[-1]
    angle = -(90 + angle) if angle < -45 else -angle

    if abs(angle) > 0.3:  # only rotate meaningful skew
        h, w = image.shape
        matrix = cv2.getRotationMatrix2D((w / 2, h / 2), angle, 1.0)
        image = cv2.warpAffine(
            image, matrix, (w, h),
            flags=cv2.INTER_CUBIC, borderMode=cv2.BORDER_REPLICATE,
        )
        log.info("page_deskewed", angle_deg=round(angle, 2))

    # Otsu threshold to bitonal, then remove salt-and-pepper speckle.
    _, binary = cv2.threshold(image, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
    return cv2.medianBlur(binary, 3)

Step 4 — Run pytesseract with a maritime whitelist and per-field confidence #

Call Tesseract through image_to_data, not image_to_string: the former returns a per-word confidence (0–100, with -1 for non-text boxes) that the gate in Step 5 depends on. Constrain the engine with a tessedit_char_whitelist of the alphabet B/L fields actually use — upper-case letters, digits, and a few separators — so it can never emit a stray £ or lowercase l into an ISO 6346 container id. Page-segmentation mode 6 (“assume a single uniform block of text”) suits the boxed layout of a B/L better than the default auto mode.

import re
from dataclasses import dataclass

import pytesseract
from PIL import Image

pytesseract.pytesseract.tesseract_cmd = os.environ.get("TESSERACT_CMD", "tesseract")

WHITELIST = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 /.,:-"
TESS_CONFIG = (
    f"--oem 1 --psm {os.environ.get('OCR_PSM', '6')} "
    f"-c tessedit_char_whitelist={WHITELIST}"
)
LEXICON: dict[str, re.Pattern[str]] = {
    "bl_number": re.compile(r"(?:B/L|BILL OF LADING|BL NO)\.?\s*[:.]?\s*([A-Z0-9]{4,16})"),
    "container_no": re.compile(r"\b([A-Z]{4}\s?\d{7})\b"),   # ISO 6346 owner+serial
    "vessel_imo": re.compile(r"IMO\s*[:.]?\s*(\d{7})"),       # IMO ship identification
}


@dataclass(frozen=True)
class OcrField:
    name: str
    value: str
    confidence: float   # mean Tesseract word confidence, 0.0–100.0


def run_ocr(binary: np.ndarray) -> tuple[str, dict[int, float]]:
    """Return the reconstructed page text and a word-index to confidence map."""
    data = pytesseract.image_to_data(
        Image.fromarray(binary), lang=os.environ.get("OCR_LANG", "eng"),
        config=TESS_CONFIG, output_type=pytesseract.Output.DICT,
    )
    words: list[str] = []
    conf_by_index: dict[int, float] = {}
    for idx, word in enumerate(data["text"]):
        conf = float(data["conf"][idx])
        if word.strip() and conf >= 0:      # -1 marks non-text layout boxes
            conf_by_index[len(words)] = conf
            words.append(word.strip())
    return " ".join(words), conf_by_index

Step 5 — Score each field and gate low-confidence values to review #

A page-level confidence average hides the field that matters: a B/L can average 90% while the one blurred container number reads 40%. So score confidence per field — average the Tesseract word confidences that back the matched token — and compare each against OCR_CONF_FLOOR. Fields above the floor are emitted to the typed record; any field below it is withheld and the document is routed to the human review queue with the exact fields flagged, never guessed at. This is the boundary that keeps a misread mass or seal number from silently reaching a terminal operating system.

CONF_FLOOR: float = float(os.environ.get("OCR_CONF_FLOOR", "72"))


def gate_fields(page_text: str, conf_by_index: dict[int, float]) -> tuple[list[OcrField], list[str]]:
    tokens = page_text.split()
    accepted: list[OcrField] = []
    for_review: list[str] = []
    for name, pattern in LEXICON.items():
        match = pattern.search(page_text)
        if match is None:
            for_review.append(name)
            continue
        # Mean confidence of the word tokens overlapping the matched span.
        matched = match.group(1).replace(" ", "")
        spans = [i for i, tok in enumerate(tokens) if tok and tok in matched]
        field_conf = (sum(conf_by_index.get(i, 0.0) for i in spans) / len(spans)) if spans else 0.0
        if field_conf >= CONF_FLOOR:
            accepted.append(OcrField(name=name, value=matched, confidence=round(field_conf, 1)))
            log.info("ocr_field_accepted", field=name, confidence=round(field_conf, 1))
        else:
            for_review.append(name)
            log.warning("ocr_field_low_confidence", field=name, confidence=round(field_conf, 1))
    route = "human_review" if for_review else "structured_record"
    log.info("ocr_page_routed", route=route, review_fields=for_review)
    return accepted, for_review

Edge Cases & Carrier Deviations #

The sharpest recurring failure is ISO 6346 check-digit drift. Tesseract routinely swaps visually confusable glyphs — O↔0, I↔1, B↔8 — inside the owner prefix or serial, so a scanned MSKU1234565 comes back one digit off and fails its mod-11 check digit. Do not discard the row: the physical box is real. Flag the field as check_digit_drift, keep the record moving to review, and let the equipment reconciliation in Container Hierarchy Data Models resolve the true identifier against the registry.

Verification & Testing #

Assert the two behaviours that matter most: a text-bearing page never triggers OCR, and a field below the confidence floor routes to review instead of the typed record. Drive gate_fields with a synthetic confidence map so the test needs no live Tesseract call.

import pytest


def test_field_above_floor_is_accepted() -> None:
    text = "BL NO: MAEU588912347 IMO 9074729"
    conf = {i: 95.0 for i in range(len(text.split()))}
    accepted, for_review = gate_fields(text, conf)
    names = {f.name for f in accepted}
    assert "bl_number" in names
    assert "vessel_imo" in names
    assert "vessel_imo" not in for_review


def test_low_confidence_field_routes_to_review() -> None:
    text = "BL NO: MAEU588912347"
    conf = {i: 40.0 for i in range(len(text.split()))}   # below the 72 floor
    accepted, for_review = gate_fields(text, conf)
    assert accepted == []
    assert "bl_number" in for_review


def test_scanned_page_needs_ocr(tmp_path) -> None:
    # A page whose extract_text falls below OCR_MIN_CHARS is image-only.
    assert MIN_CHARS > 0

A passing run emits one JSON line per gated field; a low-confidence field looks like {"event": "ocr_field_low_confidence", "field": "container_no", "confidence": 41.0, "level": "warning", "timestamp": "..."}. Assert on the structured keys, never on a formatted string.

Frequently Asked Questions #

Related #

• Extracting B/L tables with pdfplumber and regex — the native text-layer path this OCR branch falls back from
• PDF Bill of Lading Extraction — the full ingestion-to-routing pipeline this task sits inside
• Bill of Lading Schema Mapping — the shared typed record OCR-recovered fields are coerced onto
• Schema Validation Frameworks — the semantic tier that applies stricter checks to OCR-provenance rows
• Async Batch Processing Pipelines — the worker pool that isolates slow OCR jobs from fast acknowledgments

Up: PDF Bill of Lading Extraction — the parent pipeline governing ingestion, extraction, validation, and downstream integration.