Container Hierarchy Data Models #

Container Hierarchy Data Models are the deterministic representation of physical cargo stacks — the machine-readable tree that maps a vessel’s bays, tiers, and rows down to individual ISO 6346 boxes and their packages — so that terminal throughput, stowage optimization, and customs clearance run on structured data rather than guesswork. Within the Core Maritime Architecture & Taxonomy, this specification owns equipment relationships: it consumes normalized documents from the Bill of Lading Schema Mapping layer and stowage messages from Document Ingestion & EDI Parsing Workflows, then resolves them into low-latency, queryable hierarchies that terminal operating systems (TOS), stowage planners, and customs gateways can trust. For shipping operations, port authorities, and Python automation engineers, the hierarchy is the join key that keeps a physical container, its declared paperwork, and its live position from drifting apart.

The operational hierarchy follows a strict parent-child topology: Vessel → Bay → Tier → Row → Container → Package/Unit. Each node carries immutable operational metadata — ISO 6346 identifiers, gross and tare weights, IMDG hazard classes, reefer setpoints, and electronic seal states — and every downstream service reads the same tree so that a crane sequence, a customs hold, and a reefer alarm all reference one authoritative structure.

Ingestion Boundary & Protocol Handling #

Hierarchy data does not arrive as a tree. It arrives as flat stowage messages at the protocol edge: UN/EDIFACT BAPLIE (bay-plan/occupied-and-empty-locations) and COPRAR (container discharge/loading order) over AS2 or SFTP, ANSI X12 322 (terminal operations) over VANs, and increasingly carrier REST/JSON slot feeds over HTTPS. Each message describes container placements as independent LOC/EQD segment groups; the tree is something the ingestion layer reconstructs, it is never delivered pre-assembled.

A resilient ingestion boundary performs three jobs before any hierarchy logic runs:

1. Detect and unwrap the transport envelope. BAPLIE payloads open with a UNB interchange header and a UNH/BGM message header; X12 wraps in ISA/GS/ST. Strip the wrapper, normalize line endings, and honour the UNA service-string advice for delimiters — falling back to the UN/EDIFACT default set :+.? ' when UNA is absent rather than aborting the parse.
2. Apply codec fallbacks. Legacy translators emit mixed encodings; attempt utf-8, then iso-8859-1, then windows-1252, and record which succeeded so forensic replay is deterministic.
3. Emit a structured receipt before transformation. Log correlation_id, source_system, message_type (BAPLIE vs COPRAR vs X12 322), and a raw_payload_hash so the raw bytes are traceable end to end.

Positional parsing of BAPLIE stow-location codes is where most pipelines break. A location like LOC+147+0060802:139:5 encodes bay-row-tier in ISO 9711-1 stowage notation, not a delimiter-friendly path. The ingestion layer extracts bay, row, and tier as fixed-width sub-fields, then defers assembly into a path until every segment group in the message has been read — a container can reference a bay whose header segment appears later in the interchange. If the envelope is structurally corrupt (unbalanced segment counts in UNT, unrecoverable encoding), the whole payload routes to a dead-letter queue with a PARSE_FAILURE status; no partial tree is ever committed. The upstream normalization that feeds this boundary — segment extraction, positional indexing, memory-safe serialization — is detailed in Document Ingestion & EDI Parsing Workflows and its IFCSUM EDI Message Parsing reference.

Python Data Structure Mapping #

Once segments are parsed, the flat placements map to explicit, typed Python structures. Implicit dictionaries are unacceptable in production; the hierarchy is modeled with pydantic.BaseModel (or typing.TypedDict for hot loops) so that malformed nodes are rejected at construction rather than deep in a stowage calculation.

Two complementary storage patterns coexist, and mature pipelines write both:

1. Adjacency lists store each node as (child_id, parent_id). They are ideal for incremental BAPLIE deltas and real-time TOS syncs, where a single container moves without rewriting the whole tree.
2. Materialized paths serialize the full lineage as a VARCHAR string — for example VES01/BAY04/T03/R05/MSKU1234567 — enabling prefix queries (all boxes in bay 04) without recursive CTEs.

The Pydantic node enforces the schema contract at the ingestion boundary:

import structlog
from typing import Optional
from pydantic import BaseModel, Field

logger = structlog.get_logger()

class ContainerNode(BaseModel):
    node_id: str = Field(..., pattern=r"^[A-Z]{4}[0-9]{7}$")   # ISO 6346 owner+serial+check
    parent_id: Optional[str] = None
    # Stow position: 2-digit bay + 2-digit row (ISO 9711-1), e.g. "0401"
    bay_code: str = Field(..., pattern=r"^[0-9]{4}$")
    gross_weight_kg: float = Field(..., ge=0)
    tare_weight_kg: float = Field(..., ge=0)
    # IMDG division format: class[.division], e.g. "1.1", "2.3", "9"
    imdg_class: Optional[str] = Field(None, pattern=r"^[1-9](\.[0-9])?$")
    reefer_setpoint_c: Optional[float] = None
    materialized_path: str

Field coercion and unit normalization are non-negotiable. Weights arrive as MEA+WT+G+15000:KGM (kilograms) from one carrier and short tonnes from another; normalize everything to kilograms with an explicit unit tag and reject implausible values (a gross weight below tare is a unit-confusion signal, not a valid node). Stow positions arrive in several notations; canonicalize to the four-digit bay_code before building the path. UN/LOCODEs for the container’s next port are upper-cased and validated to five characters. This typed contract maps directly onto the database schema in Designing ISO container hierarchy trees in PostgreSQL, which enforces the same invariants as CHECK constraints and ltree path columns at the storage boundary.

Validation, Quarantine & Compliance Auditing #

Hierarchy integrity is enforced in three ascending tiers, and a node must clear each before it is committed. Collapsing them into a single check is the most common source of silent corruption.

1. Structural validation. Apply Pydantic strict mode at the boundary: every node has a well-formed ISO 6346 node_id, a four-digit bay_code, and a non-negative weight. Malformed identifiers are rejected immediately.
2. Semantic validation. Cross-reference against the physical and declared world. Verify the ISO 6346 modulo-11 check digit against the ISO 6346 freight container coding standard. Confirm every parent_id exists in the vessel’s approved stowage plan — a container assigned to a bay absent from the plan is an orphan and must not be committed. Check that cumulative tier weights respect deck-load limits and the vessel’s stability curve.
3. Regulatory validation. Enforce compliance invariants that carry legal weight. IMDG segregation rules must hold: incompatible hazard classes cannot share a bay or adjacent rows. Every export box must carry a SOLAS Verified Gross Mass (VGM); reefer units must match the terminal’s cold-stack power capacity. Seal numbers must reconcile against the Bill of Lading Schema Mapping record for the same container.

Routing on failure is deliberate. A node that fails structural validation is a permanent defect and goes to the dead-letter queue. A node that parses cleanly but fails a semantic or regulatory check — an unknown bay, a missing VGM, a check-digit that a legacy translator miscomputed on a physically real box — is a business exception and goes to a quarantine topic that operations staff can triage without halting throughput. Mixing the two floods engineers with recoverable errors and buries genuine corruption.

Every validation decision writes an immutable audit event carrying node_id, materialized_path, rule_applied, original_value, transformed_value, and compliance_status. This chain satisfies customs and port state control audit requirements and makes forensic reconstruction — why a container was held, moved, or reweighed — a query rather than an investigation.

import polars as pl
from typing import Tuple

def validate_stowage_integrity(
    df: pl.DataFrame, allowed_bays: list[str], max_deck_load_kg: float
) -> Tuple[pl.DataFrame, pl.DataFrame]:
    """Split nodes into committable and quarantined sets.

    Enforces parent existence (bay is in the approved stowage plan) and
    cumulative deck-load caps. Structural rejects are handled upstream by
    the Pydantic boundary; this gate is the semantic tier.
    """
    valid = df.filter(
        pl.col("parent_id").is_in(allowed_bays)
        & (pl.col("gross_weight_kg") <= max_deck_load_kg)
        & (pl.col("gross_weight_kg") > pl.col("tare_weight_kg"))
    )
    quarantined = df.filter(~pl.col("node_id").is_in(valid["node_id"]))
    logger.info(
        "stowage_validation",
        committed=valid.height,
        quarantined=quarantined.height,
    )
    return valid, quarantined

Downstream Integration #

A resolved hierarchy is not a destination; it is the join key the rest of the port depends on. Four consumers read it, and each imposes a distinct contract on the tree.

• Terminal operating system (TOS) and stowage planners consume subtree queries — every box in a bay, every reefer plug on a tier — to drive crane sequencing and yard allocation. Materialized-path prefix lookups keep these queries within TOS latency budgets even on 10,000-slot vessels.
• The Bill of Lading Schema Mapping layer attaches commercial context: a single B/L may reference many containers, each with distinct seals, cargo descriptions, and HS codes. The hierarchy is where consignee, seal, and DG data bind to a physical node, so a mismatch surfaces as a compliance hold rather than a silent overwrite.
• The Port Call Workflow Design state machine consumes hierarchy mutations as milestone triggers. When a node transitions to LOADED_ONBOARD or GATE_OUT, it must propagate as an idempotent event keyed on (node_id, milestone, event_hash) so an AS2 retransmission or API retry cannot double-count a move.
• Container Tracking & AIS Event Synchronization reconciles the declared tree against live reality: an AIS-derived vessel arrival inside a port geofence should confirm, not conflict with, the hierarchy’s ON_VESSEL node states, and a divergence flags an exception for reconciliation.

Because all four read the same structure, the hierarchy must publish mutations with explicit state flags and never expose a partially assembled tree. Consumers subscribe to committed nodes only; quarantined and dead-lettered payloads stay invisible downstream until they clear.

Fallback Chains & Uptime Guarantees #

Terminal planning cannot stall because one upstream feed times out, so the ingestion path degrades along an explicit chain rather than failing hard:

• Primary: REST/JSON slot ingestion with synchronous validation.
• Fallback 1: BAPLIE/COPRAR file drop over SFTP, processed as an asynchronous batch, preserving message ordering and deduplication.
• Fallback 2: A dead-letter queue with exponential-backoff retry (base_delay * 2^n + jitter, capped at three attempts) for transient network or schema-version faults.
• Fallback 3: A manual reconciliation dashboard for customs holds or missing-seal exceptions that no automated path can resolve.

Circuit breakers isolate each downstream dependency independently. Five consecutive 5xx or timeout responses from the customs gateway open that breaker and fall back to cached registry lookups, without touching the unrelated stowage-planner path. The distinction between the dead-letter queue and the quarantine topic is the load-bearing decision here: unrecoverable corruption goes to the DLQ for engineering, recoverable business exceptions go to quarantine for ops, and the two never share a queue.

import structlog
from tenacity import retry, stop_after_attempt, wait_exponential

logger = structlog.get_logger("maritime.hierarchy")

@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=2, min=4, max=30))
def push_to_dlq(payload: dict, error_code: str) -> None:
    """Route an unrecoverable node payload to the dead-letter queue.

    Retries only guard transient publish failures against the broker;
    the payload itself is already known to be a permanent defect.
    """
    logger.warning(
        "dead_letter_queued",
        node_id=payload.get("node_id"),
        error=error_code,
        schema_version="2.4.1",
    )
    # Production: publish to Kafka/SQS with a retention policy and alert hook.

Step-by-step Implementation Guide #

Build the hierarchy resolver in five steps, each runnable in isolation.

1. Parse stow-location segments into flat placements. Read every LOC/EQD group from the BAPLIE interchange, extract bay, row, and tier as fixed-width sub-fields per ISO 9711-1, and emit one placement record per container before any tree assembly.
2. Coerce placements into typed nodes. Instantiate a ContainerNode per placement so structural validation (ISO 6346 shape, four-digit bay, non-negative weights) runs at construction; collect construction failures for the dead-letter queue.
3. Assemble the tree and materialize paths. Resolve each node’s parent_id against the vessel stowage plan, then compute the materialized_path from root to leaf so prefix queries work without recursion.
4. Run the semantic and regulatory gates. Apply validate_stowage_integrity for parent existence and deck-load caps, then layer IMDG segregation, VGM presence, and seal reconciliation, routing failures to the quarantine topic with an explicit reason code.
5. Commit and publish mutations. Persist committed nodes to the ltree-backed store, emit an immutable audit event per node, and publish idempotent milestone events for downstream consumers.

import structlog
from typing import Iterable

logger = structlog.get_logger("maritime.hierarchy")

def resolve_hierarchy(
    placements: Iterable[dict], allowed_bays: list[str], max_deck_load_kg: float
) -> list[ContainerNode]:
    nodes: list[ContainerNode] = []
    for raw in placements:
        try:
            nodes.append(ContainerNode(**raw))          # steps 1-2: typed structural gate
        except Exception as exc:
            push_to_dlq(raw, error_code="STRUCTURAL_REJECT")
            logger.warning("node_rejected", node_id=raw.get("node_id"), error=str(exc))
    # step 3-4: semantic/regulatory validation happens on the assembled frame
    logger.info("hierarchy_resolved", node_count=len(nodes))
    return nodes

Troubleshooting Common Failures #

Frequently Asked Questions #

Related #

• Bill of Lading Schema Mapping — deterministic translation of B/L payloads that bind commercial context to hierarchy nodes
• Designing ISO container hierarchy trees in PostgreSQL — the ltree schema and index strategy behind this model
• Port Call Workflow Design — the idempotent state machine that consumes hierarchy milestone events
• Container Tracking & AIS Event Synchronization — reconciles the declared tree against live vessel positions
• Document Ingestion & EDI Parsing Workflows — the upstream domain that normalizes BAPLIE and COPRAR into placements

Up: Core Maritime Architecture & Taxonomy — the parent framework this model belongs to.