SC ReconciliationReconciliation Automation
  1. Home
  2. Ingestion
  3. Schema Validation Using Pydantic
  4. Coercing and Validating Decimal Quantities with Pydantic

Coercing and Validating Decimal Quantities with Pydantic

↑ Part of Schema Validation Using Pydantic.

A quantity or unit price that enters your model as a Python float carries binary rounding error the instant it is parsed, long before any business rule runs against it. Decimal("1234.50") != Decimal(str(1234.50 * 3)) is not a hypothetical — it is what happens every time a receiving quantity is multiplied by a unit cost using float arithmetic and then compared against an ERP total computed in fixed-point. This page addresses one precise boundary decision: how do you force every financial quantity and price field through Pydantic v2 as Decimal, reject float input outright instead of silently coercing it, normalize the comma-formatted and over-precise strings real feeds send, and route anything that fails to a quarantine record with a reason code your downstream systems can act on? It is the numeric-precision companion to the contract design in the parent Schema Validation Using Pydantic reference and to the field-level coercion patterns in Validating Supplier Data Payloads with Pydantic Models.

Operational Trigger Signals

Add an explicit Decimal-only coercion layer for quantity and price fields — instead of relying on Pydantic’s default lax numeric coercion — when your reconciliation telemetry shows any of these measurable conditions:

  1. Phantom quantity or price variances appear without a source discrepancy. A quantity_received of 150.1 and an ERP total of 150.10000000000002 fail an equality check even though no physical unit is in dispute — the variance exists only in the arithmetic, not the shipment.
  2. Rounding drift accumulates across batch aggregation. Summing thousands of float line totals produces a batch total that diverges from the sum of the same values computed in fixed-point by a cent or more, and the drift grows with batch size rather than staying constant.
  3. Fields silently accept more precision than the ledger can post. A unit_price of 14.123456789 passes validation as a bare float or unconstrained Decimal, then fails at ERP import because the target column is NUMERIC(14,6) and truncates or rejects the row.
  4. Thousands-separated or locale-formatted strings raise a parse error instead of being normalized. Vendor exports send "1,234.50" or "1 234,50"; a naive Decimal(value) call raises InvalidOperation on the comma, and the record is rejected for a formatting difference a human would resolve in one glance.
  5. Scientific notation or negative quantities reach fields that must never carry them. A malformed export emits "1.5E+3" for a quantity field, and without an explicit precision and sign constraint it coerces successfully into a value three orders of magnitude off.

When any of these signals persist across consecutive ingestion runs, the fix is not a looser equality check downstream — matching and reconciliation already assume decimal-safe inputs, which is why Setting Quantity and Price Tolerance Windows defines tolerance in fixed-point terms. The fix belongs at the schema boundary, before a float-tainted value can enter any downstream calculation.

Step-by-Step Implementation

Build the coercion as a field_validator with mode="before" so it runs ahead of Pydantic’s own type coercion, combined with Field constraints that enforce precision after the value is typed. The procedure:

  1. Reject float input immediately — do not attempt to recover it via Decimal(str(value)). A float has already lost precision by the time your validator sees it; only str, int, and Decimal inputs are trustworthy.
  2. Normalize string input — strip whitespace and thousands separators, and reject anything that still fails Decimal() parsing with a structured error rather than an unhandled exception.
  3. Quantize to the field’s declared precision using an explicit rounding mode, never the ambient default.
  4. Enforce max_digits and decimal_places as Field constraints so a value that is technically a valid Decimal but exceeds the ledger’s column precision fails validation instead of silently truncating on write.
  5. Catch ValidationError, extract per-field reason codes, and route the record to quarantine rather than raising past the ingestion boundary.
PYTHON
import logging
from decimal import Decimal, InvalidOperation, ROUND_HALF_UP
from typing import Annotated, Optional

from pydantic import BaseModel, ConfigDict, Field, ValidationError, field_validator

logger = logging.getLogger("ingestion.decimal_quantities")

# Precision matches the ERP's NUMERIC(12,4) receiving column.
QuantityDecimal = Annotated[
    Decimal,
    Field(ge=0, max_digits=12, decimal_places=4, description="Receiving quantity, 4dp"),
]
# Precision matches NUMERIC(14,6) to absorb sub-cent FX-converted unit costs.
UnitPriceDecimal = Annotated[
    Decimal,
    Field(gt=0, max_digits=14, decimal_places=6, description="Unit price, base currency, 6dp"),
]

_QUANTUM_BY_FIELD = {"quantity_received": Decimal("1.0000"), "unit_price": Decimal("1.000000")}


class ReceivingLine(BaseModel):
    model_config = ConfigDict(strict=True, extra="forbid")

    line_number: int = Field(ge=1)
    sku: str = Field(pattern=r"^[A-Z0-9\-]{4,20}$")
    quantity_received: QuantityDecimal
    unit_price: UnitPriceDecimal
    supplier_currency: Optional[str] = Field(default=None, pattern=r"^[A-Z]{3}$")

    @field_validator("quantity_received", "unit_price", mode="before")
    @classmethod
    def coerce_decimal(cls, value: object, info) -> Decimal:
        """Force Decimal at the boundary: reject float, normalize strings, quantize."""
        field = info.field_name
        if isinstance(value, float):
            # A float has already lost precision before this validator runs;
            # str(value) would launder the error instead of surfacing it.
            logger.error("float_rejected field=%s raw=%r", field, value)
            raise ValueError(f"{field} must be str/int/Decimal, not float (got {value!r})")

        if isinstance(value, str):
            cleaned = value.strip().replace(",", "").replace(" ", "")
            try:
                value = Decimal(cleaned)
            except InvalidOperation as exc:
                logger.error("decimal_parse_failed field=%s raw=%r", field, value)
                raise ValueError(f"{field} is not a parseable decimal string: {value!r}") from exc
        elif isinstance(value, int):
            value = Decimal(value)

        if not isinstance(value, Decimal):
            logger.error("decimal_coercion_failed field=%s type=%s", field, type(value).__name__)
            raise ValueError(f"{field} must coerce to Decimal, got {type(value).__name__}")

        if value.is_nan() or value.is_infinite():
            raise ValueError(f"{field} must be a finite decimal value")

        quantum = _QUANTUM_BY_FIELD.get(field, Decimal("1.0000"))
        return value.quantize(quantum, rounding=ROUND_HALF_UP)


def validate_receiving_line(raw: dict) -> tuple[Optional[ReceivingLine], Optional[dict]]:
    """Validate one receiving line; return (model, None) on success or (None, quarantine_record)."""
    try:
        return ReceivingLine.model_validate(raw), None
    except ValidationError as exc:
        errors = exc.errors()
        logger.warning(
            "quarantine line=%s error_count=%d fields=%s",
            raw.get("line_number"), len(errors), [e["loc"] for e in errors],
        )
        quarantine = {
            "raw_payload": raw,
            "failure_reasons": [
                {"field": ".".join(str(p) for p in e["loc"]), "type": e["type"], "msg": e["msg"]}
                for e in errors
            ],
        }
        return None, quarantine

The order of operations inside coerce_decimal is what makes the field trustworthy: the float check runs before any string handling can accidentally re-admit a float that was passed as f"{value}" upstream, and quantization always runs last so every value that reaches the Field constraint check has already been rounded to the field’s declared precision — a value can fail decimal_places only because it genuinely exceeds the ledger’s column width, not because of an unrounded remainder.

Raw value to typed Decimal coercion and quarantine flow A raw field value enters a float type gate; any float is rejected immediately and routed to a quarantine record tagged FLOAT_REJECTED. Values that pass are normalized (stripped separators) and parsed into Decimal, with parse failures routed to quarantine as DECIMAL_PARSE_FAILED. The parsed value is quantized to the field's decimal_places using ROUND_HALF_UP, then checked against max_digits and decimal_places constraints; a constraint failure routes to quarantine as PRECISION_EXCEEDED. A value that clears every stage commits as a typed Decimal field on the Pydantic model. Raw field value str · float · int · Decimal Float type gate isinstance(value, float)? Normalize + parse strip commas · Decimal(cleaned) Quantize to decimal_places ROUND_HALF_UP Constraints satisfied? max_digits · decimal_places · ge Typed Decimal field commit to model Quarantine record payload · field · reason code FLOAT_REJECTED DECIMAL_PARSE_ FAILED PRECISION_ EXCEEDED not float pass pass float invalid fail

Quantization is the step that most implementations get subtly wrong by relying on the Decimal context’s default rounding rather than declaring one explicitly. Given a target precision of dd decimal places, the quantize step computes:

q(x,d)=round⁡(x×10d)10dq(x, d) = \frac{\operatorname{round}(x \times 10^{d})}{10^{d}}

with the rounding rule pinned to ROUND_HALF_UP (or ROUND_HALF_EVEN if your finance team mandates banker’s rounding for tax calculations) rather than left to whatever the ambient decimal.Context happens to be configured with in that process. A max_digits constraint additionally requires that the total digit count — integer part plus dd fractional digits — never exceed the column width the value will eventually be written to, which is precisely the check that catches a value that quantizes cleanly but still would not fit NUMERIC(12,4).

Configuration Reference

Field-level precision should mirror the destination column exactly, not a convenient round number. Set max_digits from the target table’s NUMERIC(p, s) definition and decimal_places from s.

Field Pydantic type max_digits decimal_places Extra constraint Notes
quantity_received Decimal 12 4 ge=0 Fractional units (kg, L, m) need 4dp; eaches can use 0dp.
unit_price Decimal 14 6 gt=0 Sub-cent precision absorbs FX-converted unit costs.
line_total Decimal 16 6 ge=0 Computed field; validate as output, never accept from the feed.
exchange_rate Decimal 12 8 gt=0 Matches the precision used across the multi-currency reconciliation frameworks.
tolerance_pct Decimal 5 4 ge=0, le=1 A ratio, not a currency amount; still must reject float input.
rounding mode decimal.ROUND_* — — ROUND_HALF_UP default Pin per field; never inherit the process-wide Decimal context.
model_config.strict bool — — True Blocks Pydantic’s own implicit str/int coercion outside the validator.
model_config.extra str — — "forbid" An undocumented numeric field is a contract change, not noise to drop.

Debugging & Recovery

Precision bugs are silent by construction — a float-tainted quantity validates successfully and only surfaces three stages downstream as an unexplained variance. Recovery has to start from the quarantine record, not from the reconciliation exception:

  • Trace phantom variances to their coercion path first. Before treating a small quantity or price mismatch as a business exception, check whether either side of the comparison ever passed through float. A variance that is a repeating binary fraction (0.1000000000000000055511151231257827021181583404541015625 truncated to look like 0.1) is a coercion bug, not a shipment discrepancy, and belongs back at this validator, not in the exception queue documented for Setting Quantity and Price Tolerance Windows.
  • Replay quarantined records after a schema fix. Because validate_receiving_line never mutates the original payload, a corrected max_digits or an added currency alias can be deployed and every quarantined record reprocessed idempotently without re-fetching from the source system.
  • Distinguish InvalidOperation from ValidationError. Decimal() parsing raises InvalidOperation for genuinely unparseable strings; Pydantic’s ValidationError carries the full field path and error type. Catching InvalidOperation inside the validator and re-raising as ValueError is what lets it surface through the structured ValidationError.errors() list instead of crashing the batch.
  • Watch for rounding-mode drift across services. If the ingestion validator quantizes with ROUND_HALF_UP but a downstream aggregation service uses the Decimal module’s default ROUND_HALF_EVEN, the two totals will diverge by a cent on every batch with an odd number of .5 boundary values — pin and document the rounding mode once, at the schema boundary, and treat any other service that re-rounds the same field as a bug.
  • Alert on a rising PRECISION_EXCEEDED share, distinct from FLOAT_REJECTED. A rising float-rejection rate usually means a new integration emits typed JSON numbers instead of strings and needs an explicit str() cast upstream; a rising precision-exceeded rate usually means a supplier changed their unit-of-measure granularity and the field’s decimal_places genuinely needs revisiting.

FAQ

Why does Pydantic’s default lax coercion corrupt financial quantities if float is a valid Python number?

Because Pydantic’s default (non-strict) numeric coercion will accept a float for a Decimal field by calling Decimal(value) directly on the float object, which reproduces the float’s exact binary representation — not the decimal value a human typed. Decimal(1234.50) is Decimal('1234.5000000000000454747350886464118957519531250'), not Decimal('1234.50'). The mode="before" validator in this pattern intercepts the value before Pydantic’s own coercion runs and rejects the float outright, forcing the caller to supply a string or an already-correct Decimal instead.

Should decimal_places match the currency’s minor unit or the source system’s precision?

Match the destination ledger’s column precision, not the currency’s textbook minor unit. Most currencies post at 2 decimal places, but unit prices computed after FX conversion routinely need 6 to avoid rounding a sub-cent difference to zero across thousands of lines, and physical quantities in kilograms or liters often need 3–4dp that has nothing to do with currency at all. Setting decimal_places from the NUMERIC(p, s) definition of the table the value will eventually be written to is the only choice that prevents a value from validating successfully and then failing on write.

What rounding mode should the quantize step use for financial quantities?

ROUND_HALF_UP is the safest default because it matches how most ERP and accounting systems round for display and posting, and it is unambiguous to audit. ROUND_HALF_EVEN (banker’s rounding) is sometimes required for tax calculations specifically because it eliminates a slight upward bias across large populations of .5 values — use it only where a specific business rule mandates it, and never let two services in the same pipeline quantize the same field with different modes.