pain001-loader-xlsx Architecture¶
A map of the codebase for new contributors and maintainers.
The pipeline¶
payments.xlsx / .xlsm (first worksheet; row 1 is the header)
|
v
pain001_loader_xlsx.loader.XlsxLoader registered under the pain001.loaders entry point
| load(path) / load_streaming(path, chunk_size)
| cell values normalised to the text the CSV pipeline would carry
v
pain001 (SchemaValidator, scheme rulebooks, official XSD, XML generation)
|
v
ISO 20022 pain.001 XML
The loader teaches the core to read workbooks directly, so no “Save As
CSV” step exists to lose leading zeros or reformat an IBAN. It registers
itself through the pain001.loaders entry point and the core dispatches
.xlsx and .xlsm paths to it at process start; pain001 -t ... -d payments.xlsx -o out/ needs no configuration.
Module map¶
Area |
Module |
Responsibility |
|---|---|---|
Public API |
|
Exports |
Loader |
|
|
Normalisation |
|
Cell value to text, honouring the cell’s number format so |
Tests |
|
Loading, streaming, the guard, malformed workbooks, examples, the suite conformance file |
Examples |
|
Runnable scripts, executed by the tests |
Benchmarks |
|
Load throughput, scaling and memory guards |
Suite |
|
The shared checks every suite member carries |
Key design decisions¶
The IBAN guard. Excel turns anything numeric-looking into a number. If any cell in an IBAN column arrives as a numeric type, the load stops with an error naming the row; a corrupted debtor account is the failure a bank cannot detect.
Read, never run. Formulas resolve to their last-saved values (
data_only=True); macros in.xlsmfiles are never executed.One sheet, one batch. Multi-sheet workbooks are out of scope by design; the audit trail stays unambiguous.
Streaming. Workbooks open read-only and
load_streamingyields fixed-size chunks, so a batch of hundreds of thousands of rows never has to fit in memory.Structural conformance to the core’s
AbstractLoaderprotocol, no subclassing, with the plugin API version declared so the core can refuse an incompatible pair.
Extension points¶
A new cell type or format: extend
_normalise.pyand add a workbook fixture that exercises it.A new file format: a separate loader package on the same entry point; this one stays Excel-only.
Where to look first¶
Runnable examples:
examples/Roadmap:
ROADMAP.mdRelease process:
RELEASING.mdParent library:
pain001