When you cannot call the real data source

It happens on most engagements. You find the numbers the business actually runs on, and then you find that the only way to retrieve them is a person at a desk. The workbook pulls them through a vendor’s Excel add-in. The system of record has an export button and no API. The API exists, but it is a separate entitlement nobody has bought.

You can go back to the client with a dependency and wait, and three weeks disappear. Or you can build straight against whatever data you can reach, in which case the stand-in becomes permanent, because by the time the real feed arrives the pipeline has been shaped around the substitute.

Write the contract from the downstream end

Write down what the pipeline needs, then satisfy it from a source you can reach today.

The discipline is in which direction you write it. The contract is set by what the work downstream requires: these instruments, at this grain, as at this time, in these units. Write it the other way round, describing whatever the interim source happens to hand you, and the adapter is really a coupling with an interface painted on the front. Swapping the source later then means changing everything behind it.

The swap should be a config change

Done properly, adding the real source later touches one adapter. No transforms, no models, no outputs.

The test is whether you can run both sources at once and diff them. If you can, the boundary is in the right place, and you have a reconciliation harness ready for the day the real feed turns up. That is the day you find out whether the two sources agree about what a closing price is.

Often they do not. Vendors differ on timing, on adjustments, on which venue a price came from. Running the diff over a known period tells you the size of the gap before a client asks about it.

Say where the number came from

While a stand-in is in place, that belongs in the data, and where the output is client-facing, on the output. This figure came from this source, as at this time. It costs one column.

Otherwise the numbers are right and nobody can show why. Six months on, the interim feed has been the authority for two quarters, nobody remembers it was meant to be temporary, and working out which vendor said what takes a week nobody has.

This is the normal case

Most constraints in enterprise data work look like this. The system holding the truth was never built to be integrated with, and the road to it runs through a person, a spreadsheet or a licence negotiation. Treat it as a temporary embarrassment and it hardens into the design. Treat it as a substitution that is isolated, labelled and testable, and it stays cheap to correct.