Why it matters, unemotionally
This is not an argument about which provider is best. It is an argument about optionality. Provider behaviour changes, pricing changes, availability changes, and a customer occasionally arrives with a contractual position on whose infrastructure their data may touch. Each of those is survivable if switching is a configuration change and an evaluation run, and each is a quarter of unplanned work if it is not.
The cost of building the seams is small when you build them at the start and large when you retrofit them, which is the same shape as most architectural decisions and the reason it is worth a paragraph in the design document.
Seam one: a single call boundary
Every model call in the system goes through one interface that takes a task, not a provider payload. Provider specific request shaping, retries, timeouts, token accounting and tracing live behind it and nowhere else.
The test: search the codebase for the provider's SDK import. If it appears in more than one file, the seam does not exist, and every feature that imported it directly is a separate migration.
Seam two: prompts as versioned artefacts
Prompts live as identified, versioned artefacts with their own history, not as string literals interpolated at the call site. Each one records which task it serves and which evaluation cases cover it.
This is the seam teams skip, and it is the one that hurts, because prompt behaviour is the part that is genuinely provider sensitive. When prompts are artefacts, a provider switch is a set of variants to evaluate. When they are literals scattered across forty files, it is an archaeology project.
The test: ask how you would run last month's prompt against today's evaluation set. If the answer involves a git blame, the seam does not exist.
Seam three: an evaluation suite that can run twice
The suite has to be able to run the same cases against a different provider and produce a comparable report. Without that, a switch is a leap of faith dressed up as an architecture decision, and no responsible engineering lead will authorise it, which means in practice you are locked in whatever the code looks like.
The test: whether provider is a parameter of a suite run or a constant compiled into it.
What agnostic does not mean
It does not mean every provider performs identically on your task. They do not, which is precisely why the evaluation suite is the third seam rather than an optional extra.
It does not mean writing to the lowest common denominator either. Use the capability you need. Just make sure the place where you use it is behind the boundary, so that the day you have to move, the work is bounded, priced and testable rather than open ended.
And it does not mean naming your provider on your website. A version published on a marketing page is out of date within the quarter and tells a buyer nothing about whether the architecture would survive changing it.