writing · oct 2026 · 6 min read

Making an LLM layer provider-agnostic

How I put Claude in production and Ollama in development behind one interface at IPSER Labs, with structured outputs as the contract and guardrails outside the model.

  • LLMs
  • Backend
  • IPSER Labs

Every LLM product hits the same question sooner or later: what happens when you want a different model? A cheaper one for development, a better one for production, or next quarter's release. If the answer is "rewrite every feature," the model choice has quietly become your architecture.

From January to June 2026 I was the primary backend engineer on IPSER Labs' Smart Pantry app, reporting to the CTO. It had eight AI features, from scanning grocery receipts into an inventory to suggesting meals from what's in the pantry, and almost all of them needed a language model. We wanted Claude in production for quality and a local model through Ollama in development, so we could iterate without paying for every test run. That gave me one design rule for the whole backend: switching models should be a configuration change, not a code change.

  • 8AI features on one LLM service
  • 0code changes to switch models
  • 94%receipt extraction accuracy
  • 5→2 minreceipt processing time

One interface, many providers

Feature code never imports a vendor SDK. Every feature calls one internal LLM service with a task, an input, and the shape of the answer it expects. Behind that interface sit two adapters, one for Claude and one for Ollama, and environment variables decide which one runs. The service supported two deployment paths (Claude in production, Ollama in development) and two local model configurations, and moving between them took zero code changes.

Provider-agnostic LLM layer Receipt parsing, meal generation and six more features call one LLM service with a task and a schema. Environment variables route the call to a Claude adapter in production or an Ollama adapter in development. Responses are validated against the schema before anything is written to the database. Receipt parsing Meal generation + 6 more features LLM servicetask · input · schema Claudeproduction Ollamadevelopment Validateschema check,then database env vars
Features depend on the interface; environment variables pick the provider.

Simplified sketch of the pattern (not production code)

class LLMProvider(Protocol):
    def generate_json(self, task: str, prompt: str, schema: dict) -> dict: ...

PROVIDERS = {"claude": ClaudeProvider, "ollama": OllamaProvider}

def get_llm() -> LLMProvider:
    return PROVIDERS[os.environ["LLM_PROVIDER"]]()   # config, not code

def parse_receipt(text: str) -> list[ReceiptItem]:
    raw = get_llm().generate_json("parse_receipt", text, RECEIPT_SCHEMA)
    return [ReceiptItem.model_validate(item) for item in raw["items"]]  # reject bad shapes

The payoff shows up later, not on day one. When a new model ships or prices change, you swap one adapter instead of touching eight features.

Structured outputs are the real contract

An interface only works if every provider returns the same thing. Free text can't promise that, so every feature asks for JSON with a fixed shape:

  • Receipt parsing returns items with a name, quantity, unit and product metadata.
  • Meal generation returns 3–5 ranked recipes that prioritize ingredients you already have, what's about to expire, and dietary and cuisine constraints.

Once the output is structured, the rest of the backend doesn't care which model produced it, and every response can be checked: anything that doesn't match the expected shape is rejected before it reaches the database.

Put the guardrails outside the model

The receipt pipeline shows this best. A model reading a crumpled receipt photo will sometimes be wrong, so the model is only one step in a longer pipeline:

  1. Image preprocessing cleans up the photo, then AWS Textract extracts the text.
  2. Validation rules and fallback parsing filter out lines that aren't items: totals, taxes, store addresses.
  3. If the OCR looks malformed, the user reviews it in the app before anything is written to their inventory.

Across 500+ uploads that pipeline reached 94% extraction accuracy and cut processing time from about 5 minutes to 2. Most of that reliability came from the steps around the model, not from the model itself.

Treat the model as one unreliable step in a reliable pipeline, not as the pipeline.

What I'd tell someone building one

  • Design the interface around tasks, not vendors. "Parse this receipt into this schema" survives a model swap; "call Claude with this prompt" doesn't.
  • Validate every response. Treat model output like user input: check the shape and the values before you trust it.
  • Never let the model write to your database directly. Put validation, and for risky writes a human, in between.
  • Prompts don't transfer perfectly between models. Keep a handful of real examples per feature and check them whenever you switch providers.

The same idea runs this site: Guru calls Claude through one small endpoint, and if that fails it falls back to an offline engine built from the same facts.