In short
Running a model to get an answer. It costs money per call in the cloud, or electricity on your own machine. Where it runs is a choice.
Inference is the act of using a trained model: send it text, get text back. Every run of an agent that reads or writes is inference.
It can run in the cloud, billed per use, with the strongest models available. Or it can run on a machine in your office, with a smaller open model, at no per-call cost and with the data never leaving the building. Both are legitimate. The choice depends on how sensitive the data is, how much volume there is, and how good the answer needs to be.
We have built systems that enrich tens of thousands of records on local inference at zero cloud cost, and systems that use the best cloud model for a hundred careful decisions a day. The instruction files are the same shape either way.
Example
What it looks like.
A lead engine on a desktop machine: scrape, classify, enrich, score, overnight, on a local model. The only bill is power.Related
Read next.
- modelThe intelligence an agent calls when it needs to read, write, or decide. Replaceable. When a better one ships, the same instructions run on it and the results improve.Read the definition
- railsOur hosting, email, backend, and models, run for you. One bill, nothing to babysit, always on current software. Your files stay portable.Read the definition
- trust boundaryThe written line between what an agent may touch and what it never does. Set before anything runs. Autonomy and safety move together.Read the definition