The team wants to fine tune an open model instead of calling a hosted API. How would you decide?

Decide on four axes: quality on your specific task, cost at your actual volume, latency and control requirements, and data or compliance constraints. Start with the hosted API plus prompting and retrieval to establish a quality bar cheaply. Fine tune when you have a stable task with good labeled data, volume high enough for the economics to flip, or a requirement to keep data inside your own environment.

I would push to answer it with numbers rather than preference. First, build the hosted baseline: a good prompt, retrieval if the task needs context, and an evaluation set. That usually takes days and tells us what quality is achievable, and often it is already good enough, at which point fine tuning is a solution looking for a problem. If quality is genuinely short, I ask whether the gap is knowledge, which retrieval fixes, or behavior and format, which fine tuning fixes. Then the economics. Hosted cost scales linearly with tokens; self hosting is mostly fixed accelerator cost, so there is a crossover volume, and I compute it with real traffic projections rather than hopes. Below the crossover, self hosting loses before you have even counted an engineer's time. The cost people forget is ownership: serving, autoscaling, upgrades, re evaluation when you swap base models, someone on call. That is a permanent commitment. Where it decides itself is compliance. If the data cannot leave our environment, the decision is already made and the discussion is only about which open model, not whether.