Should we fine-tune a model or just write better prompts?

What it is

This is a build decision for teams automating a repeatable task, such as classifying tickets or drafting product copy in a fixed format. Fine-tuning trains a model further on your examples. Prompting changes only the instructions and examples you send.

How it works

1. Write a clear prompt with a few examples, and measure it with an eval. OpenAI describes optimization as a loop of evals, prompt engineering and fine-tuning.
2. If the model lacks facts, add retrieval (RAG) rather than tuning.
3. If format is still inconsistent at volume, try structured outputs.
4. Only then test fine-tuning on a held-out set, and compare cost and quality against the prompted version.

Vendor availability (checked 2026-10-17). OpenAI’s deprecations page says self-serve fine-tuning is winding down: new organizations lost access on May 7, 2026, and active customers can’t create new jobs from January 6, 2027. Mistral’s docs say its fine-tuning feature is deprecated, without a date. Open-weight models can still be tuned on your own infrastructure.

Benefits

• Avoids spending on training you don’t need.
• Keeps your system easy to update, because prompts and retrieved documents change in minutes.
• Gives a clear, evidence-based reason when you do tune.

Limits and risks

• Ties you to a model version. A tuned model has to be redone when its base model is retired.
• Data must be clean and permitted. Training examples need the right to be used this way.
• The service can go away. Vendors change which models can be tuned, and some are ending self-serve tuning.