Fine-Tuning vs Prompting: Which AI Approach Is Right?

The question behind the question

Business owners rarely arrive asking about fine-tuning. They arrive saying "the AI doesn't sound like us" or "it keeps getting our pricing wrong," and somebody in the conversation suggests training a custom model. That suggestion is usually wrong, and understanding why saves both money and several weeks.

The two approaches solve different problems. Prompting changes what you ask for. Fine-tuning changes how the model behaves by default. And there is a third option, retrieval, that most people conflate with fine-tuning and that actually solves the problem they described.

Prompting, properly understood

Prompting is giving the model instructions and context with each request. Most people's experience of prompting is a single sentence typed into a chat box, which is why they conclude it does not work well. Serious prompting looks different: a written set of instructions describing the role, the audience, the tone, what to avoid, the format required, plus two or three examples of the output you want.

That last part matters more than anything else. Showing the model three examples of a review response you were happy with will do more for consistency than any amount of adjectives about tone. Examples are instructions the model can copy directly.

What prompting handles well

• Tone and voice, if you supply real samples.
• Output format and structure.
• Rules and boundaries: never quote a price, always mention the licence number, never use these five words.
• Task-specific behaviour that changes from one job to the next.

Retrieval: the option people actually need

If the complaint is that the model does not know your prices, your policies, your service area or your product catalogue, neither prompting nor fine-tuning is the right fix. What you need is retrieval: a system that finds the relevant few paragraphs from your own documents and includes them in the request.

This is worth understanding because it is the most common misdiagnosis in small business AI projects. Fine-tuning teaches a model patterns of behaviour; it is an unreliable way to teach it facts, and facts taught that way cannot be updated without retraining. Retrieval keeps your information in a document you can edit on a Tuesday afternoon. When your prices change, you change the document.

Fine-tuning: what it actually is

Fine-tuning takes an existing model and continues training it on hundreds or thousands of your own examples, adjusting its default behaviour. Done properly, the result needs shorter prompts, responds faster and more consistently, and can pick up a genuinely distinctive style.

The requirements are the reason most businesses should not do it:

• A real dataset. Typically hundreds of high-quality example pairs, consistent in style and correct in content. If your best writer produced them, fine. If they are inconsistent, you are teaching the model to be inconsistent.
• Evaluation. A held-back set of test cases and a way to score whether the tuned model is better. Without this you cannot tell improvement from wishful thinking.
• Maintenance. Base models get replaced. A fine-tune built on one version does not automatically move to the next, so you are signing up to redo the work periodically.
• Skills. Someone who has done it before, or a willingness to pay for that.

How to decide

Work through this in order and stop at the first yes.

1. Is the problem missing facts? Use retrieval. Put the facts in a document the system can look up.
2. Is the problem inconsistent quality? Improve the prompt and add examples. Most "the AI is bad at this" problems disappear here.
3. Is the problem cost or speed at high volume? Fine-tuning a smaller model can genuinely beat prompting a large one, because the instructions no longer need to be sent every time. This is a real and underrated reason to fine-tune.
4. Do you need behaviour that instructions cannot describe? A highly specific classification task, or a house style nobody can articulate but everyone recognises. This is the honest case for fine-tuning.

For a business running fewer than a few thousand AI requests a month, the answer is almost always one of the first two. The economics of fine-tuning depend on volume, and small volumes do not justify the setup and maintenance.

The unglamorous middle path

What works for most small businesses is a written prompt library. One well-crafted set of instructions per recurring task, stored in a shared document, with real examples attached, and a note about what to check before sending. It costs nothing, anyone can update it, and it captures the same institutional knowledge people imagine a fine-tune would.

Version it lightly. When someone improves a prompt, they replace the old one and note what changed. Six months of that produces something genuinely valuable, and unlike a fine-tuned model, it does not become obsolete when the underlying model is upgraded.

If you are weighing this against other AI investments, our guide to AI marketing costs for small businesses sets out where the money usually goes at this scale.

Questions we get

No. Training a model from scratch requires enormous data and computing resources and is out of reach for almost every business. Fine-tuning starts from an existing model and adjusts it with your examples, which is vastly cheaper. When vendors say "we train a model on your data," they usually mean fine-tuning or retrieval, so ask which.

Vendor documentation often cites a minimum in the dozens, but useful results typically need hundreds of consistent, high-quality examples. Quality matters more than quantity: a few hundred carefully checked examples beat thousands of inconsistent ones. If assembling that dataset feels impossible, that is a strong signal to solve the problem with prompting instead.

Not reliably, and expecting it to is a common and expensive mistake. Fine-tuning shapes style and behaviour rather than installing facts. To reduce invented information, give the model the correct source material with each request through retrieval, and instruct it to say when it does not know rather than guessing.

On business and API tiers, vendors generally state that a fine-tune is private to your account and that your data is not used to improve their general models. The specifics vary by vendor and plan, so read the terms attached to yours rather than assuming. If your data is sensitive, get that commitment in writing before uploading anything.