Should a small business train its own AI model? Almost never
Training your own model sounds powerful. For most SMBs it is expensive, slow, and unnecessary. Here is what to do instead.
For most small businesses, the answer is no. Training or fine-tuning your own model is a six-figure commitment of time, data, and infrastructure that solves a problem you almost certainly do not have. Retrieval, good prompting, and sensible system architecture will get you 95% of the outcome at a fraction of the cost.
That is the honest answer after building AI systems for SMBs across professional services, hospitality, healthcare, and education.
Why do people think they need to train a model?
Usually because someone told them their data is special. And sometimes it is. But "special data" and "needs a trained model" are two very different things.
The pitch goes like this: your business has years of proprietary knowledge, your customers speak in a specific way, off-the-shelf models do not understand your industry. Therefore you need a custom model. This is almost always wrong, and it is usually sold by people who charge by the project.
Foundation models like GPT-4o, Claude 3.5, and Gemini 1.5 have already ingested more domain knowledge than any SMB will ever produce. The gap is in how you connect the model to your specific context, not in the model's general capability.
What is the actual difference between training, fine-tuning, and retrieval?
These three things get conflated constantly, so here is a plain distinction.
Training from scratch means building a model on your own data, from nothing. Cost: millions of pounds and months of compute. Relevant to: OpenAI, Google, Anthropic. Not you.
Fine-tuning means taking an existing foundation model and continuing to train it on your specific data to shift its behaviour or style. Cost: thousands of pounds, weeks of engineering, ongoing maintenance. Relevant to: large enterprises with highly regulated outputs or genuinely unusual language tasks. Occasionally relevant to SMBs. Rarely.
Retrieval-augmented generation (RAG) means giving the model access to your documents, data, or knowledge base at query time, so it can answer from your actual content without any training at all. Cost: low. Time to implement: days to weeks. Relevant to: almost every SMB use case.
The question most businesses should be asking is not "should I train a model" but "have I given the model the right information to work with".
When does fine-tuning actually make sense for an SMB?
Rarely, but not never. Here are the conditions that genuinely justify it.
- You have a highly consistent, high-volume output task where style and format matter more than factual retrieval. Think: a legal firm generating standard clauses in a very specific house style, thousands of times a month.
- The task is classification or extraction at scale, not generation. Fine-tuned smaller models can be faster and cheaper than large models for narrow classification jobs once you are processing millions of records.
- You have clean, labelled training data already. If you need to create the data first, the economics collapse immediately.
- You have engineering resource to maintain the model as the world changes. Fine-tuned models go stale. Foundation models update underneath you.
If all four conditions are true at once, fine-tuning is worth a conversation. If any one is missing, start with retrieval.
What should you do instead?
Three things, in order.
1. Write better prompts and system instructions
Most AI outputs that feel wrong are wrong because the instruction was vague. A well-constructed system prompt, with clear persona, constraints, output format, and examples, will outperform a poorly prompted fine-tuned model. This costs nothing except time and thought.
2. Build a retrieval layer
RAG is the workhorse of practical business AI. You take your documents, your product data, your historical quotes, your policy manuals, chunk them, embed them, store them in a vector database, and retrieve the relevant chunks at query time. The model then reasons over real, current, specific information rather than guessing from general training.
This is how our Brain Builder product works. You feed it your knowledge, it becomes a working AI brain for your business, without touching a single training run.
3. Wire the model into your actual systems
The biggest gains come not from smarter models but from models connected to live data. A model that can read your CRM, check your calendar, query your database, and write back to your systems is worth ten times a model that just generates text in isolation.
This is where the work is. Not in training. In integration.
S&P Global found that the share of organisations abandoning most of their AI initiatives rose from 17% to 42% in a single year. Most of those failures were not model failures. They were integration and expectation failures.
The pilot-that-dies-in-committee problem
There is a specific failure mode I see repeatedly. A business spends three months scoping a training project, gets a quote that frightens the board, and shelves the whole thing. Meanwhile, a retrieval-based prototype could have been running in a fortnight.
Around 95% of enterprise GenAI pilots see no measurable return. The ones that do return something tend to be narrow, wired into real workflows, and built fast enough to prove value before the budget conversation happens.
The enemy here is the consultancy that sells you a six-month discovery phase before anything is built. By the time they finish the discovery, the model they specified is already outdated.
A practical decision framework
Before anyone spends money on model training, run through this.
| Question | If yes | If no |
|---|---|---|
| Can retrieval solve this? | Start there | Consider fine-tuning |
| Do you have 10,000+ clean labelled examples? | Fine-tuning viable | Not yet |
| Do you have ongoing engineering resource? | Fine-tuning sustainable | Avoid it |
| Is this a classification task, not generation? | Fine-tuning efficient | Probably overkill |
| Is the task high-volume and repetitive? | Economics may work | Unlikely to justify cost |
If you get to the bottom of that table and fine-tuning still looks right, it might be. For most SMBs reading this, you will not get past the first row.
What this looks like in practice
At our Cook-a-Long sessions, we have run more than 30 workshops with around 200 participants. In the first 11 sessions alone, attendees built 41 working tools. Not prototypes. Working tools, in the room, on the day.
None of them involved training a model. Every single one used prompting, retrieval, or system integration. That is what works at SMB scale.
The British Chambers of Commerce reports that half of UK SMEs now use AI in some form. The businesses getting real value from that are the ones treating AI as a workflow problem, not a model problem.
The one-sentence version
Your data access, your prompts, and your integrations are the bottleneck. The model is fine.
If you are weighing up whether to train a model or build something that actually works, the consultancy conversation is the right starting point. We will tell you what you need, not what sounds impressive.