Do You Need Machine Learning, or Would a Spreadsheet Do?

When a client says “we want machine learning”, the first useful question is what decision they want to improve. About half the time, the honest answer is a few rules in a spreadsheet or a database query. The other half is where a model genuinely earns its keep, and it helps to know which half you’re in before anyone writes code.

Start with the decision, not the technique

“Predict churn” is a technique. “Call the 50 customers most likely to cancel this month” is a decision. The second one tells you what the model’s output has to look like, how often it needs to run and who acts on it. If nobody can name the person who will act on the prediction, the project isn’t ready.

Try the boring baseline first

Before training anything, write down the simplest rule that might work. Customers who haven’t logged in for 30 days. Orders over a certain value from a new account. Run it against last year’s data and see how well it does. That number is your baseline, and a model has to beat it by enough to justify the extra upkeep.

Sometimes the baseline is good enough. We’d rather tell you that than build something heavier.

Check the data honestly

Three things decide whether a model is feasible:

• Enough history. A few thousand clean rows is enough for some tabular problems. Rare events need more.
• Labels you trust. If “churned” means three different things in three systems, fix that first.
• Data that will exist at prediction time. A field that’s only filled in after the outcome is known will make your test results look wonderful and your live results useless.

Judge it on the business number

Accuracy on its own is easy to game. A model that says “nobody will churn” is right most of the time. Pick a measure tied to the decision, such as how many of the top 50 flagged customers really were at risk, and test on data the model has never seen.

Plan the hand-off from day one

A model nobody can reach is a very expensive notebook. Decide early whether it runs as an API your app calls or a nightly batch job, who owns it afterwards and how you’ll notice when it starts getting worse. Customer behaviour changes, and a model trained last year slowly drifts.

When it is worth it

Machine learning pays off when there are many similar decisions, a pattern that rules can’t capture, and enough data to learn from. Pricing, lead scoring, demand forecasting and document sorting often fit. A one-off decision made twice a year usually doesn’t.

If you’re weighing this up for a client project, send us the decision and a sample of the data. We’ll tell you whether a model is worth building, and what a simple baseline would cost you instead.

You can read more about our Machine Learning Development service on the main site.