Podcast

6 min read

Why does business AI stop where the data gets interesting?

Season 2, Episode 1 of Beside Ourselves: The case for going below the surface

Most SMEs have adopted AI, but only at the surface. The real value lies in customer records, contracts and financials — exactly the data company policies keep away from AI, and for good reason: commercial LLMs were built by labs for exploration, not for trust. The answer isn’t to give up on that data. It’s about stopping treating AI as a single package and starting to own the stack it runs on. It’s more achievable than most leaders think.

Download the free guide

We’re back. After a break, Season 2 of Beside Ourselves kicks off in a Cambridge co-working room (brand undisclosed, though our coffee cups may give it away), and with a slightly different plan.

Last season, we kept circling back to one idea: there’s an awful lot of attention on the latest frontier models and what’s happening in Silicon Valley, and not much on what an SME actually needs to build on. So this season we’re taking that one theme and following it all the way down.

The short version: most businesses are using AI in a shallow way. Deploy one of the mainstream LLMs, let people draft and summarise, job done. There are sensible reasons for stopping there. But it’s not what this technology promised, and the missed opportunity is enormous.

Would you run your website on LinkedIn?

Here’s the analogy we keep coming back to.

Running your business on a commercial LLM alone is a bit like running your website on LinkedIn. You get the whole stack in one tidy package, it costs next to nothing to get started, and the reach is real. But it’s not your brand, it’s not your workflow, and once you’ve posted something, it isn’t entirely yours any more.

Nobody would seriously suggest a company host its website on a social network. We all learned long ago to keep the reach of social media and build our own platform alongside it, where we choose the hosting, the content system and the experience. AI is heading for the same resolution. Most firms just haven’t got there yet.

Five things we discussed

1. The value is in the red data

Giles has a simple way of thinking about data: traffic lights.

  • Green is public — your website, social posts, anything someone could find without being on your network. Frankly, it’s largely what the LLMs were trained on anyway.
  • Amber is internal but less sensitive — product specifications, project plans, process documents.
  • Red is the sensitive stuff every organisation holds — client data, HR files, financial records, anything regulated.

The awkward truth is that value climbs as the lights change. The green zone rarely moves a number. The amber and red zones are where the useful work lives, and they’re also where the risk lives.

Theo shared an example from a financial services firm. Clients would ask their advisers perfectly reasonable questions — how’s my portfolio doing, why did it behave that way — and expect a quick answer, given all the tools on the adviser’s desk. The answer was: sorry, we’re not allowed to give the AI that information. All that power at their fingertips, and they couldn’t answer a simple question.

2. Your AI policy may have written itself (and it shows)

We’ve all done it. Somebody says “we need a policy for X”, and the first draft comes straight out of ChatGPT, Claude or Gemini. Which goes some way to explaining why so many AI policies look identical.

A good AI policy doesn’t need to be War and Peace. One side of A4 paper can do the job. Start with why you’re using AI and what you want to gain. Map your data into the traffic lights and say plainly what can be used where. Make sure someone owns the policy — and that everyone using AI understands they’re still responsible for what it produces. That includes the policy itself, if AI helped write it.

3. Commercial AI was built for exploring, not trusting

Strip those policies back, and they all trace to the same roots: GDPR, the UK’s data protection rules, ISO standards and the other accreditations firms work towards. And that’s where the conflict sits. Commercial LLMs weren’t designed with any of that in mind.

None of this is villainous. The frontier labs are open about what they’re chasing, and fair play to them. But they call themselves labs for a reason. Imagine Salesforce or Oracle telling you there’s a small chance their product might end civilisation, but you should still buy the enterprise licence. That’s not normal business software.

Normally, technology goes from science to product before a business buys it, with someone in the middle turning it into something you can govern and depend on. With AI, many firms are buying straight from the lab and skipping that middle step entirely.

4. Context is the missing ingredient

Out of the box, LLMs have broad world knowledge and almost no knowledge of you: your customers, your products, how you work. You can supply some of that through good prompting and well-configured projects. But the richest context sits in the amber and red zones.

That matters most for the questions that start with “my”. What’s the status of my complaint? What’s my balance? When does my contract end? Answering those means connecting AI to the data that your policy currently excludes.

5. The model is only one layer of the stack

This is the bit that tends to surprise people. You don’t have to take AI as a single package from a single vendor. Like any IT system, it comes in layers — hosting, model, gateway, interface — and each can be chosen separately.

That opens up options. You can host in a region that meets your regulators' requirements. You can use European models such as Mistral, or open-weight models hosted in Europe or on your own hardware (some run happily on a fairly modest machine). You can put a gateway in front of everything and an interface on top that carries your brand. And you can use different models for different traffic lights.

Set up properly, swapping one model for a better one becomes a fairly routine job rather than a rebuild. Something that sounds more complicated actually makes you more nimble.

It’s less daunting than it sounds

We won’t pretend this is as simple as downloading an app. It needs more thought up front, and some investment.

But it isn’t rocket science, either. Giles compared it to building a website: some firms have the skills in-house, many bring in outside help to set it up, and either way the business owns and runs it afterwards. You won’t need to recruit a department of data engineers. There are open-source options at every layer; you can pick and mix, and you can start small and expand as confidence grows.

And in return you get control, flexibility, security, a much easier conversation with your auditors, and access to the use cases that only open up when the red data comes into play. In an increasingly unpredictable geopolitical climate, that control is worth rather more than it used to be.

Coming up this season

Over the next few episodes, with help from some expert guests, we’ll unpack the privacy and accreditation requirements behind all this, the use cases customers are starting to expect, and what a well-designed AI stack looks like layer by layer — including which models suit which jobs, and how to keep cost and complexity under control.

Download: Below the Surface

A deep dive into full-trust AI for SMEs. Real AI adoption needs sensitive data, and commercial LLMs can’t be fully trusted with it. Our free 14-page guide sets out what full-trust AI requires:

  • 6 models compared, beyond the frontier labs
  • 5 layers of an owned AI stack, explained
  • 4 steps to get you started

Download the free guide

Prefer to talk it through? Book 30 minutes with Giles or Theo — no agenda, no pitch, straight answers.

A deep dive into full-trust AI for SMEs.

Below the Surface: full-trust AI

Real AI adoption needs sensitive data, and commercial LLMs can't be trusted with it.

A full-trust AI stack is easier and more affordable than most think.

We compare 6 models, explain 5 layers and outline 4 steps to start.

Read the free guide
Giles Thurston and Theo Paraskevopoulos