Data is still the bottleneck
What I'm learning building with AI in clinical healthcare.
People love talking about model capabilities and the hype-topics like agents and evals. Very few want to talk about the boring bedrock beneath all of it: the data.
I’ve spent the last couple of years in clinical healthcare AI at startups, and there is definitely a pattern that you notice.
Availability, completeness, richness, in that order. If the data isn’t there, nothing downstream really matters…
If the data is there, that’s step one.
If the data is limited, then your AI-powered product or workflow is limited.
If the data is not very rich or descriptive and there’s not much of note to derive from it, no worries. AI will infer what’s not there (hallucinate) and you’ll end up with a blob of nonsense JSON informing an opaque pipeline of LLM and tool calls downstream.
Please… If you are not evaluating your entire pipeline with the knowledge of these limitations, then there is nothing standing between you and factual incorrectness.
Why does this happen? Speeeeed… Gotta go fast!
Startups aren’t wrong to move fast
Speed-running to a demo is normal for a startup. I’m not knocking it. I have been part of so many teams where we did late nights to ship a product or feature. Every founder needs something to show a room full of potential customers, and that pressure isn’t the problem.
The problem isn’t speed so much as it is the things you skip (even thinking about) to get to the finish line.
Where it goes wrong
Two things tend to happen at once, and they compound each other.
Teams build the product itself with AI: code, presentations, pitch deck flavors, etc. At the same time, they use AI to ideate on top of data that’s incomplete, sparse, or just outright wrong. Asking an LLM to reason over thin or low-quality data doesn’t get you a response like “I don’t know.” It gets you a confident, well-structured answer. Just based on crappy or even non-existent data.
That is the actual failure mode. The Large Language Model is still doing what it’s supposed to, even if you want to call it a hallucination.
Confidence vs. correctness, as data thins out
In all seriousness, in a clinical setting this stops being an abstract concern. If a recommendation gets derived from incomplete extracted data and someone downstream gives it serious consideration, there’s a real person impacted on the other end of that chain.
The problem compounded via “The build trap”
Melissa Perri described the build trap years before any of this, and it still applies: teams optimize for shipping output instead of validating outcomes.
She also wrote a book -
Today we’re just layering generative AI on top and getting a faster path into the exact same trap.
But with an even more convincing demo along the way! 🫠
Deliberate product-building steps - including building a robust data platform - and moving fast aren’t mutually exclusive.
The steps are what keep the “fast” version from becoming a liability and cautionary tale six months later.
¯\(ツ)/¯