Methodology
How ColdStart Labs runs experiments
A disciplined framework for moving from problem discovery to validated product. Every experiment follows the same lifecycle — the methodology is the constant, the domains change.
The experimentation lifecycle
Each stage has a clear purpose, defined outputs, and decision criteria. The lifecycle is designed to produce actionable learning whether the experiment succeeds or fails. Speed matters — but rigour matters more.
Problem Discovery
Every experiment begins with a problem observed firsthand — in industry work, user conversations, or operational experience. ColdStart Labs does not start with solutions looking for problems. The problem must be specific, observable, and experienced by identifiable people.
The team asks: Who experiences this problem? How often? What do they currently do about it? What would a solution need to look like to be useful? Problems that survive this scrutiny move forward.
Research
Once a problem is identified, ColdStart Labs conducts structured research. This includes market analysis, competitive landscape mapping, user interviews, and domain deep-dives. The goal is not to validate a pre-existing idea but to understand the problem space deeply enough to form a specific, falsifiable hypothesis.
Research findings are published in the ColdStart Labs research library and connected to relevant experiments. This creates a growing knowledge base that informs future experimentation.
Validation
Before building anything, the core hypothesis is tested with minimum investment. This might mean conversations with potential users, landing page tests, manual service delivery, or small-scale pilots. The question is: does this problem matter enough for people to change their behaviour?
Kill criteria are defined at this stage. What specific signal, at what threshold, would cause the experiment to be paused or archived? Defining this upfront prevents the common trap of endless pivoting.
Prototype
If validation shows signal, ColdStart Labs moves to rapid prototyping. AI-assisted development compresses the build cycle from months to days. The prototype is a functional product — not a wireframe or mockup — that real users can interact with.
The team uses AI-assisted tools to handle implementation while focusing human effort on problem framing, domain logic, and user experience. The constraint has shifted from "can we build it?" to "should we build it?"
Pilot
The prototype goes to real users in a controlled environment. Not a beta with a waitlist — a working product used by actual people solving actual problems. Usage data, feedback, and behavioural observation replace assumptions.
The pilot phase has a defined duration and success metrics. The team observes what users actually do, not what they say they would do. Behaviour is the only reliable signal.
Launch
Experiments that show clear signal during the pilot phase are launched publicly. Launch means the product is available to anyone in the target audience, with operational infrastructure to support ongoing use.
At ColdStart Labs, "launch" is not the end of the process — it is the beginning of the next phase of learning. Post-launch metrics determine whether the experiment continues to receive investment.
Scale or Archive
Based on post-launch performance, each experiment follows one of two paths. Experiments showing sustainable traction are scaled — more users, more features, more operational capacity. Experiments that plateau or fail to reach threshold metrics are deliberately archived.
Archiving an experiment is considered a success, not a failure. It means the hypothesis was tested honestly, the results were clear, and resources are freed for the next experiment. Every archived experiment contributes to the lab's growing understanding of what works and what doesn't.
Operating principles
Problems before solutions
ColdStart Labs starts with observed problems, not with technology or business models looking for applications.
Speed as a feature
The faster an experiment reaches real users, the faster it generates signal. AI-assisted development compresses the build cycle without sacrificing quality.
Kill criteria upfront
Every experiment defines what would cause it to be archived before any code is written. This prevents the common trap of sunk-cost-driven pivoting.
Behaviour over opinion
What users do matters more than what they say. Pilot phases observe actual behaviour, not stated preferences.
Transparent documentation
Successes, failures, and lessons are documented publicly. The research library and journal create institutional knowledge that compounds over time.
Parallel exploration
Running multiple experiments across domains simultaneously generates more learning per unit time than sequential, single-focus approaches.
See the methodology in practice
The research library documents findings from each stage. The journal tracks real-time progress. Case studies show complete experiment journeys from problem to outcome.