Problem/Solution Reviews: how to ensure you (& your team) are building the right thing

04 Jun, 2026

Problem/Solution Reviews: how to ensure you (& your team) are building the right thing

The unreasonable effectiveness of Problem/Solution Reviews

The best AI Product & Engineering leaders are using Problem/Solution Reviews to ensure their team is actually building the right product instead of just building software for the sake of it.

At some point in Column Tax's lifecycle, we scaled the number of projects we were working on to the point that I could no longer lead every single one¹. As co-founder and head of Product & Engineering, this was a big transition. I reached out to folks I knew and was lucky to get an amazing template from Erik Goldman (co-founder of Vanta). It helped us make sure we kept building the right things, even as I was managing more. The template was so helpful that I wanted to memorialize it (in this blog post!) for other founders and Product & Engineering leaders.

In a race, you would never want to run faster in the wrong direction

Building software is moving faster than ever. In tandem, making sure that you're building the right software is more important than ever.

It used to be the case that you'd want to think really hard about what you were building before you built it because building things took a long time and lots of engineers. Now, building things (at least prototypes) is cheap.

AI increases execution speed but not problem alignment. If you're in a race, you would never want to run faster in the wrong direction (in fact, you'd rather not move at all!)

In the age of abundant software (and equal amounts of software slop), in order to stand out, you have to deliver real value to your customers: and that means thinking hard about what to (quickly) build. When it's easier than ever to ship, you have to make an active choice to think deeply about solving your customers' actual problems.

Your methods for prototyping Solutions are different today than they used to be (e.g. use Codex to create 5 real sample UI prototypes instead of having a Designer create static designs in Figma), but the importance of solving real customers problems is the same.

The two key questions that determine if projects succeed

While projects have lots of different phases and decisions to make, there are always two key questions that changing your answer to will fundamentally change the project itself:

1. What is the exact problem we're solving?
2. How are we planning to solve it?

The better the answer to those questions, the smoother the project will go: this means clarity (e.g. what are we not solving?), alignment (i.e. stakeholders are all on board), and correctness (i.e. we have very good, evidence-driven reasons to solve this problem and not another).

Almost all failing projects fail because of issues with those two questions² above. Therefore, if you want to maximize your leverage as a Product leader, you should put your energy into making sure that every (non-trivial) project succeeds at these levels.

Once you've established the answers to these two key questions, if you have extra time, you can decide whether to audit the placement of e.g. individual buttons in a design review. But if you have great button placement for a feature that should never have been built at all, you'll have wasted your time.

The product building flow should be:

Customer -> Problem -> Solution -> Implementation

If you change a thing on the left, you will change everything to the right. For example, if you change the Problem you're solving, you'll need to change the Solution and Implementation.

Product leaders should spend their time accordingly, since small mistakes on the right (e.g. an Implementation mistake) will get overshadowed by mistakes to the left (e.g. solving a Problem the Customer doesn't actually care about). So spend most of your time on the important things on the left!

The problems, in short

• As your team & the number of projects grows, you want to make sure each project is important (truly needed) and focused on the right goals
• Engineers like building things, and you want to avoid solutions in search of a problem (time wasted on things that don't actually matter)
• It's bad to spend X weeks/months building something only to find out you never needed it

The fix: Problem & Solution Reviews

The solution to these problems is a tool called "Problem & Solution Reviews". When implemented correctly, these templates/reviews will ensure that projects solve a real customer problem and are set up well for implementation/execution³.

Thinking about projects in two parts, “Problems” and “Solutions”, is a valuable framework to clarify your (and your team's) thinking about your customer and product strategy and align your team around the correct goals.

The Problem Review (example below)

The Problem Review is a tool to concretize a vague idea into a clear understanding of an important customer⁴ problem and ensure you and the team agree it’s worth solving.

For any sufficiently large project (say more than 2 weeks of work), require that the DRI (Directly Responsible Individual / lead) of the project start by writing a Problem Review and get your sign off before spending more time on the project.

Here is the Problem Review template I've used.

This template is meant to help clarify the writer's thoughts around a set of problems. You & your team will then review the write up to make sure folks agree this is a problem worth solving before spending valuable execution cycles.

Here's how to use it:

1. Start with a vague prompt/idea of something that's not yet well defined but you're hearing or is floating around (e.g. "we need better search functionality!")
2. Gather real data⁵ about the problem: support tickets, customer calls, user research, product analytics, etc.
3. Group the data into named sub-problems (e.g. "search returns no results for searches with punctuation")
4. Create impact & reach assessments for each sub-problem (e.g. "users have no workaround to searching except for looking at every single page" and "30% of users who search make a search with punctuation")
5. Decide which are worth solving now and which can wait (e.g. "we'll work on problems 1 and 3, problem 2 we can push off for at least another month because it has low impact given that there's a workaround in place")
6. Set goals & measurable outcomes for solving the problem(s) you decided on

Important: do. not. solve. anything. in. this. phase.

People will be tempted to write down solutions instead of problems, but lack of a feature is not a problem in and of itself! An example of a bad problem statement is "we need to build better punctuation support into our search engine." Why? What specific issue is that causing for customers?

Well-defined problems should consist of a clear story/description of what is happening today, what issues that causes for the user/customer/your infrastructure/your close rate/churn/etc., and why those issues logically flow from the current behavior.

An example of a good problem statement is "our SaaS search product does not properly handle punctuation marks, leading to poor results when users try to search for programming language names like C++ or C#, which are 30% of our queries. The median user searches 2x/week, so this harms [specific workflow they're using search for] and causes significant pain for them and XX support tickets for us per week".

You'll end the Problem Review with very clear goals & success metrics for the project, e.g. "reduce support tickets related to searching by 20%". Metrics are extremely focusing. Having a clearly defined success metric for a project aligns the design & engineering team to solve a problem by any solution necessary vs. just shipping a feature where the team is just trying to parrot the requirements handed down from above (and often gets them wrong).

By agreeing on success criteria, you're much more likely to get the (good) pushback from engineering saying "I don't see why we need to build this to reduce support tickets by 20%" vs. just shipping some (bad) opaque requirements because they were blessed by a PM.

Problem Review Example

Here's an example of using the template:

Look closely at this example. The 2nd sub-problem is "Example: partners are asking for more-detailed funnel event data about users in terms of where they are in the filing flow".

This is an example of a problem ("the problem is that the customer asked me for [feature]") that needs more definition. What we want to know is "why?" What are they going to do with that data? (Run marketing campaigns.) What goal are they really trying to accomplish? (Get more users to convert.)

This is the core function of the problem review. To get beyond "we need more stats about the funnel" and to the real problem: "get more users to convert".

In a real Problem Review, we'd want to get to this level of detail/clarity, e.g. (making up an example): "[Job role] at our partners have internal KPIs for the tax completion funnel and are unable to report on them to executives because we don't surface sufficient data to them. The KPIs they want to report are X, Y, and Z and they want them presented weekly on an internal dashboard accessible by [ABC]."

Then, when moving into the Solution Review, we can ask: to achieve that, should we surface funnel statistics via the API? Should we have someone from customer success just pull the numbers for them and email them over? Or write a script that does the emailing? Should we make a dashboard on an internal page they can access? All of that sits in the Solution Review and should be kept out of the Problem Review but that nuance gets lost when the Problem is poorly stated as, e.g., "they asked us for API access to X".

The Solution Review (example below)

Once you've agreed that a problem is worth solving and what the goal of solving that problem is, you can finally start actually solving it!

"Thorough divergence, thoughtful convergence" is a fancy way of saying the goal of a Solution Review is to exhaustively think of every possible way to solve a problem, then make sure you actually pick the right solution to move forward with.

Here is the Solution Review template I've used.

The template is a forcing function to make sure you're thinking through all possible solution options and thoughtfully & practically choosing the best path forward⁶.

Once the team is aligned on & fully understands the specific problem from the Problem Review (not, "search is bad!!!", but "search isn't picking up punctuation properly, and that's causing bad results for users searching for programming language names") -- we can then ask "how might we solve this?" & brainstorm with the team.

Continuing the search example: we could write specific code to detect a few common programming language names and run them to a different search backend. We could file a feature request with our search API provider asking for better punctuation support. We could find a different search API provider that does a better job of this. We could build our own search service and stop using our current API provider, etc. etc.

The goal of this phase is to align with the engineering & design team on the specific problem and make sure they understand it very deeply. They'll also start thinking through solutions at a conceptual not just technical level, which help them to avoid building something without considering the wider implications and pros/cons.

Finally, we work with the team to ask and answer questions important for choosing the ultimate solution like estimates for building different options. For example: how long would it take to build our own search service? How open is our search API provider to building punctuation support and on what timeline? How diverse is our set of queries, or is it just that people search for C++ and C# where we can build filters for just those two phrases? etc.

Finally, we choose the natural solution that clearly flows from those questions and their associated answers. For example: the search API provider doesn't want to include punctuation support, it would take 5 months to build our own service, and 98% of punctuation queries are just the phrase 'C++' or 'C#' so we're just going to write special if-statements for those two queries and call it a day.

Implementation & execution

After you've aligned⁷ on the Solution, the Engineering DRI can blow that out into a full technical spec and begin executing⁸!

A few more notes

1/ IMO, every function should be able to do this type of "product" thinking & create these documents. Engineers in particular should be expected to be able to think through the importance of what they're building before they build it (even more so in today's world). You should probably make it an expectation of the Engineering role that someone (at least at a mid/senior level) can DRI a project from start to finish, including writing the Problem/Solution Review.

2/ The easiest way to scale yourself is to outsource these steps for projects. Start by delegating low-complexity projects and grow your delegation muscle over time. Ask a CSM or Support Lead to write a Problem Review for a feature they keep bugging you about. Ask a product-minded Engineer or analytical Designer to write a Solution Review. If an engineer keeps nagging you about a feature that "would be so cool", encourage them to write a Problem Review for it and maybe they'll discover that it's not actually solving a user need. Or maybe it is actually solving a need, in which case, great! That's free work you didn't have to do.

3/ Don't require this lifecycle for everything. It's powerful but somewhat heavy. Use these templates for complex features, highly strategic projects, or with junior folks where you want their work on rails. Or if something a PM/Engineer/etc. is proposing doesn't smell right, call for a Problem Review to confirm that everyone understands what they're doing before moving forward.

Templates & examples

• Here is the Problem Review template again.

• Here is the Solution Review template again

• A combined Problem/Solution Review template (a "lite" template) -- use this template for smaller projects or where the problem or solution is more obvious

• An example Problem Review

• An example Solution Review

Good luck and have fun building the right thing!

Thanks to Erik Goldman, Nadia Eldeib, Zach Ozer, Steven Schmatz, and Gavin Nachbar for reading & providing great feedback on drafts of this post.

To get notified (extremely rarely) when I publish a new post, subscribe here: