Thinking evolves.
The earth is round, not flat.
The earth revolves around the sun, not the other way around.
Lightning is electricity, not the wrath of the Gods.
Win Rates Are Curves, Not Numbers
A RevOps friend mentioned to me that they compute win rates following instructions they found in a blog. She was having trouble reconciling her win rates with ours.
This difference is not an esoteric nuance. Win rate is your most critical sales KPI. If computed carefully, it powers realistic forecasts, guides resource allocation, and helps you sell more. Win rates can answer questions like:
What should you expect to sell this quarter? Next year?
How much demand generation do you need to meet your plan?
Which market segments should you focus on to maximize sales?
Many blogs discuss two traditional ways to compute win rates. The blog my friend referenced is one of the more thoughtful and thorough ones. To its credit, it underscores flaws of traditional approaches—volatility and outdated data.
But traditional methods also fall short due to selection bias, manipulation, and measuring the wrong outcomes. They can’t be used for tasks like forecasting, resource allocation, or comparing cohorts.
The core issue is that win rates are curves. It makes no sense to state a win rate without also saying how long it takes to get there. We review the traditional methods detailed in the blog and present an alternative—Time-to-Event approach (TTE)—that addresses all of the shortcomings.
We recommend you:
Recognize that win rates depend on time—how long you have to work a deal
Abandon traditional win rate calculation methods
Adopt the Time-to-Event approach
Use TTE to help you sell more
Shortcuts:
Flaws in the traditional approaches
The blog my friend referenced suggests you:
Classify deals into their terminal states—won, lost, or other (derailed or no decision). Then apply three different "win rate" calculations:
Win rate, narrow = wins / (wins + losses)
Win rate, broad = wins / (wins + losses + derails)
Close rate = wins / (all deals).
Choose one of two methods to summarize “terminal” states.
Milestone (aka period) analysis: calculate win rate during a measurement period.
Flow (aka cohort) analysis: select a group of deals created in some period and calculate win rate at future points.
Which method and formula would my friend reconcile to ours?
The blog provides guidance; “… (milestone/period) win rates bounce within a fairly broad zone.” That’s right! Win rates computed this way are highly volatile [1], leading to false complacency or false concern.
Because of this, the blog advocates using flow/cohort analysis. But, they point out, “The big downside of flow analysis is you end up analyzing ancient history... If you analyze too early, too many opportunities are still open." And you'll often deal with limited data. [2]
So, that’s six different ways (3 formulas x 2 analyses) to get either volatile “win rate” measurements, or to analyze ancient data. [3] That’s not sounding very appealing. But there’s more.
What are these measuring
The blog shows flow/cohort examples measured at one and two years (after the cohort selection period). These are estimates of long-term win rates. That is, the proportion of deals won (using different denominators!), if you wait forever.
Though not as obvious, milestone/period analysis also estimates long-term win rates. It looks at deals resolving to a terminal state in a period—regardless of how long ago they were created. There are old deals in that calculation too.
What else is missing from the traditional approaches
In addition to the volatile measurements and ancient history issues, there are other important shortcomings that these methods elide:
Win rate is a function of time (how long you've been working a deal). The long-term win rate may be useful sometimes, but you usually don’t want that.
Timing of lost, stalled, derailed classifications can be arbitrary. You know when you win, but the timing of other terminal states are less certain. This uncertainty often invites gaming to distort results. More frustrating, batches of deals are often declared lost or derailed in "clean ups,” well after they were really lost/derailed. [4]
Two of the three formulas omit open deals. This is selection bias and leads to optimistic results.
Each of these issues makes the calculations less meaningful.
Despite the limitations of flow/cohort analysis, one advantage is that it can produce a coarse estimate of win rates at different times after starting. Simply track the wins from a cohort over time. [5] In fact, there is an accurate way to estimate win rates over time without any of these limitations!
Let’s back up. What do you want to use the win rates for? Typically, it’s for sales forecasts, or resource allocation (by comparing the performance of various business segments). For these, you need a reliable, statistically powerful computation.
The traditional methods discussed above simply can't make statistically valid comparisons (without a massive transaction volume). And for sales forecasts, you are typically predicting for a short-term, like the current quarter (< 90 days). So, you need an accurate short-term win rate. Not a volatile estimate of a long-term win rate.
The alternative, Time-to-Event (TTE)
Fortunately, the solution is to simplify. One method replaces all six. Use the Time-to-Event method to compute win rates and avoid all the issues discussed. The key insight is that, instead of a single number, TTE produces a win rate curve. Here’s what that looks like for a small example dataset (see appendix).
This says that the cumulative win rate of these example data at five quarters after starting an opportunity is 40%. And that the cumulative win rate after one quarter is 3%. You can see the danger of using the long-term win rate (40%) if you are trying to forecast for the quarter.
There is a lot of other insight that can be gleaned. The most likely time for a deal to be won is between 1 and 2 quarters after starting (where we see the biggest jump in cumulative win rate). And, if an opportunity has not closed within five quarters after starting, that there is empirically no incremental evidence of closing such a deal. Can you win a deal after five quarters? Of course. It just hasn’t happened yet in this data. With statistics, you can quantify how unusual an event that would be.
TTE analysis does not need to be limited to a cohort in the distant past. It can accommodate all data, including the most recently created opps, with daily (instead of quarterly) resolution. Moreover, TTE can even pick up changes in short-term win rates—something you can’t do any other way.
This analysis can be segmented however you like (Opportunity Type, Region, Product, Team, RFPs, sales vs. competitor A…). It produces statistically powerful comparisons which could be used, for instance, for resource allocation. And, TTE models can be created for entry into each stage in the sales process. These curves can power high-fidelity open pipeline forecasts. For example, here is a daily win rate curve (on a larger dataset) for multiple sales stages. The shaded areas are 95% confidence intervals on the cumulative win rate measurements.
Summary
None of the six win rate methods that my friend referenced reflect that win rate is a curve that depends on time—not a single number. It makes sense to use a model that reflects this. TTE is a simple and remarkably effective temporal model of win rate. Models like this have been proven in engineering, epidemiology, actuarial, and many other fields over decades. It's high time they are adopted in sales.
See how to sell more. Try Funnelcast. |
Appendix
How to calculate win rates with TTE
Let’s use this small sample dataset. Imagine, you are at the end of Q8.
For TTE, you care about wins. Losses, derails, stalls, no decision, paused, anything else… These are all equivalent: NOT wins. The idea is to align all of the starting times, as if each deal in your measurement period were started at the same time. Then measure how long it takes to win each deal that is won.
When we ignore the losses and derails, many deals are still open. Some deals haven’t been “in play” for as long as others. Knowing for how long they have been in play is the key to adjusting for this in the computation. With that in mind, here is a table view of the data:
Deal | Created quarter | Won quarter | Won |
1 | 1 | 5 | 1 |
2 | 1 | 8 | 0 |
3 | 1 | 3 | 1 |
4 | 1 | 8 | 0 |
5 | 1 | 8 | 0 |
6 | 1 | 8 | 0 |
7 | 1 | 8 | 0 |
8 | 1 | 3 | 1 |
9 | 1 | 8 | 0 |
10 | 1 | 2 | 1 |
11 | 1 | 8 | 0 |
12 | 1 | 8 | 0 |
13 | 1 | 8 | 0 |
14 | 1 | 4 | 1 |
15 | 2 | 8 | 0 |
16 | 2 | 4 | 1 |
17 | 2 | 8 | 0 |
18 | 2 | 7 | 1 |
19 | 2 | 8 | 0 |
20 | 2 | 8 | 0 |
21 | 2 | 8 | 0 |
22 | 2 | 8 | 0 |
23 | 2 | 8 | 0 |
24 | 2 | 4 | 1 |
25 | 2 | 6 | 1 |
26 | 2 | 5 | 1 |
27 | 2 | 8 | 0 |
28 | 2 | 4 | 1 |
29 | 2 | 8 | 0 |
30 | 2 | 4 | 1 |
Note that for open Opportunities (not yet won), we have entered the current quarter (Q8) but they are marked with a zero for not won.
Here is the recipe for TTE. You can replicate this in a spreadsheet or you can use our Excel TTE win rate calculator. Just paste your data into it. Everything else is automatically computed. Or you can register for the free FUNNELCAST Essentials and see your win rates instantly.
Select a measurement period (Q1 to Q8 in this example). The dataset provided does not contain opps created after Q2 but you can include opps right up to the current time.
Assemble a list of all the opportunities with a create date at any time in the measurement period.
Build a table with three columns: Time (from creation date), Wins, and “In_play”.
Populate the Time column with intervals chosen at your discretion. In this example 0, 1, 2, 3… quarters. You typically should do this in days.
In the Wins column, tally the number of opportunities that were won in the first interval (0 – 1 quarter in this example)
In the In_play column, tally the number of opportunities that were in play (open) at the start of the first period. Include opportunities that were won in the first interval.
Repeat 5 and 6 for each interval.
Compute a fourth column called W(t) = Wins/In_play for each interval.
Compute a fifth column called WinRate (t) = W(t)*(1-WinRate(t-1))+WinRate(t-1). This is your cumulative win rate as a function of time.
Plot Win Rate(t) on the y-axis vs. Time on the x-axis. This is your win rate curve.
When finished with step nine, you should have a table that looks like this.
Time (Qs) | Wins in Q | In play (at start of Q) | W(t) | Win rate (t) |
0 | 0 | 20 | 0% | 0% |
1 | 1 | 20 | 5% | 5% |
2 | 3 | 19 | 16% | 20% |
3 | 1 | 16 | 6% | 25% |
4 | 2 | 15 | 13% | 35% |
5 | 1 | 13 | 8% | 40% |
6 | 0 | 12 | 0% | 40% |
7 | 0 | 5 | 0% | 40% |
And here are sample formulas for spreadsheet geeks. Have fun!
Footnotes:
A longer measurement period reduces volatility.
A longer cohort selection period helps. But, there’s a tradeoff between how broad an opportunity-created window you have (to get more data in a cohort) and how useful your measurements are because the windows are so large.
Maybe five ways. The blog did not show using “Close rate” with milestone analysis.
This contributes to the volatility of win rate (narrow) and win rate (broad).
In the blog example, the cohort selection is in Q1, and one measurement is one year later, at the end of Q5. It is wrong to think of this as the one-year win rate. Deals in that cohort will have had between up to three and up to five quarters to be won. It’s a very coarse measurement. A shorter cohort and measurement period could help, but at the expense of a more volatile measurement because there will be fewer deals.
Comments