Predictive JTBD Data and Metrics Decoded

The 30-Second Framework to Guarantee JTBD Success

This post is for those of you who want to invest in a very precise method that wraps innovation in a framework that actually measures the progress from initial problem-framing, to innovation research, and ultimately to go-to-market offerings. Did they succeed? Did they improve metrics along predicted pathways, or some other way? While this was historically time-consuming, AI has helped make this so much simpler. And your research will benefit from it.

Are you concerned about how you will measure the effectiveness of your Jobs-to-be-Done research and the ultimate implementation of strategies? You’re not alone. I’ve wrestled with this myself over the years because it’s a question that gets asked a lot.

Not only do we rarely see robust, upfront determinations of what success will need to look like, when we do they are often very subjective.

What I’m about to share is early stages, and may only apply to existing core processes, or customer journeys. I have more thinking to do on this matter. But, I do need to point out that this isn’t my idea. In Tony Ulwick’s first book Business Strategy Formulation (1999) he had some appendices about predictive metrics. When I first read the book, my eyes glazed over because I was already overwhelmed. But within the past year I revisited this chapter and tried to grasp the details of how they were employed.

There’s a bit more to it than what I’ll cover in here. But this appears to be the hard part. I’ve spoken to someone that had experienced this process on an early project and was told that this was very laborious. It’s probably why it was never incorporated into Outcome-Driven Innovation. That’s just my assumption, since I’ve never had a conversation with Tony about this.

It’s potentially extremely cool. And I’ve simplified this down to 30 seconds worth of work. 😉

Premise: People Want to Measure & Confirm Success

Many transformation strategies fail. Maybe most. Maybe >90%. The problem isn’t how well they were executed, it’s that no one bothered to identify and agree to, specifically, what improvement looked like, and how it be measured.

This is a common probably with strategy consulting. The beauty of your PowerPoint deck is not how we measure success. Implementing a process like this seems laborious, but maybe it’s the only way to dramatically increase the likelihood of success…even if you have great insights.

Concept Overview

Predictive analytics is a powerful tool that aims to forecast how well a potential solution will satisfy customer needs, even before it's fully developed. This forward-looking approach helps organizations make informed decisions, avoiding costly mistakes and maximizing value creation.

At the heart of predictive analytics lies the concept of "predictive metrics". These metrics are carefully defined parameters that are measured and analyzed before a solution is implemented. By understanding these metrics and how they relate to customer desires, organizations can gain valuable insights into a solution's potential for success.

Imagine you are developing a new medical device. Predictive analytics could help you determine which features will be most valued by surgeons and patients, even before a single prototype is built. By analyzing these metrics, you can adjust your design, ensuring the final product meets the needs of your target market and delivers maximum value.

The ultimate goal of predictive analytics is to formulate "breakthrough solutions" – solutions that not only meet but exceed customer expectations, giving organizations a competitive edge in the marketplace.

Followers of JTBD and ODI already know these things:

1. Traditional strategy formulation is inherently flawed: "If you are like most individuals, the immediate focus when involved in formulating a strategy or plan of any kind is the brainstorming of potential solutions." This solution-centric approach often leads to incremental improvements rather than breakthrough innovations.

2. Customers define value through desired outcomes, not solutions: "Customers do know, with certainty, what desired outcomes they want to achieve. They may not know they want a microwave oven... They do know, however, that they want to minimize the time it takes to prepare a meal." Understanding these desired outcomes allows for the creation of solutions that truly resonate with customers.

3. Desired outcomes are stable, solutions are not: "Desired outcomes, on the other hand, are stable over time. This stability is a fundamental characteristic of a desired outcome... What does change over time is the level of importance that individuals place on each outcome and the degree to which the outcome is perceived to be satisfied." This stability makes desired outcomes a reliable foundation for long-term strategy.

Predictive metrics bring something more to the equation:

1. Predictive metrics provide a quantifiable link between outcomes and solutions: "A predictive metric is a parameter that when measured and controlled will predict the satisfaction of a desired outcome... It states the parameter that must be measured and controlled, prior to the implementation of a strategy or solution, to predict if a desired outcome will be satisfied." This allows for the objective evaluation of potential solutions.

2. The Universal Strategy Formulation Model (USFM) provides a structured framework for strategy formulation: The USFM, likened to a mathematical equation, utilizes desired outcomes, constraints, and the desired competitive position as constants. Solutions are treated as variables, tested against the constants to find the optimal fit.

3. Case studies: Apparently, Southcorp, Cordis Corporation, and Pratt & Whitney cases illustrate the successful application of this in various contexts, leading to significant improvements in market share, product development, and operational efficiency. I wonder why I never read about this in the Cordis study?

4. A customer-centric approach to strategy: "By capturing and prioritizing desired outcomes, we gain a much better understanding of the customers and what they really value." This customer-focused approach ensures that strategies and solutions are aligned with the needs and desires of the target market.

The Original Process for Developing Predictive Metrics

The process for developing predictive metrics starts with identifying the desired outcomes of the customer. This is a crucial first step, as predictive metrics are designed to forecast the degree to which a solution will satisfy these desires. This is accomplished through qualitative research such as interviews and focus groups, which allow for in-depth exploration of customer needs and expectations.

Desired outcomes should be expressed in clear, measurable terms that are free of any specific solutions or technologies. For example, a desired outcome for a surgical system might be "to minimize tissue damage" rather than "to use a smaller incision." This distinction is important because predictive metrics need to be stable over time and adaptable to evolving technologies.

The next stage involves translating these desired outcomes into "predictive metrics." Each metric should directly correlate with the satisfaction of its corresponding desired outcome and be measurable before the solution is implemented.

For instance, the desired outcome of "minimizing tissue damage" during surgery could be linked to a predictive metric such as "reducing the amount of force applied to tissue." This metric can be measured during the design and testing phases of the surgical system, providing early indications of its potential to fulfill the desired outcome.

Defining predictive metrics requires expertise and an understanding of the factors that influence the achievement of desired outcomes. This often involves collaboration between individuals familiar with the subject matter and those experienced in executing predictive analytics.

Prioritizing predictive metrics is another essential step in the process. This involves analyzing the relationships between each metric and all desired outcomes to identify those with the most synergy. Metrics that predict the satisfaction of multiple, important desired outcomes hold higher priority. This prioritization helps organizations focus on the metrics that have the greatest impact on overall customer satisfaction.

This 👆 is where the fun begins…

Results and Reasoning

When I first got a grasp on this (and I still may not grasp it completely) I immediately decided that this would have been a lot of work. So, I turned it over to AI to do the following:

1. Develop the 1:1 predictive metrics for each desired outcome

2. To identify the relationships between a predictive metric and other desired outcomes where there is a relationship; albeit weaker than a direct 1:1 relationship.

I wrote a prompt. Here are the inputs:

Job: Submitting a Parcel into the induction process

End User: Induction operation

Step: Prepare parcel for induction

Desired Outcomes:

• Orient the parcel correctly for induction, e.g., aligning barcodes, positioning address labels, ensuring proper facing, etc.

• Minimize the time taken to prepare each parcel, e.g., quick handling techniques, efficient movements, streamlined processes, etc.

• Ensure the parcel's stability for induction, e.g., secure loose items, reinforce weak packaging, balance uneven shapes, etc.

• Verify the parcel's suitability for the chosen induction method, e.g., size compatibility, weight appropriateness, shape considerations, etc.

• Remove any obstructions that could hinder induction, e.g., protruding labels, loose straps, attached documents, etc.

• Ensure proper spacing between parcels for induction, e.g., maintaining gaps, preventing clustering, avoiding overlaps, etc.

• Identify and address any last-minute issues, e.g., leaking contents, compromised packaging, missing information, etc.

• Minimize physical strain during parcel preparation, e.g., using ergonomic movements, employing assistance devices, maintaining proper posture, etc.

• Maximize the readability of parcel information for automated systems, e.g., flattening crumpled labels, clearing obstructed barcodes, enhancing visibility of critical data, etc.

• Ensure compliance with any special handling instructions, e.g., "This Side Up" orientations, fragile item precautions, temperature-sensitive positioning, etc.

Here is what the output looks like 👇. This was from a single prompt. Apologies for Substack’s inability to output tables. I’ve screen-shotted them for you:

<cot_switch>on</cot_switch>

<analysis> Let's break down the problem step-by-step.

Step 1: Side-by-Side List - Analyzing Desired Outcomes and Identifying Predictive Metrics

First, we need to analyze each desired outcome and determine a suitable primary predictive metric. We need to think about what measurable aspect would indicate the successful achievement of each outcome.

• Orient the parcel correctly for induction: We need to measure how accurate the orientation is. We can measure this by the percentage of parcels correctly oriented according to specifications.

• Minimize the time taken to prepare each parcel: The most direct metric here is the average time taken to prepare each parcel.

• Ensure the parcel's stability for induction: We can measure stability by the number of parcels that shift or fall during a simulated induction process. Let's use the percentage of parcels remaining stable during simulation.

• Verify the parcel's suitability for the chosen induction method: The key is whether a parcel is rejected by the induction system due to incompatibility. The metric can be the percentage of parcels accepted by the induction system.

• Remove any obstructions that could hinder induction: The presence of obstructions would likely lead to induction failures or jams. We can measure the percentage of parcels with no obstructions.

• Ensure proper spacing between parcels for induction: We can measure the average distance between parcels or the percentage of parcels that meet the minimum spacing requirements.

• Identify and address any last-minute issues: This is a bit broad, but essentially, we want to reduce the number of issues that could cause problems later. We can measure the percentage of parcels with no identified issues.

• Minimize physical strain during parcel preparation: We could measure this in a few ways, but let's use a subjective scale combined with observation of proper ergonomic practices. Let's call it an ergonomic compliance score (e.g., 1-5, with 5 being perfect compliance).

• Maximize the readability of parcel information for automated systems: The metric here can be the successful read rate by automated scanning systems.

• Ensure compliance with any special handling instructions: The metric can be the percentage of parcels where special instructions are correctly followed. </analysis>

Step 2 & 3: Matrix Creation & Determine Relationship Strengths - Mapping Metrics to Outcomes and Assessing Relationships

<analysis> Now, we need to create a matrix and map all these metrics to each desired outcome. Then, for each cell in the matrix, we'll assess the strength of the relationship (Strong, Medium, Weak, or None). We need to think carefully about how each metric relates to each outcome, not just the primary one.

For example:

• % Parcels Correctly Oriented will have a strong relationship with "Orient the parcel correctly" but might have a weak or medium relationship with "Ensure the parcel's stability" (as correct orientation can influence stability) and a medium relationship with "maximize readability of parcel information."

• Average Time per Parcel will have a strong relationship with "Minimize the time taken" but might have a weak relationship with "Minimize physical strain" (as rushing could lead to poor ergonomics).

• % Parcels Remaining Stable (Simulated) will have a strong relationship with "Ensure the parcel's stability," a medium relationship with "Orient the parcel correctly," and potentially a weak relationship with others.

• % Parcels Accepted by Induction System strongly relates to suitability verification but also indirectly relates to orientation, obstruction removal, and spacing.

Let's go through each cell and make a judgment based on this logic. </analysis>

Step 4: Generate Legend

<analysis> This step is straightforward. We just need to define the symbols we will use for the matrix. </analysis>

Step 5: Explanation Table - Justifying Relationship Strengths

<analysis> Finally, we need to provide a rationale for each relationship strength we assigned in the matrix. This is important for transparency and to ensure the logic is sound. For each cell in the matrix, we will write a brief explanation of why we assigned that particular relationship strength. </analysis>

Here are the outputs based on the analysis above:

Side-by-Side List:

The table above goes on and on. Every single cell in the matrix is explained.

Pretty cool, huh?

Calculating the Prioritization

I’m not going to get into this here, but this is an overview of the step-by-step breakdown of the normalized importance calculation used to prioritize predictive metrics in the process:

Step 1: Gather the Necessary Data

• Importance Ratings: Obtain the importance rating for each desired outcome. This data is collected through quantitative research, typically surveys, where customers rate the importance of each desired outcome.

• Predictive Relationships: Determine the relationship between each predictive metric and each desired outcome using matrix analysis.

  • Assign a numerical value (1-9) to each relationship based on its strength, with 1 representing a weak relationship and 9 representing a strong relationship.

  • If a metric does not predict the satisfaction of a particular desired outcome, assign a value of 0.

Step 2: Calculate the Sum of Each Row in the Matrix

• For each desired outcome, add the predictive values across all the predictive metrics.

• This represents the total predictive strength associated with each desired outcome.

Step 3: Calculate the Weighted Importance for Each Desired Outcome

• Divide the importance rating of each desired outcome by the sum of the corresponding row calculated in Step 2.

• This value represents the relative contribution of each predictive metric to the overall importance of the desired outcome.

Step 4: Calculate the Normalized Importance Value for Each Metric

• For each predictive metric, multiply the weighted importance value (from Step 3) of each desired outcome by the corresponding predictive value assigned in the matrix.

• Sum these products across all desired outcomes that have a relationship with the predictive metric.

• This sum represents the normalized importance value for the metric.

Step 5: Adjust for Customer Weighting

• Multiply the normalized importance value of each metric by the weighting assigned to the corresponding customer type.

• This weighting reflects the relative importance of different customer types in the overall strategy.

Step 6: Adjust for the Number of Desired Outcomes

• Divide the adjusted normalized importance value (from Step 5) by the total number of desired outcomes for the corresponding customer type.

• This ensures that metrics from customer types with more desired outcomes are not unfairly weighted.

Step 7: Calculate the Final Percentage

• Calculate the percentage representation of each metric’s adjusted normalized importance value (from Step 6) relative to the sum of all adjusted normalized importance values.

• This results in a final normalized importance value for each predictive metric, representing the percentage of total value contributed by each metric.

Step 8: Prioritize Metrics

• Rank the predictive metrics in descending order based on their final normalized importance values.

• The metric with the highest percentage value is the most important and should be given the highest priority.

This process systematically prioritizes predictive metrics based on their impact on the satisfaction of customer desires, considering both the importance of each desired outcome and the synergy between the metrics and the outcomes.

Conclusion

This is one of those laborious things that no one wants to do because … it’s laborious. But it could be extremely important when formulating strategies and making product decisions after your Jobs-to-be-Done research.

I would point out that I haven’t deeply reviewed these metrics or their correlations. I’m sure a human could do it much better. Having said that, I would rather start with a 30 second strawman that spend days or weeks stressing over this work. I was reading about it, I can only imagine the pain suffered while doing the work 25 years ago.