Using OKRs to accelerate Relentless Improvement

Introduction

In my last 2 articles, I first introduced the concept of Objectives and Key Results (OKRs) with an illustration based on personal goal-setting then explored applying them to improvement of Product Strategy.

After experimenting on myself with personal OKRs, my first applications of the concept in a coaching setting have been in the area of improvement objectives.  I was particularly excited about this idea as I encounter so many teams and ARTs who lose heart with their retrospectives and Inspect and Adapt sessions due to failure to follow through on their great ideas.

I’ve lost count of how many discussions I’ve had in either the training room or a coaching setting with people who drag themselves to retrospectives feeling they are a waste of time.   The common refrain is that “we just keep talking about the same issues and nothing ever changes”.  A little probing generally exposes the following:

• Teams use the same format every time (my pet hate is the “What went well, what didn’t go well, what puzzles me” retro).
• Teams spend 80-90% of our session examining the past
• The (rushed) improvement idea generation yields concepts beyond their power to change or very fuzzy
• Few teams retrospect on the effectiveness or application of the ideas they do generate 

I’ve gifted many a copy of Esther Derby and Diana Larsen’s classic “Agile Retrospectives” to ScrumMasters and RTE’s.  Likewise, I’ve facilitated countless sessions on time-boxing your retrospective or I&A to leave at least 50% of it for the “improvement” part of the discussion.

Good facilitation will at least get you to the point of emerging from the session with some good ideas.  However, in the words of Michael Josephson “Ideas without action are like beautifully gift-wrapped boxes with nothing inside”.
 

Crystallizing your great ideas with OKRs

A couple of years ago, I started experimenting with the Improvement Kata to assist teams with actioning their ideas.  As I researched it, I came across a great article describing an “Improvement Theme Canvas” by Crisp’s Jimmy Janlen.  The canvas was beautifully simple, taking teams on a great journey with 4 questions:

• Where are we now?
• Where do we dream of being (What does awesome look like?)
• Where can we get to next? (And how would we measure it?)
• What steps would start us moving?

I used it again and again, in settings ranging from team retrospectives to executive workshops.  And it helped.     But it was still missing something.  There was a recurring trend – a lack of measures.
As I started experimenting with OKRs, there was an obvious synergy.   I had found the Key Result setting to be the moment where I did my deepest thinking myself, so I tweaked the canvas a little:

My first opportunity to apply it was the second day of a management workshop.  The first day had been spent exploring challenge areas and selecting focus areas for improvement.  We had a fairly clear view of current state and definition of awesome (which represented 18 months+ of hard work), and the job on Day 2 was to address the right side of the canvas with a 3 month view. 

We began with some objectives, then the hard work began – how would we measure them?  The result was incredibly powerful.  We moved from some very fuzzy, amorphous objectives into real clarity on how to generate momentum.  When you start to talk about how to measure movement, something changes in the conversation.  On the one hand, it grounds you in reality but on the other the fact that you are looking for a measure ambitious enough to only have a 50% chance of success stretches you out again. 

As an illustration, I’ve developed a sample based on a commonly occurring theme in Inspect and Adapt sessions – struggles to develop momentum with Test Automation.

As a final hint on the canvas generation, the order in which you complete it is crucial.  There’s a great temptation to rush to “Enabling Activities” because people love to begin with solutions.  

• Start with “Current state”: fishbone diagrams, 5 whys, and causal loop diagrams are all great tools for generating the data and shared understanding
• Move to “Definition of Awesome”.  The word “Awesome” is incredibly powerful in helping move minds into the art of the possible, and also encourages alignment around desired direction.
• Step 3 is to agree on your timebox (often 3 months or one Program Increment) and figure out how far you can move from your current state to the definition of awesome as you set your “Target State OKRs”.  You will start flushing assumptions, and more importantly discovering alignment challenges.  The movement to quantified measures with your Key Results will expose many an unvoiced insight.
• Only now do you set the “Enabling activities”.  They will come very quickly.  If you’ve set your OKRs well, your key activities will be very obvious.

So far we only have an Idea, what about Action? 

I can recommend nothing so strongly as the weekly priority-setting cycle from Radical Focus I described in my first OKR article.  The reality is, despite many schemes people use for creating space for follow-up on improvement ideas it is amazing how often they stumble.  Life gets busy, delivery pressure is high, and the first thing that gets sacrificed is time to work on improvement ideas. 

The beauty of setting yourself a few priorities each week on moving towards your OKRs and spending a little time reflecting on them at the end of the week is that it helps you “find time”.  A few months into applying personal OKRs I can attest to this.  I live a very hectic life.  Most weeks involve 5 long days facilitating and coaching with clients, 2-3 flights, and lots of “after hours admin” whilst coming up for air to be a dad, husband and human 😊  Its very easy to find excuses.  But every week starts with “how can I move forward this week?”  And in the back of my head every day are my weekly goals.  To be brutally honest, my learning cycle at the end of the week often focuses on “how to find time”.   I’m not waiting 3 months to look back and find out I made no progress and lose heart.   

My final hint is this: Make them public.  If you’re applying the technique to Inspect and Adapt,  your improvement OKRs should appear in your PI Objectives and progress discussed  in your System Demos and PI Demo.  Include the Key Result evaluation in the metric portion of your Inspect and Adapt.  There is both power and motivation in transparency about improvement objectives.

Improving SAFe Product Management Strategy with OKRs

Introduction

In my last article, I introduced the concept of Objectives and Key Results (OKRs) and provided an illustration based on personal application.  However, I only tried out personal OKRs because I wanted a way to “learn by doing” before I started trying out some of my thinking on incorporating them in SAFe.   This article will explore the first application in SAFe: Improving Product Strategy.

Late last year, Product Management guru John Cutler wrote a brilliant article called “12 signs you’re working in a Feature Factory”.  Some highlights include:

• No connection to core metrics.  Infrequent discussions about desired customer and business outcomes.
• “Success Theater” around shipping with little discussion about impact.
• Culture of hand-offs.  Front-loaded process in place to “get ahead of the work” so that items are ready for engineering”.  Team is not directly involved in research, problem exploration, or experimentation and validation.
• Primary measure of success is delivered features, not delivered outcomes.
• No measurement.  Teams do not measure the impact of their work.
• Mismatch between prioritization rigour (deciding what gets worked on) and validation rigour (deciding if it was, in fact, the right thing to work on).

His blog is a gold-mine for Product Managers, and I found much of his thinking resonated with me when it came to the difference between a healthy ART and a sick one.  OKRs are a very handy tool in combating the syndrome.

Introducing OKRs to Feature Definitions

The official guidance says “every feature must have defined benefits”.  We teach it in every class, and our Cost of Delay estimates should be based on the defined benefits but it is frightening how few ARTs actually define benefits or expected outcomes for features.  At best, most scratch out a few rarely referenced qualitative statements.

Imagine if every feature that went into a Cost of Delay (COD) estimation session or PI planning had a clearly defined set of objectives and key results!

Following is an illustration based on an imaginary feature for a restaurant wanting to introduce an automated SMS-based reservation reminder capability.

Applying OKRs to PI Objectives

I’ve lost count of the number of long discussions I’ve had about the difference between PI Objectives and Features.  They almost always start the same way.  “Surely our PI Objective is to deliver Feature X”.  Or, in an ART with a lot of component teams it might be “Support Team A in delivering Feature X”.

WRONG! 

 

Good PI objectives are based on outcomes.  Over the course of the planning event, the team unpacks their features and generates stories to capture the ongoing conversation and identify key inter-dependencies.  Eventually, we consolidate what we’ve learned about the desired outcomes by expressing them as PI objectives.  The teams don’t commit to delivering the stories, they commit to achieving the objectives.  They should be working with their product owners all the way through the PI to refine and reshape their backlogs to maximize their outcomes.  

If every feature that enters PI planning has a well-defined set of OKRs, we have a great big hint as to what our objectives are.  An ART with good Devops maturity may well transpose the Feature OKRs straight into their PI objectives (subject to trade-off decisions made during planning).  They will ship multiple times during the PI, leveraging the feedback from each deployment to guide their ongoing backlog refinement.   

SAFe Principle 9 suggests enabling decentralization by defining the economic logic behind the decision rather than the decision itself.   Using well-defined OKRs for our PI objectives provides this economic logic, enabling Product Owners and teams to make far more effective trade-off decisions as they execute.

However, without good DevOps maturity the features may not have been deployed in market for sufficient time by the end of the PI to be measuring the predefined Key Results.  In this case, the team needs to work to define a set of Key Results they will be able to measure by the time the PI is over.  These should at minimum be demonstrating some level of incremental validation of the target KRs.  Perhaps they could be based on results from User Testing (the UX kind, not UAT).  Or maybe, measured results in integration test environments.  For example:

• 50% of test users indicated they were highly likely to recommend the feature to a friend or colleague
• End-to-end processing time reduced by 50% in test environments. 

Last but not least, we’ve started to solve some of the challenges ARTs experience with the “Business Value” measure on PI objectives.  It should be less subjective during the assignment stage at PI Planning and completely quantitative once we reach Inspect and Adapt!

Outcome Validation

By this point, we’ve started to address a number of the warning signs.   But, unless we’re some distance down the path to good DevOps we haven’t really done much about validation.   If you read my recent Feature Kanban article, you’ll have noticed that the final stage before we call a feature “done” was “Impact Validation”. 

This is the moment for final leverage of our Feature OKRs.  What happened once the feature is deployed and used in anger?  Do the observed outcomes match expectations?  Do they trigger the generation of an enhancement feature to feed back through our Program Backlog for the next Build/Measure/Learn cycle?  Do they affect the priorities and value propositions of other features currently in the Program Backlog?

Linking it back to the ART PI Metrics dashboard

In the business impact quadrant of my PI metrics model, I proposed that every ART should have a “fitness function” defining the fashion in which the ART could measure its impact on the business.  This function is designed to be enduring rather than varying feature by feature – the intended use is to support trend-based analysis.  The job of effective product managers is, of course, to identify features which will move the strategic needles in the desired direction.  Business Features should be contributing to Business Impacts, Enablers might be either generating learning or contributing to the other three quadrants.

Thus, what we should see is Key Result measures in every feature that exert force on the strategic success measures for the ART.   Our team level metric models should be varying with feature content to enable them to steer.  This helps with the first warning sign Cutler identifies:

1. No measurement.  Teams do not measure the impact of their work.  Or, if measurement happens, it is done in isolation by the product management team and selectively shared.  You have no idea if your work worked.

Conclusion

Whilst OKRs obviously have a lot to offer in the sphere of Product Strategy, this is not the only area in which they might help us with SAFe.  In my next article, I’ll explore applying them to Relentless Improvement.

An Introduction to Objectives and Key Results (OKRs)

Introduction

In late January I was working on a final draft of one of my metrics articles when someone suggested I take a look at OKRs.   I’d heard of them, but hadn’t really paid much attention other than to file it on my always long backlog of “things I need to learn something about”.  Then a couple of days later I was writing my abstract for a metrics session at the Agile Alliance 2017 conference and in scanning the other submissions in the stream discovered a plethora of OKR talks.  This piqued my interest, so I started reading the OKR submissions.   At this point I panicked a little.  The concept sounded fascinating (and kind of compelling when you realize its how Google and Intel work).  I was facing into the idea that my lovingly crafted “SAFe PI Metrics revamp” might be invalidated before I even finished the series.

So, I went digging.  The references that seemed to dominate were Radical Focus by Christina Wodtke, a presentation by Rick Clau of Google, and an OKR guide by Google.  A few days of reading later, I relaxed a little.   I hadn’t read anything that had invalidated my metric model, but I could see a world of synergies.  A number of obvious and compelling opportunities to apply OKR thinking both personally and to SAFe had emerged.

In this article, I will provide a brief overview of the concept and use my early application to personal objective setting as a worked example.  My next article will detail a number of potentially powerful applications in the context of SAFe.

OKRs – an Overview

As Wodtke states in Radical Focus, “this is a system that originated at Intel and is used by folks such as Google, Zynga, LinkedIn and General Assembly to promote rapid and sustained growth.  O stands for Objective, KR for key results.  Objective is what you want to do (Launch a killer game!), Key Results are how you know if you’ve achieved them (Downloads of 25K/day, Revenue of 50K/day). OKRs are set annually and/or quarterly and unite the company behind a vision”

Google provides some further great clarifying guidance:

• “Objectives are ambitious and may feel somewhat uncomfortable”
• “Key results are measurable and should be easy to grade with a number (Google uses a scale of 0-1.0”
• “The sweet spot for an OKR grade is 60-70%; if someone consistently fully attains their objectives, their OKRs are not ambitious enough and they need to think bigger”
• “Pick just three to five objectives – more can lead to overextended teams and a diffusion of effort”.
• “Determine around three Key Results for each objective”
• “Key results express measurable milestones which, if achieved, will directly advance the objective”.
• “Key results should describe outcomes, not activities”

Wodtke dials it up a notch when it comes to ambition:

• “OKRs are always stretch goals.  A great way to do this is to set a confidence level of five of ten on the OKR.  By confidence level of five out of ten, I mean ‘I have confidence I only have a 50/50 shot of making this goal”.

Personal OKRs – a worked example

I’m a great believer in experimenting on myself before I experiment on my customers, so after finishing Radical Focus I decided to try quarterly personal OKRs.  Over the years, I’ve often been asked when I’m going to write a book.  After my colleague +Em Campbell-Pretty  published Tribal Unity last year, the questions started arriving far more frequently.

After a particularly productive spurt of writing over my Christmas Holidays I was starting to think it might be feasible so I began to toy with the objective of publishing a book by the end of the year.  Em had used a book writing retreat as a focusing tool, so it seemed like planning to hole up in a holiday house for a few weeks late in the year would be a necessary ingredient but that was far away (and a large commitment) so the whole thing still felt very daunting.

As I focused in on the first quarter, the idea for an objective to commit to scheduling the writing retreat emerged.  Then the tough part came – what were the quantitative measures that would work as Key Results?  Having now facilitated a number of OKR setting workshops, I'm coming to learn that Objectives are easy, but Key Results is where the brain really has to engage.  My first endeavors to apply it personally were no exception.  Eventually, I came to the realization that I needed to measure confidence.  Confidence that I could generate enough material, and confidence that people would be interested in reading the book.  After a long wrestle, I reached a definition:

Working the Objectives

As Wodtke states when concluding Radical Focus, “The Google implementation of OKRs is quite different than the one I recommend here”.  Given that I had started with her work, I was quite curious to spot the differences as I read through Google’s guidance.  At first, it seemed very subtle.  The biggest difference I could see was that for Google not all objectives were stretch.  Then the gap hit me.  Wodtke added a whole paradigm for working your way towards the objective with a lightweight weekly cycle.   I had loved it, as I’d mentally translated it to “weekly iterations”.

The model was simple.  At the start of each week, set three “Priority 1” goals and two “Priority 2” goals that would move you towards your objectives.  At the end of the week, celebrate your successes and reflect on your insights.  Whilst she had a lot more to offer on application and using the weekly goals as a lightweight communication tool, the simplicity was very appealing personally.  After all, I had a “day job” training and coaching.  My personal objectives were always going to be extra-curricular so I wanted a lightweight focusing tool.

Whilst writing a book was not my only objective, it was one of two so features heavily in my weekly priorities.  Following is an example excerpt:

Clarifying notes:

• Oakleigh is a suburb near me with a big Greek population.  They have a bunch of late night coffee shops, and its my favorite place to write.  I head up at 10pm after the kids are in bed and write until they kick me out around 2am.  This article is being written from my newly found “Brisbane Oakleigh”.
• In week 2 I utilised OKRs in an executive strategy workshop I was facilitating (with a fantastic result), and shared the concept with a friend who was struggling with goal clarity.  In both cases, I used my writing OKR as an example 

Conclusion

Hopefully by this point you are both starting to understand the notion of the OKR and perhaps inspired enough to read some of the source material that covers it more fully.  I can’t recommend Radical Focus highly enough.  It’s an easy read, fable style (think The Goal, The Phoenix Project). 

You might have noticed along the way that my priority goal this week was to “Publish OKR article”.  As seems to happen regularly to me, once I start exploring an idea it becomes too big for one post.  I never actually got to the “SAFe” part of OKRs.  To whet your appetite, here are some things I plan to address in the second article:

• Specifying OKRs as part of a Feature Definition
• Applying OKR thinking to PI Objectives
• Applying OKR thinking to Inspect and Adapt and Retrospective Outcomes