So you've finished a project and you're happy with the result. How do you really know if it was a "success" or a failure? This is something you will need to define and determine early in the process or product cycle. In this chapter, we'll examine measures for success and how teams can build in time to learn from failure.
On any project, it is important to start out knowing why you're designing or building something. This will inform your goals. Defining goals can help you set priorities so you know where to focus your energies.
Measure success through KPIs
Defining the problem and goals help ensure that you are building the right product to solve actual problems faced by users. You need to consider how you will measure the success of a project. These are often referred to as KPIs, or key performance indicators.
Focus on key information through KPIs with Bernard Marr, founder and CEO of Intelligent Business Performance. [1:52 min]
According to UX and psychology expert Joe Leech (aka Mr. Joe), for metrics to be useful and reflect an ROI (return on investment), they should include:
a timescale – how long something takes
a benchmark – a point of comparison
a reason to be reported – value added
an associated action – behavior that results from metric
The slides above are from Joe Leech's talk on UX & ROI: What to measure and what to expect.
UX expert Jared Spool considers the elements that make up UX KPIs as:
behavior based – behavior as it relates to the product
key to business – relevant to business goals, which includes at least one of the following:
increase revenues
decrease costs
increase market share
increase revenue from existing customers
increase shareholder value
include a performance indicator – such as time to complete a task
unique to how the business or industry runs – each business will need different metrics
are easy to measure
have a diagnostic value to look for trends – uncover patterns and issues based on the recorded data
Understand what success means to stakeholders
Talking with key decision makers throughout the process is a good way to ensure you are working towards a common, well-defined goal. Stakeholder interviews with key members of the client team, customers, bosses, or other players—both inside or outside the organization—are one way to understand concerns and gauge measures of success. Through these conversations, themes will emerge that will help you prioritize features and define KPIs. Working as a team, asking questions, and rethinking how things were done in the past, will also add value to these conversations.
Project goals can vary from reaching a new target audience, to increasing the number of return visitors, to growing the number of paid users. With so many social platforms, there are a large number of metrics – including number of visitors, accounts created, conversion (sales), or time spent on site – that can be considered.
As technology develops, it's also important to question how metrics should be used and which are the most effective measures. Consider a scenario where the number of page views to your website goes up. How do you know that it's because people loved your content and not because people stumbled upon you by accident? In addition to testing products on users, learning to question metrics is part of your job as a UX designer. You may want to consider multiple ways to measure success.
If part of your experience involves an in-person or offline component, or if key metrics you need are not available in the public domain, how can you effectively track data? It's important to work closely with your team to consider many different possibilities for quantifying success. Be sure to involve the developers early in the process so that they can help build ways to record and measure data into the product.
The ability to measure success is also important if you intend to apply for funding or a grant. Chances are, the people financing the project will want proof that their investment will be worthwhile. Everyone loves hearing a success story! Being able to tell it with data is always helpful.
Set OKRs
Objectives and Key Results, otherwise known as OKRs, are another way organizations set and work towards goals. The objective tends to be ambitious, uncomfortable, and quantitative. They are written in such a way as to encourage teams to stretch, resulting in action. They are also shared publicly in order to establish a sense of transparency. Key results are measurable through quantitative metrics.
OKRs should be:
Action oriented
Written with verbs in addition to signalling achievement
Have measurable outcomes
Teams like Google and LinkedIn use OKRs to measure success (Google shares their process on their Re:Work blog). It is not assumed that all objectives will be met. If that were the case, then the goals would not have been set high enough. Instead teams need to aim for "stretch goals" in order to grow. At the same time, unachievable goals can set a team up for failure, so it's about finding a balance between pushing too hard and not pushing hard enough.
Learn from failure
Failure sounds bad, but great ideas can come from it. Products, as we see and love them today, weren't "born" that way. They were refined through iteration, and no doubt mistakes were made during the process! It's better to fail early in the UX process because it's far less expensive to change a product early on, rather than having to start again at zero when you've already invested a lot of time and resources.
At OpenClassrooms we embrace failure as a learning opportunity. Failing is part of our company culture! [3:52min]
In high pressure work environments, it's easy to feel like one is always two steps behind. It's easy to rush through things, always focusing on what is next. However, it's important to stop, take a deep breath, and reflect from time to time. It's incredibly valuable to learn from your experiences in order to avoid mistakes in the future and make improvements. Just as a product is refined through iteration, experience can also be refined, whether it's improving methodology or figuring out how tools can be updated or adapted in the future.
Build in time for reflection
Design teams should build in time for retrospectives or postmortems so that they can learn from each experience. Every project is going to be different and will host new lessons to be learned. Team members should feel they have permission to be honest, as it's the only way to truly be able to grow, learn, and improve from the future.
Retrospectives are ongoing review and reflection sessions that happen every few weeks. They are built into the Agile project management process in order to celebrate what is working, to pinpoint what's not working, and to make a plan for fixing any problems. The goal is to have actionable takeaways at the end of each retrospective, as teams work towards constant improvement through a culture of learning and sharing.
Postmortems are done at the end of a project (when it's pronounced "dead"). They need to be done as a team where everyone is focused on the goal of learning from the experience. Teams should:
Reflect on what worked well (or better than expected).
Reflect on what didn't go as planned (or was more challenging than expected).
Reflect on why certain things may have happened in a certain way.
Consider tone and voice when delivering feedback.
Make sure not to blame or point fingers.
Postmortems and retrospectives are not complaining sessions. Rather, the time should be used to focus on growth through actionable takeaways. In addition to writing a recap or report for the team, consider sharing it with the wider office in order for everyone to learn. Small, incremental change adds up to make a big difference.
Let's Recap!
Define how success will be measured at the beginning of a project. KPIs are Key Performance Indicators and OKRs are objectives and key results.
Stakeholder interviews are an excellent way to gain a better understanding of project goals and expectations through talking with key players and decision makers involved in the project.
See failure as something you can learn from.
Postmortems and retrospectives are valuable ways to reflect on a project or process.
Prioritization is easier to do when you have a clear idea of what you're working towards.
