Discovery Audits: Why They Matter & How to Do ‘Em

When we’re staring down the barrel of a redesign project for a web app or mobile application, too often we forget that we have a huge pool of data and insight to draw from already – the existing product.

We’re talking about an audit: the evaluation of a product to determine its effectiveness and/or implementation quality.

Auditing an existing application allows us to see what is already working well in addition to what is clearly not working well. It allows us to keep and improve upon the former, while helping us avoid the latter and prevent us from making the same mistakes. We can use the learnings, data, and resulting recommendations from the audit going forward in the project production process.

We call this a Discovery Audit.

It’s a bit of a no-brainer, right? Before we get into what to audit and how to do it, let’s talk about some of the different types.

Discovery Audits vs. Other Types of Audits

Different audits serve different purposes, but they are driven by one of two high-level scenarios:

1. You’ve already made the decision to build a new digital product.
2. You’re unsure of the state of affairs around your existing software, and whether or not a rebuild needs to happen.

In scenario 1, the purpose of the audit is to uncover information to inform the successful design of the next version of the product. It is to look for opportunities and learnings, rather than technical things to fix on an existing property. This calls for a Discovery Audit.

In scenario 2, the purpose is to validate the implementation quality of an existing tool in order to recommend improvements, updates, or a total overhaul. The outcome or purpose of the audit can be tailored to a specific vertical, whether accessibility-related, user experience-related, content-related, etc., or it could be a broad review of the product’s functionality, looking for implementable improvements.

Because you’re here reading about the entire Discovery phase for an imminent digital project, we’re assuming you’re in the former group – you’re looking for a Discovery Audit, and not a Technical Audit, Content Audit, Security Audit, etc. If you are looking for a Technical Audit or otherwise, drop us a line and we’ll help you out or point you in the right direction.

For the Discovery Audit inquirer, let’s get down to the nuts & bolts of how to tackle a Discovery Audit.

When to Do a Discovery Audit

We normally tackle Discovery Audits after user story generation in the overall digital project discovery process. That’s because we generate user stories in our kickoffs, which is the first post-contract touchpoint with the client. We also unpack user persona information in the kickoff, which has to happen pre-audit. Thus, it’s more efficient to slot in the audit at that point, and then make sure we’re adding in any new user stories discovered in the audit.

That said, a Discovery Audit can also be tackled pre-user-stories, post-personas if absolutely required. There is a slight risk you might miss some items in the audit because you haven’t gone through user story generation and prioritization yet, but it’s a relatively small risk.

This is an Entirely New Project – So I Can Skip an Audit, Right?

Well, yes and no. You won’t be able to dig through data around goals, or dive into analytics, or tackle those types of items that are directly related to digital products. However, odds are – especially if we’re talking about web applications or mobile applications – processes already exist. They just don’t exist in a dedicated digital form yet.

Say you’re building a scheduling application for a company that delivers workshops. They don’t have an application for this right now – but guess what? They still have a process for scheduling workshops. It’s either an analog or offline one, a disconnected digital one using a series of unrelated pieces of software (Google Docs, Outlook, etc.), or a hybrid. The important part is that a process does exist, so you should audit that process.

Sit with the customer and observe them. Take notes. Ask them: How do you accomplish [task] right now? Why? What’s working well? What isn’t? How should a digital version of this process work, ideally?

This falls under the category of user research. Or a process audit. We’re not big into arguing semantics – what matters is seeing how real users fulfill an objective currently and why, to inform a better user experience down the road. So even if you don’t have an existing digital product that’s being rebuilt, you likely still have some auditing work to do.

Note: Keep in mind there are entire fields and textbooks devoted to user research. If you really want a deep dive, Observing the User Experience: A Practitioner’s Guide to User Research by Elizabeth Goodman is an outstanding book on the subject. But just like anything in life, user research, conversations, process audits and the like are not all-or-nothing. Even a little bit is better than none.

Alright, enough with the asides. Let’s get to the practical stuff.

Download the Free Discovery Audit Workbook

Further down, we’ll explain exactly how to run the audit. But the first step is having the right tool in place. We’ve made the Discovery Audit Workbook we use internally at Paper Leaf free to download for you, below. Grab it and then keep reading.

If you’ve been reading this and are realizing you really shouldn’t be spending your time running a Discovery Audit, or you simply don’t have the time internally, then shoot us an email and we’ll chat about handling that for you.

Categories and Items to Audit

When we’re running a Discovery Audit on an existing product to help inform its future redesign, there are a consistent set of categories or criteria we can look to audit – and within these categories, general items and specific-to-the-project items (which need to be generated) to review. Holistically, these audit categories and the items within them comprise a great starting point for your existing digital product audit.

Generally, here’s what these categories are and what they mean:

• Goals & KPIs. What goals / KPIs were set for the digital product, and how well is the digital product meeting those goals?
• Analytics. What can we infer from analytics with regards to audience, technology, location, content, channels, and traffic sources?
• Usability & Design. How intuitive is the product to use? Which user stories are working well, and which ones aren’t?
• Platform. Does the platform allow administrators to effectively do what they need to do?
• Content. Does the content clearly communicate what you do to your users? What content is missing that would be valuable to your users? How effective is the copy in helping users complete key stories or tasks?

As mentioned, there are some staple items to review in each of these categories, but it’s important to note that many of the items that should be audited will be unique to the project itself. For example, not every mobile app has the same KPIs. Not every application has the same goals. And not every product is serving the same users, so they won’t have the same user stories.

The simplest method to ensure a comprehensive Discovery Audit is to start with the examples we’ve given you in the Discovery Audit Workbook, then generate new audit items that are informed by and tied back to the outcomes we’ve generated to date already in the discovery phase: the project brief (using the business goals, KPIs, and success criteria to append unique items to your Audit list); the user personas (using the goals, needs, and content preferences to add unique items to your Audit list); and the user stories.

Now that you know generally what to audit, and you have the tool in your inbox, let’s talk about how to execute an audit.

How to Execute a Discovery Audit

Here’s what to do:

1. Download our free Discovery Audit Workbook via the form above.
2. Go through the work you’ve done to date – brief, personas, user stories – and add unique tasks to the Audit Workbook based on that content.
3. Remove any sample items we included in the Workbook that don’t apply to you (for example, if you’re auditing a mobile app, the entire SEO section won’t apply).
4. Give the items unique IDs so you can reference them later.
5. Divvy up the task categories if there’s more than one person who will be helping out in running the audit. If it’s just you, ignore this step.
6. Get to auditing! Analyze the data, review the current implementation, and fill out the Observations and Recommendations column for each audit item. We’ve included some sample content for you in the Workbook you can download.

Discovery Audit FAQs

Here are some common questions and answers that will to help you run a successful audit:

What if my goals and KPIs weren’t outlined at the start of this project? Sometimes Goals & KPIs won’t have been outlined at the start of the original project. That’s alright – nothing you can do about that. Realistically, you should be using your new goals along with existing goals to paint a comprehensive picture. If that’s the case, try to evaluate the existing product against the Goals & KPIs that have been set for the upcoming project. For example, if trial account signups are a goal and the old product allows people to sign up for trials, get some data on how many average signups you’re getting right now, per month.

Where can I find my goal and KPI data? Some Goals and KPIs won’t be easily accessible from Google Analytics, or Firebase, or whatever your data tracking solution is. They might be accessible in the product’s back-end, a Google sheet somewhere or hell, even your CFO’s balance sheet. Make sure to figure out where the data is, who the gatekeeper is, and talk to them in order to get that data.

What if my current product isn’t tracking data? Some existing projects won’t be tracking data at all, which is unfortunate but somewhat common. If this is the case, we’ll often set up analytics tracking quickly – usually via Google Tag Manager – and start tracking data immediately so as to establish a baseline of some sort we can use later. I’d recommend doing that.

What do I do with the data? Looking at data or evaluating a user story is just the first step. The real key is what we can infer from it. What is the hypothesis? Why is revenue per visit so low? Try to establish a reasonable root cause, because that’s the real learning that can be applied to the new product. Note: it can be difficult to do this effectively or accurately without domain expertise.

Who should I involve in the audit? The input for items in the Usability & Design section – which will largely be about end-user user stories – should come from the actual end users whenever possible. It’s not about what we think is intuitive. It’s what the actual user thinks is intuitive. Where possible, sit down with the end user and get them to run through the existing user stories, and ask them for their input. The same thinking applies for the items around analytics and reporting. What do actual managers think about the intuitiveness and usefulness of the data that’s being collected? Consider this line of thought for everyone who touches the product, from product administrators to the individuals updating your Frequently Asked Questions section.

Wrapping it Up: The Discovery Audit Summary

We’re all busy people, and while comprehensiveness is valuable, so is succinctness. With that in mind, once you’ve completed the Discovery Audit and made some observations and recommendations, it’s time to wrap up the key findings into a summary (don’t worry, we’ve included the Discovery Audit Summary template in the Workbook download, which you can grab at the bottom of this post).

The Summary simply wraps up the key findings from the entire audit into an easily digestible, quick-reading format. It’s also somewhat easier to read than the entire spreadsheet used for the audit. Finally, it’s another step completed in the holistic digital project discovery phase, which gets us closer and closer to a comprehensive view of what a successful product will look like.

Next, we move into Information Architecture & User Journeys, where we use what we’ve uncovered so far to start actually mapping out the architecture of the product itself, and how users will navigate throughout that content. Read that post here!