Now the fun part. Here are six frameworks to help you examine the app:
Jobs to be done. Uncovering the core objective.
Personas. Goals, motivations, scenarios.
User familiarity. New users, intermediate users, and experts.
Expert evaluation. Using best practices and industry standards.
Inclusivity. How accessible is this app?
Zooming in and out. Aesthetic, functional, and strategic layers.
Many of these frameworks complement each other, and you’ll use a combination of them in your critique.
Jobs to Be Done: Uncovering the Core Objective
Do you know what job a milkshake is hired to do? In his famous example, Clayton Christensen recounts the story of a struggling milkshake company trying to increase revenue. They try everything—making milkshakes thicker, larger, but nothing works. Only after observing customers buying milkshakes do they come to a realization.
In the morning, customers prefer milkshakes because they’re a portable source of energy when commuting to work. The milkshake fills them up, it’s easy to hold in the car, and it doesn’t leave a mess. In the evening parents swing by with their kids to treat them to the shake. But kids struggle. The straw is difficult to use, and the packaging is clumsy at best for small hands.
As you can see, the job varies depending on the context. Uncovering the jobs-to-be-done, or JTBD, will provide you objective criteria to critique the app against.
What is the customer trying to accomplish?
If the app didn’t exist, what would the customer do?
Is the alternative better or worse than the app? Why?
A JTBD might be enough context to start getting into app flows and the UI.
Personas: Goals, Attitudes, and Behaviors
While JTBD provides an objective high-level framework, personas can give you an extra layer of empathy with details. They’re especially helpful when you’re addressing users who are very different from you and the interviewer.
In the app critique context, you won’t have research handy, so you’ll need to make these up as you go. One way of organizing this info is through proto-personas, as coined by Jeff Gothelf. If you have a whiteboard handy, jot down your assumed persona characteristics:
Goals. What objective(s) are they trying to accomplish?
Motivation. Why are they motivated to use the app?
Attitudes. How do they feel about the task they’re trying to accomplish?
Behaviors. Where in the context of a user’s life does the app fit in? Where do they use it—home, work, or on their commute? How do they use it? What else is happening in the background?
Pairing personas with a scenario (think mini customer journey) will help you make the most of them by putting yourself in their shoes and imagining how they would use the app to accomplish a task.
You can segment your prototypical persona further, based on how familiar they are with the app.
- Welcoming newcomers. New users offer a different type of challenge compared to your everyday users who are already bought in. The app will need to communicate quickly to entice the user to continue. Depending on how much time you have, consider stepping all the way back to the initial app discovery phase. How do the users find the app? What gets them to download and try it out?
As a new user who downloaded the app:
What is the value proposition? Can I easily find it? Does it resonate with my needs?
Do I need to sign up or can I try it out without commitment?
How much effort do I need to make in the beginning?
What barriers or friction do I encounter during onboarding?
Is it clear what I need to do next?
How much guidance do I need in the onboarding process?
How does this app incentivize me to use it more?
The new-user perspective might be easiest to take during the app critique. It doesn’t assume prior app knowledge and it gets you and the interviewer on the same page by starting out fresh.
- Becoming a regular user. With continued use of the app, the customer gains experience and becomes an intermediate user. Tasks that took a long time are a snap to complete. For an app critique, this provides a rich exploration area—how do I as a new user become a repeat customer?
How does the app guide me?
Where can I find shortcuts? What accelerators exist to improve my workflow?
How does the experience grow with my needs?
How can the experience be more personalized?
This approach works well with popular apps like Facebook that you and the interviewer use frequently. Be sure to establish some common ground first, though, as they might be proficient on parts of the app that you don’t use.
- Expert users. Depending on the application, experts have taken the time to squeeze every last bit of efficiency out of the tool. Similar to intermediate users, you can think about evaluating the experience from a transition perspective:
How does the app level up intermediate users?
Where can you find advanced functionality?
What tools are missing that might be helpful for an expert?
For the app critique, this might be the hardest perspective to take, but it might be worthwhile for tools that are highly task oriented. Usually these tend to be enterprise apps like Photoshop or Visual Studio. Taking this perspective shows that you know the user well and that you’ve developed a point of view on how to meet your user’s needs.
Expert App Evaluation: Industry Best Practices
JTBD, personas, and different user types will help you define a critical path that can help you stay focused in the limited time that you have. As you’re putting yourself in the shoes of your prototypical user and thinking out loud, you want to also overlay your design-expert commentary.
One way to communicate your expertise is by referring to existing popular guides, such as Nielsen’s 10 Usability Heuristics, which can be applied for different platforms. Depending on the app that you’re evaluating, you can also lean in on official recommendations, such as Apple’s Human Interface Guidelines or Google’s Material Design.
As an expert, your knowledge doesn’t have to stop at these guidelines of course. This is an opportunity for you to demonstrate your knowledge based on experience. What patterns have you seen hold up over time? What patterns work well for this industry or this audience segment? What have you seen fail or work in practice when testing concepts?
Inclusivity and Accessibility
So far, we’ve talked about ideal user journeys and expert evaluation techniques with no mention of edge cases or customer impairments. If you want to step up in the critique, consider accessibility. Ensuring apps don’t exclude users is not a nice-to-have—in many parts of the world it’s the law. The Google Design team created an accessibility card deck to generate different scenarios.
Figure: Google’s Accessibility Card Deck
The Google Design team created an accessibility card deck to generate different scenarios.
I won’t be able to do justice to accessibility here, but for the purpose of the app critique you can think about the following impairments:
Vision. What affordances does the app make for users with low vision, color blindness, or blindness? Think size, color, contrast, or environmental factors like screen glare.
Hearing. How does the app make use of sound? Does it provide backup cues? Does it have alternative text or captions? Think about the app’s use in a busy environment—for example, a busy train station.
Motor. Does the app require intricate touch gestures? Are tap targets large and easy to discover? Think about using the app while walking.
Cognitive. How much cognitive load does the app require of the user? Does the user need to memorize instructions or remember complicated operations?
For more info, take a look at Inclusive Design by Microsoft.
Zooming In and Out on the Experience
As you’re going through the customer journey, you’ll at times be going back to the strategic goals of the critique to make a point, diving into app functionality or zooming all the way in to a particular micro-interaction.
Figure: Cover Functional and Strategic Elements
Consider both visible and invisible layers.
One way to think about this is a top-down structure that encompasses visible (aesthetic) and invisible (functional and strategic) layers:
Aesthetic. How it looks. Domain of visual design, motion, sound, touch.
Functional. How it works. Domain of information architecture and interaction design.
Strategic. Why it exists. Domain of value props and business strategy.
This layer is a summation of all the parts that you experience when you open up the app. How does this application achieve visual design and consistency?
Brand. What message is the app trying to convey?
Visual. How does the app use the visual language (color, shape, type, and so on) to reinforce its identity? What is the aesthetic experience like?
Motion. How does animation help orient the user? Are there any signature moments?
Sound. How does the app use sound to inform or entertain?
Touch. What gestures exist and are they easily discoverable?
Pairing this layer with personas will be helpful. This is also a prime layer to think about accessibility and inclusivity.
Below aesthetics lies functionality—these are core mechanics that help the user accomplish their goal.
Information architecture. How is the information labeled and organized? What paths exist to browse or search?
Interaction design. How does the app work? What macro and micro flows exist? What patterns are being used, what are the trade-offs?
This layer pairs well with personas and expert evaluation.
In this layer you’ll find problem framing and value proposition.
App. Why does this app exist? For whom? What problem is it solving?
Business. What problem is the company trying to solve? What are the values that we believe in? How does the company want to portray itself? How does it make money? What’s the company’s mission? What values does it abide by?
The strategy layer overlaps significantly with JTBD, so you’ll probably be going back and forth between the two.