Case-Study-Header-Alt-3

Background

Disclaimer: Microsoft/Xbox neither sponsored nor endorsed this project.

  • Project type: Speculative
  • My roles: End-to-end product designer

Problem

Because the Xbox Game Pass service offers a varied selection of so many games, it can be difficult for users to find something of interest to play.

Hypothesis

Adding social discoverability features to the Game Pass app on Xbox consoles could make it easier for users to find a new game to play and increase satisfaction with the service.

Project highlights

Recruitment setbacks

Gamers aren't afraid to share their thoughts online, but I learned the hard way that getting them to do so in an interview can be difficult.

An invalidated hypothesis

Users were generally interested in improvements to discoverability on Game Pass, but they weren't fans of my hypothesized social discoverability solutions.

Testing limitations

To best simulate the experience of using Game Pass, I wanted to test my prototype with an Xbox controller but remote testing solutions don't yet support it.

Designing for a large interface

Following Microsoft's guidelines for the "10-foot experience" constrained what I could design and implement.

Splash-Image-3

Understanding users

Xbox-Game-Pass-PC-Family_Key-Art

Secondary research

To better understand Xbox Game Pass's impact on the video game market and the kind of users who subscribe to and use the service, I researched the ways Game Pass has shaken up the industry and general sentiments among subscribers.

  • As of April 2021, Xbox Game Pass had 23 million subscribers
  • Game Pass subscribers spend 20% more on games than non-subscribers
  • Game Pass subscribers play 30% more games than non-subscribers
  • Some users rely on third party tools like gamepassport.net to filter the Game Pass library

User survey

Finding users to speak with was a challenge due to gatekeeping and strict rules on the online platforms where gamers typically congregate. Eventually, I got permission to post on r/XboxGamePass to ask for interview participants. 62 users responded.

  • Users were frustrated with the lack of filtering options in the Game Pass app
  • Users wanted review scores of some sort displayed for games in the catalogue
  • Users wanted a "Because you played this game, try this game" feature
Screen-Shot-2021-10-27-at-5.06.21-PM
Quote-Card-1

User interviews

I asked Game Pass subscribers about their general experience with the service and a potential playlist solution I was considering — but they were lukewarm about the proposed social discoverability features.

The poor quality of game recommendations on the service frustrated users

Social recommendations were acceptable to users as long as they weren't "shoved in their face"

A lack of friends on their Xbox friend list made users skeptical of the usefulness of social discovery features

User-Journey-Map-Cropped

The user journey

To better understand and visualize the areas where users were getting stuck when trying to find a new game to play on Game Pass, I created a journey map.

Users feel overwhelmed by the amount of games on Game Pass

The limited recommendations the service already makes are irrelevant or inaccurate

The inability to filter the catalogue for relevant games is the biggest pain point in the journey

Research showed that while users were unhappy with discoverability, they weren't interested in proposed social features to solve it.

Generating solutions

Revising the hypothesis

Initially, I hypothesized that the solution to the discovery problem would be to add a feature that allowed users to view and create custom "playlists" of games on the service that they could collaborate on and share — but users strongly preferred personalized recommendations based on the games they were already playing.

How-Might-We-Transparent
Game-Pass-Discovery-User-Flow-V3

Go with the flow

While I could've added recommendation features to existing pages, I realized early on that doing so would make things too busy and difficult to navigate, so I had to create a new flow that centered around a new "Picks for you" page — a central hub for all recommendations.

Recommendation station

Creating a dedicated page for recommendations would give me the ability to present different types of them without "shoving them in the face" of the users — which they made clear in interviews they didn't want.

I broke the recommendations down into four main categories:

  • "Because you played" — recommendations based on a specific game a user has played
  • "What your friends are playing" — a personalized list of games popular with a user's top Xbox friends
  • "Most popular game" — Recommendations based on what's trending among all subscribers
  • "Picks based on your top genres" — curated selections of games based on the type of games a user plays most on the service (first-person shooter, role playing game, etc)
For-You-Page-V1

We all have preferences

To determine the best layout for the Picks for you page, and which sections users would most like to see at the top of the page, I ran a preference test between three versions. 80% of users preferred version 1 of the layout. 

Preference-Test-Annotated

I needed to test in high fidelity to give users full context, but I couldn't find a way to simulate using a gamepad.

Hi-fi tests & revisions

Screen-Shot-2021-10-29-at-3.22.15-PM

Testing trigger fingers

Because I wasn't able to test my high-fidelity prototype with an Xbox gamepad, I ran a first-click test with static images of the designs and a series of follow-up questions instead. While not ideal, it still provided valuable insights.

90% of users successfully completed each of the 10 tasks in the test

Users were confused by the icons on the game details modal, namely the thumbs up/down icons

28% of users found the Picks for you page to be too "busy" visually

Game-Modal-Updates

Revision: unclear icons

The revisions I had to make to my designs after user testing were minimal, and many of them were a result of not having access to the original product's icon set. The new solution isn't perfect either, but requires further testing to validate.

1 + 2: Changed the thumbs up/down recommendation refinement icons to up and down arrows

3: Updated "Join club" icon from a controller to a group of people

Picks-for-you-revised

Revision: visual clutter

A sizable amount of users said they found the overall pages to be too busy thanks to the sheer amount of information presented on each game's card. To address that, I cut out some of the unnecessary visual indicators.

1: Removed "New release/Leaving/Trending" status banners from game art

2: Removed icons indicating friends are playing certain games

However, I retained the Metacritic scores because users overwhelmingly loved their inclusion.

Designing & testing for a large, unusual interface was challenging, but I loved stretching my skills.

Reflection

Image-from-iOS-3

Next steps

Test revisions

Though the revisions I made were small, I still want to test the new upvote/downvote icons I chose on the game details modal. I'm not sure they're any clearer than the thumbs I used before.

Add filter and sort options

The ability to search or narrow the library with advanced filters was one of the most-requested features among users, but tackling that was outside the scope of this project.

Re-test with gamepad support

I wasn't able to make this happen so I'm not fully confident in my testing results, but there must be a way to create a prototype that supports a gamepad and then send that to users to test remotely.

Image-from-iOS

What I learned

How to design for larger interfaces

Until this project, I'd only ever designed for PC and mobile interfaces. Learning to design for TV and other larger displays was a challenge, but I'm glad to have the experience.

Sometimes less is more

The tight scope of this project (no more than 80 total work hours), really pushed me to keep my often overly ambitious impulses in check and focus on what was feasible to deliver.

UX in the games space is nascent

That surprised me considering it's an industry where user interaction is the core experience. But across competitors and in Xbox's own product I saw how much room there is to improve.