Steam Product Page — Probability of Liking a Game
Study Goals
This study proposes to understand the decision-making process of users who purchase videogames online. The gaming platform Steam was the object of study (which I’m not affiliated with). The inspiration for this study arose with Google Maps’ probability of liking a restaurant, i.e., “match score” feature.
Semi-Structured Interview and Contextual Inquiry
● 2 users were interviewed
● Contextual inquiry: after the semi-structured interview users were asked to use the Steam product page to mimic their own steps in the decision-making process of buying a game
● Reward: coffee and cake
Most Meaningful Insights
Female, 18, Student
● Budget sensitive
● Watches gameplays on YouTube and Twitch to decide how she feels about the game
● Trusts opinions of social media influencers
● Doesn’t trust opinions of friends/family because they are not pro gamers
● More prone to decide to buy another game from the same studio if previous experiences with other games were positive
● Usually purchases on PS store games on sale (compares prices on other websites)
● Also knows Steam, UPlay, Xbox store, Nintendo e-Shop
Male, 30, Computer Science Engineer
● Appreciates a good deal but also buys pre-orders if he’s really interested in the game
● Cares about reviews on YouTube and Twitch channels by watching (without spoilers) — his favorite is beforeyoubuy (which I’m not affiliated with)
● Trusts friends’ opinions about a game if they have the same “taste” in games
● Informed of latest news and trends
● Usually decides to purchase games after seeing their demos on E3
● Usually purchases on PS store games on sale (doesn’t compare prices)
● Also knows Steam, HumbleBundle, UPlay, GOG, G2A, Itch.io
Most Meaningful Suggestions of Improvements
Female, 18, Student
● Calendar with updates of online events (e.g. GTA V)
● History of purchases, wishlist, favorites
● History of reviews she has previously made to other games from the same studio
● Tab of games recommended for her
● Recommended games section should be higher on the page (above the fold) — likes to analyze more details by scrolling
● Assignment of Likes/Dislikes on games
● Recommendations of new friends to make in new games (e.g. Overwatch)
Male, 30, Computer Science Engineer
● Link his YouTube and Twitch accounts with Steam to get updates on recommendations of gameplay or reviews his favorite channels/influencers make
● Improve tags attribution as they usually don’t match correctly
● Improve recommendations section as recommended games are most times not connected at all with the type of game presented
● Show what games other players similar to him play in order to “convince” him this specific game is worthwhile
● Cares the most about reviews, video gameplay, game tags and price (above the fold) — doesn’t scroll
Competitive Analysis
The competitors which were analyzed for the purposes of this study were Itch.io, GOG, Origin, UPlay, Epic Games, GamersGate, and Blizzard’s Battle.net. Note that none of these competitors use the metrics they collect to advise the user regarding the probability of them liking the game that they are viewing. The closest feature that users mention that relates to this is the OpenCritic, Metacritic or IGN ratings of the game seen on Steam and Epic Games Store.
This list was collected by searching public information on the web and with the confirmation of users. Another way to do it would be to do usability testing of competitors’ websites, pages, products, and analyze what are the positives and the challenges that our product has.
Personas & Storyboarding
For the Storyboarding of Henry Mills, the most important pain point is the diminished retention on the page, whereas he begins by browsing the game on Steam, leaves the page and then returns to add the game to Steam’s wishlist in the end of the process, because Steam is one of the platforms with the most competitive prices. Nonetheless, the search for confirmation through reviews and gameplays is done outside of Steam.
For the Storyboarding of Annette Dale, the most important pain points are: the confusing tags attributed to games, which also negatively affect the recommended games section on Steam’s product page; and the page is
extremely long but seems to not contain enough information that users are looking for to help them make the final decision to purchase the game in a timely fashion.
Information Architecture
Information Architecture with Card Sorting
● Improve the labeling, grouping and organization of information
● Jakob Nielsen recommends around 15 users for this technique to be fruitful
● The Card Sorting was applied in an open mode to provide more freedom to users and allow more flow of ideas
● Users were asked to group concepts together as these made sense to them, as well as name each category. Users were also given the option to modify any card name to words that were better understood by them. To finish, users were asked to organize each concept by order of importance inside each category and, then, to organize each grouping by order of importance — this made possible to see the concepts on the far left at the top are perceived as the most important for users and on the far right at the bottom are the least important concepts for users
Card Sorting Results
● Both users attributed equal importance to the “Probability of you liking the game”, whereas both placed it in first place on the second category
● Given that there are only 2 results it is challenging to gather the necessary data to be able to draw conclusions of the most commonly used category
names, as well as similar distributions of concepts by categories
● Future work could involve making a closed Card Sort or launching one online in order to receive a high volume of responses
Paper Prototyping
Rating Scale Equal to Steam’s
The rating scale used was as follows:
1-Overwhelmingly Likely
2-Very Likely
3-Likely
4-Mostly Likely
5-Mixed
6-Mostly Unlikely
7-Unlikely
This scale was changed in a way to maintain the standards of the website, therefore there are 4 positive degrees, 1 neutral and 2 negative degrees. Further tests are needed to find out if users interpret these labels correctly as there have been some doubts about what mostly likely means
Guerrilla Testing & RITE — Test Scenario
“You found this game named “The Game” and you opened the game’s page on Steam. You analyze the details of the game to understand if it is appropriate for you. Let me know your thoughts about the subsection “Probability of you liking the game”, your suggestions of improvements and what you understand this is”
Note: test was performed to 7 users in total resulting in 5 iterations
Iteration 1
● 3 users tested the prototype
● “I don’t know why I’m going to like it. Is it because it’s the same genre?”
● They suggested adding reasons why the game would be recommended to that specific
player, such as, how many people bought the game, how many people played the game,
popular opinions
● The decision to add a tooltip activated by hovering the mouse was based on the same
behavior being demonstrated on other parts of the webpage, such as reviews
Iteration 2
● 1 user tested the prototype including the previous improvements users had suggested
● Suggestion: add “Because you liked xyz similar game”
Iteration 3
● 1 user tested the prototype including the previous improvements users had suggested
● Suggestion: possibly change the rating scale to a number as it is more perceivable in value and less vague
● This user mentioned that he would like to see more personalized information in this tooltip, therefore, the first 2 topics were changed to make use of the tags to communicate the same information in a more personalized way
Iteration 4
● 1 user tested the prototype including the previous improvements users had suggested
● Suggestion: change the number to percentage; and add how many friends are playing the game, which the user mentioned should come before how many people bought the game and how many people are playing the game as it is more personal and related to the individual
Iteration 5 (Final Iteration)
● 1 user tested the prototype including the previous improvements users had suggested
● Suggestion: use reduced notation (e.g. 1K) as the granularity is not required in this case to convey the message; remove how many people play the game because users can already see this information in a more personalized way through their friends on topic 3; and add the user’s favorite streamer to the list, stating if and when the streamer has streamed this game (both Twitch and Steam accounts, and possibly YouTube gaming, would have to be linked to get a list of followed streamers)
● The design of all reviews should be consistent — either all percentages or all Likert-type ratings. In this case, given that this feature is being implemented in a pre-existing design, the Likert-type rating was chosen as it is the same as other details
Wireframe (made in Axure)
Design Principles at play:
● Gestalt Law of Proximity — group closer-together elements, separating
them from those farther apart, allowing for white space to make this separation, supported by the Card Sorting exercise
● Hierarchy was changed based on the Card Sorting exercise
● The tooltip’s style was preserved for consistency, although bullet points were added to increase legibility
● Principles of alignment, color, typography, contrast, were kept to preserve consistency, and to avoid adding more variables
Hi-Fi Prototype (made in HTML & CSS)
Suggestion: Heuristic Evaluation
This is a step that was not taken during the process of this study and that would have been a positive addition to better prepare the prototype to test with real-users
Usability Testing
Task Scenario and Main Task
“You found this new game called Sims 3, but you are not sure if you are going to like it. You decided to navigate to the Sims 3 game page in Steam’s website. You are free to navigate the page as you would in a real-life situation in order to make a decision to purchase it or not. Please make sure you speak aloud and explain every step you are taking in order to make your decision, even if it means leaving the page.
You are finished with this task when you decide:
1. You don’t want to buy the game
2. You’re interested in the game and want to add it to the wishlist
3. You’re interested in the game and want to add it to the shopping cart
Please note that, for the purposes of this test, we’re not looking for an analysis of the game, only the information you can find on this page.”
Metrics for the Purposes of Collecting Quantitative Information
● Metrics for quantitative information: page retention, noticing new feature, decision to buy/not buy/add to wishlist, task success, need to scroll the page
● A good addition to this study would have been to add analytics collection to the code in order to gather these metrics and connect these to Google Analytics (also valuable for non-moderated and mid-large scale tests)
Interpretation of Results
User Insights
● Time on page was useful to establish the percentage of retention on the page, in which one user left the Steam website to see gameplay videos of influencers on Twitch — his total time on page was 31% (this user may be considered biased as he was already familiar with Steam and didn’t notice the new feature)
● 2 users scrolled down the page. One of them analyzed the page thoroughly; the other was looking for the price, which has been mentioned by a third user* that it should be above the fold along with all the overall information of the game
● Adding the game to the wishlist seems to be connected to less impulsive buyers and the fact that Steam holds frequent deals and promotions that users are aware of (users also get notified when the price of the item in the wishlist changes)
● One user* simply decided not to buy, explaining that he needs to think about it, but has not scrolled down the page saying that the information he analyzed above the fold was sufficient (except for the price, which he feels should also be there)
● The tooltip was viewed as a positive addition by most users and as a good collection of information to see at a glance. Even though one user read the tooltip, he wasn’t sure where that information came from and was skeptical about it
Future Work & Iterations
● After the usability testing is completed and results are analyzed, further iterations need to be contemplated
● Consider exploring if users react positively to a change of the location of the price to a place above the fold
● Consider creating a subsection on the page with detailed information about how the metrics on the tooltip were gathered, as none of the items in the tooltip are links (and cannot be transformed into links)
● Further tests are needed to interpret if users understand the rating scale used in the new feature
● Implement or interpret already implemented analytics data on the page
● Eye-tracking studies would be helpful to understand further where to best position this element
