Narrative

People come to Yahoo Home to stay up to date on what matters to them most. The key here is ‘what matters to them’. This makes our job to ‘Sift through the noise and only surface relevant and personalized content for each user. The pace and precision with which we can get to a state where most of what a user sees on the homepage is based on their interests, the more likely we are to retain the user.

Strategy and kickoff

To improve our understanding of our users and their interests, we will be asking them to provide us with overall feedback about content in the stream. This feedback will be a general/broad-spectrum on the entire content piece and allow us to reinforce our inferred signals to understand the user better. We want to launch this in phases in order to rapidly deploy and gather lots of data points quickly.

From a user standpoint, this feature will function as a Show More / Fewer tool. The feedback will not be editable or exposed via any feedback management tools. The user will not be required to sign in to start with but will be offered the opportunity to do so.

User Journey

I created a simple user journey flow to show proof of concept of how the icons may look, location, and actions a user can take. Straight forward with only two main actions, show more and show less. With Show more, we invite them to log in to save their preference, and with show less we ask for additional feedback.

Desktop Prototype

I build a fully functioning prototype in Principle to use during our moderated user testing sessions. I opted for an animated prototype rather than a Invision click through because I thought I could learn a lot just from the micro interactions and animations such as the hover effects, and tooltip flyouts. How was the speed? Did users notice them? Did they read them? Are they distracting? I wouldn't have gotten the same quality of responses from a static screen and click through prototype.

Key insights

• P2 felt it was difficult to respond with a Thumbs Up or Down without reading the article and was concerned it would contribute to people reading fewer articles.
• Most participants (P1, P3, P4, P5) thought that they would respond based on whether they liked the article. Others (P2) thought they would respond whether they would recommend the article.
• All participants understood they would see more basketball, ESPN, or sports stories if they clicked Thumbs Up
• 4 participants saw that the message changed from “Show me more stories like this” to “Great, we will show you more stories like this” when they clicked Thumbs Up. (P4 clicked too quickly to see.)
• Participants weren’t sure at what level these settings were. Overall, all participants thought they should manage content at the Topic level

For mWeb, I wanted to have a very similar experience, but instead of traditional icons, I wanted to try to use native gestures to simplify the process for users. This included challenges for discoverability, but I remedied those by adding hint animations to educate the user.

User Journey

Variations

I tried a few different variations on the swipe design. I tried multiple styles and interactions. For example, in order to fit 3 actions, I tried a single sided swipe with all three actions in buttons. You would swipe and then tap the option you wanted. This felt cumbersome, so I tried a variant of a double sided swipe where as soon as you swipe the action occurs (think Tinder) and I moved the "more" outside of a swipe gesture and into an icon on the stream. This icon would open up an action sheet that slides from the bottom.

User testing and prototyping

For our mobile responsive prototype, I did it in Principle again because I needed to see how users interaction with the native gestures. We first tested a mobile web version of the feature, with multiple variations tested in random sequence to reduce bias. These variations included multi-sided swipe, single-sided swipe, variations of language, and variations of icons.

Key insights

• P1 and P4 were able to discover Show More/Less when prompted to manage their feed. Other participants guessed it would be in settings, profile, or under the ellipses (in two-sided Swipe)
• Between the one-sided Swipe and two-Sided Swipe, all participants preferred the one-sided swipe. They were mainly worried about errors and felt it was easier to understand.
• All participants understood Show More/Less would increase/decrease the amount of similar content in their feed. Some participants thought it would be instantaneous with articles being added/removed immediately, while others thought they would need to refresh the page. Most participants thought it would manage content at a topic or category level, like sports, entertainment, baseball, or Cardinals.
• Participants expected a wide variety of options under the Ellipses/More Options button including hiding articles, saving/sharing/downloading articles, and refining the Show More/Less setting to a specific topic or category.
• A few participants misread “Hide posts from Yahoo Lifestyle” as “Hide post from Yahoo Lifestyle” and thought it would hide the article with no other effects. Participants also misunderstood Yahoo Lifestyle as managing the category of Lifestyle and not the publisher Yahoo Lifestyle.
• All participants felt positively towards the language used. P2 and P3 thought it could be more specific. P5 felt the color was not intuitive.
• Participants were asked how they felt about the heart and crossed heart icons and whether they would prefer a thumbs up and down. Overall, participants did not have strong opinions about the icons used.

One little detail that I needed to address was the iconography for this feature. I explored many different options and we tested a couple different ones during user testing .

A few of these icons made more sense than others, while some were more abstract. At this time, Yahoo had been using the heart with the line through it to signify "don't show me this publisher anymore" but we decided that the heart wasn't a good symbol for showing something that people wanted to see more or less of. In fact the emotional weight the heart carried could be in direct conflict with the tone of the content itself. For example, a piece of content may be around a terrible tragedy or natural disaster, and although people would be sad and not "love" it, they would want to stay informed and up to date and see more news about it. So it caused some internal conflict and hesitation. After testing several different variations, the Thumbs actually resonated with people the most and didn't suffer from the same kind of emotional attachment as the heart. Instead of users thinking of the thumb as like/dislike in regards to the subject matter of the content, they saw it more as a green light / red light to show them more or not. Very interesting.

Since users are generally more vocal about things they dislike than things they like, we wanted to gather additional feedback to further personalize their stream. This also gives the user the confirmation that we are actually listening to what signals they are giving us. I designed a very simple and optional feedback form if they tell us they wanted to see less of something. This would help train our ML algo to serve up more relevant content.

Deliverables

After user testing and multiple iterations on the overall flow and design. We decided to scrap the "why didn't you like this" portion" for this phase. We wanted to get this out and start receiving a lot of data points for users. We decided to launch this alongside phase 2. For the desktop, I created a style guide fore the developers to use. You can find that below.