Project Overview

My design team worked with *PARLAY - a Product startup company developing a tool that will help companies test feature viability for their websites and applications. Parlay provides actionable data from users who can evaluate proposed features before they are fully developed.

Duration: 3 week Team Sprint

My Team: Julie Vera, Ezra Tollett, Eric Pierce

Tools: Pen, Paper,  Optimalworkshop, Sketch, Invision, Xtensio, Figma

Device: Desktop

MY Role: User Research | Competitive Analysis | Sketching| Ideating | Prototyping

* PARLAY - formerly called Parlay, now renamed as Parlor.io

PARLAY IN ACTION

The above images show how the Parlay preview will display in Instagram. We are previewing a new button in red next to the image. The preview asks the users if they are ready to answer a few questions, based on their response to the CHI (Customer Happiness Index) model. CHI model represents 3 sentiments- Positive, Neutral and Negative.

Based on the users CHI response; whether they like, dislike or are undecided about the new button, the survey asks them quick close ended questions to gauge their feedback. 

The preview is a static representation of the product and contextual in asking questions related to the new feature. This feedback is then translated to the product teams for further analysis.

User and admin facing screens

There are two core elements of the Parlay product -

1) Preview and User-Facing “Chatbot” survey. No AI, just a set of static, predetermined questions in the form of a survey.

2) Admin-Facing survey analytics dashboard which collects its metrics from the Survey.  This would enable admins and data analysts to see high level trends as well as analyze the data.

Client Brief

Find a novel way to capture and measure user feedback on digital feature previews.

• While maintaining a good user experience.

• And providing actionable data.

The deliverables

1. 3 chatbot Surveys: Based on the CHI model, there will be a quick 5 question survey that will be contextual, conversational and closed-ended. The survey questions will be different for all 3 sentiments.

2. Analytics Dashboard: To design an actionable dashboard based on the users survey feedback, so that product teams are able to make proactive design decisions.

What is the benefit of parlay?

• Saves money on feature development

• Keeps Product Teams connected to Users 

• Engages Users

our design approach

User Research

Research and user interviews formed an extensive part of our project. When a new product is to be launched into the market, it is important to study, analyze and get a good frame of reference of the task at hand. We had two major areas of analysis: discovery and data collection. 

competitive research

We carried out a competitive market analysis of some of the direct and in-direct competitors of Parlay. These competitors had similar survey tools or sales tools. We wanted to understand how they utilized and executed these products. The clients also suggested we take a look at some of them and how they worked.  

Feature Analysis

Competitive Analysis Takeaways

•  A lot of other companies have most of these features, but none of them are specifically for direct product feedback.

• NONE of them really do true contextual feature previews!

COMPETITOR DASHBOARDS

These are images of some of the dashboards of similar competitor websites. Team Parlay wanted us to take a step beyond the regular dashboard. They did not want the usual graphs and bar charts depicting data. Data was to be represented in an unique way so that different users (product teams and developers) accessing the dashboard are able to gain actionable insights from it. We needed to think out-of-the-box.

research interviews

We interviewed 12 industry experts from the field of product development. We wanted to analyze how products get built. We asked them how design decisions get made, and how feedback gets collected, and how all that information comes together to build a product.

“How do you get user feedback on newly-added features?”

“Are developers interested in feedback?”

“How do you use analytics to inform your product decisions?”

“What metrics are most key to your design decisions?”

TAKEAWAYS

We found out that these people were having a hard time getting proactive feedback. They try getting feedback  in a lot of different ways. Sometimes they just wait for customers to suggest a new feature, sometimes they go out do guerilla testing, or consult a group of core users, but there isn’t one specific way of doing it.

personas

Based on our research interviews, we created 3 personas of people who are potential users of the product. All user personas need different levels of data and insight from user feedback. 

• Product Manager

  • What do the users want?

• Software/Hardware/Web Developer

  • What is being prioritized and why?

• Data Analyst

  • What is working/not working?

Different Needs for different roles

All product team members need different levels of data and insights from user feedback.

PRODUCT PEOPLE

Assurances = Need to understand that what they’re working on is worthwhile and users will respond to the new product or feature without sinking a ton of money into it. They have the advantage of using previews before fully developing the product.

DEVELOPERS

Actionable Insights = Need to understand the implications of a feature. If they get feedback that one feature is preferred over another, how can they get ahead and plan for that? This will helps with development costs. Tech Debt.

DATA ANALYSTS

More Data = More data is always helpful. This helps product owners and other business people decide what is working and what isn’t.

UX DESIGNERS

Usability Testing = Need on-demand feedback from power-users. These users aren’t always represented in user surveys and usability studies.

DESIGN DEVELOPMENT : Using UX Design Principles

After competitive analysis and user interviews, it was time to brainstorm our ideas for creating a short conversational survey and making that survey feedback actionable on the analytics dashboard for the product team.  

We started with the surveys first. Based on Parlays CHI model for the survey, we started sketching the survey question flow. As per the CHI model, there are 3 sentiments: Positive, Neutral and Negative.

Based on the users Positive, Neutral or Negative response, we designed different questions for different sentiments. During one of our brainstorming sessions, we came up with the idea of assigning Heuristics to each survey question. 

To validate each survey question, we thought heuristics would be the best metric as it is a process used to test usability. 

Why Heuristics?

Heuristics are an established convention within the design and development communities.

In User Experience Design, Heuristics are 

• Experience-based techniques for problem solving, learning, and discovery.

• Proceeding to a solution by trial and error or by rules that are only loosely defined.

• Give a solution not guaranteed to be optimal and can pick up a lot of irrelevant data.

Heuristic methods speed up the process of finding a satisfactory solution via mental shortcuts to ease the cognitive load of making a decision.

The Heuristic Model, Refined

By their nature, though, Heuristics are intrinsically customizable, meaning that we can create our own heuristics and definitions. This makes them both easily adopted, as well as scalable to almost any preview or feature a development team could conceive of and test.

Since we’re designing a survey in which users will be evaluating proposed features before they’ve been fully developed, our heuristics needed to be tweaked in order to accommodate speculative feedback.

We shortlisted these heuristics with the help of our client and a community-wide survey in which respondents were asked to list their top five established heuristics.

Top Heuristics

• Relevance - Is this feature a priority?

• Impact - Will it change how you use the site?

• Value - Will it add value?

• Predictability - Will it be easy to understand?

• Consistency - Will it fit with the larger site?

• Learnability - Do you feel you could pick it up quickly?

Tying survey data to heuristic performance - going from quantitative to qualitative - will give product teams a straightforward means of understanding how a proposed feature did in the eyes of users, and help them make more informed decisions regarding its viability or iteration.

What’s the best way to visually represent heuristic performance?

In our user interviews, one of the insights given by our interviewees was about how they use QFD matrix (Harvey Balls) to measure qualitative information. QFD is a focused methodology for carefully listening to the 'voice of the customer' and then effectively responding to those needs and expectations. 

Harvey Balls

Harvey Balls are round ideograms used for visual communication of qualitative information.

They are commonly used in comparison tables to indicate the degree to which a particular item meets a particular criterion.

Proactive feedback system

To tackle the task of a proactive feedback system, we assigned each heuristic to a preview survey question. And the answers to the survey questions are then used to measure heuristic performance of the previews. The corresponding harvey balls then reflect the result of the survey feedback. This helps product teams gain actionable insights directly from the users.

Developing the Customer survey

The preview for the survey followed the CHI method with 3 different sentiments. We needed to design 3 different surveys for each sentiments based on the heuristics. The heuristic assigned to each question would be different.

Since we wanted to keep the users engaged and get consistent feedback from them, the survey questions needed to be:

•  Conversational

•  Brief

•  Contextual

•  Closed-ended

Ideating the Survey Design

POSITIVE sentiment survey

The Survey Preview starts by asking the user a few questions. Depending on the option chosen, the next question is displayed. Bub, Parlays chatbot assists in asking each question. Based on the CHI model, we designed similar surveys for the neutral and negative sentiments.

Positive Sentiment Survey Flow

Analytics Dashboard

After we designed the 3 survey flows, it was time to tackle the dashboard. Earlier, I mentioned about using Harvey balls and connecting them to each survey question to make the feedback more actionable. Our main challenge was to design a dashboard which speaks the same language for all the product teams like the Developer, UX Designer, Data Analysts and Product teams. 

At a glance, the dashboard would give a preview of the Feature Version tested. It would state a percentages of the CHI sentiment as well as the number of users surveyed. It would also give product teams the option to call for action; as whether to launch the feature or kill it completely. The heuristics with the corresponding Harvey balls will inform the teams how the feature performed for the users. The comments section will also give them direct feedback from active users.

DASHboard design

Parlay Dashboard

Team Parlays Implementation of the Heuristic framework in the Negative Survey

How Heuristics sets the framework for Design Decisions for the Product Teams

The brief given by the client was to design a conversational survey and a proactive analytics dashboard. We designed a customized heuristics framework that lays the foundation for the usability and efficiency of the product for which it is designed. The clients were appreciative of the heuristic framework and implemented it seamlessly into their own iterated design. 

Parlay is an innovative product that will change the way product team make design decisions. User research made it apparent how having actionable insights from users will benefit businesses and also save a lot of effort and frustration for developers. It was an amazing experience to be a part of Parlays dynamic journey and also interesting to see how it will impact businesses and products in the future.

The next steps would be to perform usability testing of the chatbot survey and the dashboard functionality and iterate more on the design and development.