flex.amazon.com - FAQs & "I Flex For"
This study had high visibility and a quick turnaround to assess effectiveness of Amazon Flex website's frequently asked questions (FAQs) content and the "I Flex For" branding message for existing and potential Amazon Flex delivery partners (DPs).
I cannot publish the findings and insights of this study due to confidentiality of this study. However, provided below are the process and research methods used for this study.
Helped design the study, interview protocol, and survey; observed and took notes of interviews and usability studies; performed quantitative and qualitative analyses; helped write the final report and presented to key stakeholders and leadership.
We conducted three different studies to derive meaningful insights:
- Interviews with Amazon Flex's existing DPs, and novices (individuals who have little to no knowledge of Amazon Flex)
- Usability tests of the website and app (with both DPs and novice users)
- A survey that was deployed to DPs nationally
Interviews and Usability Tests with DPs and Novices
The interviews and usability tests were conducted with 7 existing DPs to understand their perception of the website content, the “I Flex For” branded videos, and the effectiveness of the FAQs in addressing their information needs. DPs were chosen to interview based on the types of packages they delivered (Fresh (groceries), Amazon packages (AMZL), or Prime Now packages).
Existing DPs tended to already have more knowledge of the terms and processes through experience. To test the content, branding, and navigation of the website and app from the perspective of a potential DP, we interviewed 5 novice users (Amazon summer interns who had no knowledge of Amazon Flex). Interns were chosen as a novice group due to ease of access, short timeline, and budget constraints. Novice interviewees were chosen based on their answers to a brief survey on their level of knowledge of Amazon Flex.
Methods - Interview and Usability Test
DPs were initially interviewed to understand their history as Amazon Flex drivers, their motivations around driving for Amazon Flex, as well as key information pain points they encountered as DPs. Next, users were asked to freely explore the Amazon Flex website, and talk through what they saw and how it influenced them and their experience. Questions asked tried to focus on the DPs' perceptions and thoughts around the effectiveness and influence of "I Flex For" videos, the benefits of Amazon Flex, and if the website provided them with helpful and relevant information needed to sign up and perform the job of a DP. Next, drawing focus to test the effectiveness of the FAQs, users were tasked with navigating to the FAQs, and asked to read through the updated content for each of the different categories - Scheduling, Payment Process, Getting Started etc. Pain points in navigating were observed and recorded, and questions asked revolved around comprehension of the content and visuals. Both the interviews and usability tests recorded the face of the user, the computer screen, and the app, in order to capture capture facial expressions and non verbal cues, in addition to what was on the screen.
After looking at the website, users were then asked to use the Amazon Flex app mainly to navigate to the FAQs. Users were given 5 different tasks around finding specific pieces of information in the FAQs on the app, and were observed as they used the app and talked through their actions and thought processes. It was important for us to test the usability of both the app and the website to test navigation, and reveal any pain points or confusion between the two devices and interfaces.
The only difference in methods between the existing DPs and novice users was the focus on comprehension of the content on website as a whole, as well as the FAQs with the novice users. Since the website was initially designed to help new and potential DPs, we wanted to ensure that the information provided was useful, and could be understood by someone who had little to no knowledge of Amazon Flex.
Based on information collected in the usability tests, we wanted to get a broader understanding of these behaviors. In order to collect data at a scale, we deployed a survey to thousands of DPs all over the country to gather data of their website usage behavior with regards to FAQs, “I Flex For” videos, and the perception of the overall message and value proposition. This data was triangulated with the qualitative data collected through usability tests and interviews, leading us to some valuable insights.
This study revealed key insights around the purpose of the website from the DP perspective, how and why they visited the website in the first place, and the role of the "I Flex For" message to DPs. This study disproved many assumptions the team had, and shed light on actual behavior and perceptions, which led to discussions around several design, content, and brand changes for the website and the app. The insights and recommendations provided from these studies have been assigned owners, and are currently in the process of being incorporated to the Amazon Flex website and app.