HP Instant Ink’s Subscription Starter Kit:
End-to-End, Moderated Study
Case Study Summary:
This case study focuses on an end-to-end (E2E), moderated study conducted on HP Instant Ink's pilot Subscription Starter Kit (SSK), aimed at understanding its low conversion rate and high return rate. As the sole UX Researcher on the Instant Ink Design Team, I orchestrated an in-person study to evaluate users' experiences with the cartridge packaging and set up experience. This piece will cover the research goal, recruitment process, methodology, and key findings.
Impact: As a result of my E2E study, I unveiled insights into critical usability and comprehension issues, informing iterative design enhancements and establishing metrics to measure their impact. This study marked a significant breakthrough in understanding user experiences and prioritizing design efforts effectively, ultimately contributing to the improvement of HP Instant Ink's user experience. In the second iteration of this study, there was an 80% improvement in the SSK’s instruction card.
This case study is NDA protected.
HP Instant Ink Background
HP Instant Ink is an as needed, subscription-based ink replenishment service designed for specific HP printers, providing customers with a convenient, unique, and cost-effective way to maintain a steady supply of ink. As the sole UX Researcher on the Instant Ink Design Team, I played a crucial role by supporting various projects and spearheading multiple research initiatives.
What is the Subscription Starter Kit (SSK)
HP Instant Ink's Subscription Starter Kit (SSK) is an ink cartridge kit customers can buy in brick-and-mortar stores that introduces them to the convenience and benefits of Instant Ink. Users need to enroll in Instant Ink before they can print from the cartridges. The SSK’s packaging and contents were redesigned and scheduled to be available in stores in fall of 2023, but no testing had been conducted on the new design and contents. As a result, the SSK experienced a low conversion rate and a high return rate.
My design team and I were asked to support the rollout of the new SSK, and upon reviewing the situation, decided that conducting an E2E, in-person, moderated study was needed to evaluate users’ experience of reaching their first print with their SSK cartridges.
Front and back of the SSK cartridges
Research Goal
Our research goal was simple: What is the usability of the SSK?
It is important to know that this test does not cover users’ purchase intent (although participants can share it unprompted).
Recruitment & Research Ops:
Recruitment:
For participant selection, I recruited 24 prospective Instant Ink customers who:
Mirror the age distribution of our current customer demographic
Own an HP printer
Felt at least somewhat positive about their household subscriptions
Have previous experience purchasing ink from brick-and-mortar stores.
This approach aligns with our current and prospective user base, while still taking into account a diverse range of perspectives.
Research Ops Execution:
Additionally, in order to facilitate this test, I was responsible for the following (Research) Operation tasks:
Budget Oversight: Ensuring the entire test budget remained below $15k, including participant recruitment fees, prototype expenses, and equipment room costs
Contract Management: Coordinating and supervising the Statement of Work contract negotiations between HP and our recruitment agency
Equipment Coordination:
Orchestrated the acquisition of necessary equipment to build the prototypes and establish the test environment
Build all prototypes and create an optimal test environment, which had to be set up and torn down daily
Ensured the availability of suitable recording equipment
Conducting multiple dry-runs of the test to verify all equipment was functioning properly
Privacy and Security: Developed 24 participant tester profiles that align with HP’s security policies while safeguarded participants’ PII
Managing participants: Ensured participants’ arrival at HP’s San Diego campus, guaranteeing their comfort during test sessions, while adhering to HP’s security protocols
Although setting up an in-person test is always challenging, it was not a new monster to me. A noteworthy milestone in my journey at HP was spearheading the first, in-person, moderated test since the COVID-19 pandemic, which established a replicable framework.
Methodology
The Instant Ink SSK study consisted of 4 parts:
Part 1: Participant background interview
A brief interview was conducted to gain insights into participants’ current ink supply management practices, preferred stores, and their familiarity with retail setups.
This pre-interview serves as a primer, aiming to understand individual participants’ experiences and act as an ice breaker
Part 2: External Box Usability
This section aimed to learn the effectiveness of the SSK box communicating its contents, service offering, and setting expectations for installing Instant Ink cartridges. Participants interacted with both a control box and the prototype box (the SSK) and walked me through how they would approach these boxes in a brick-and-mortar store. Price points were intentionally withheld until the initial walkthrough to minimize the focus on purchase intent. Additionally, participants were quizzed on their recall of box details.
Participants were given a control box to recall their typical in-store ink cartridge shopping experience. This was to ensure that the unique packaging design of the SSK, which might induce social-desirability bias, was considered in comparison.
Part 3: Get to First Print
Participants were told they had purchased the SSK in-store to print out a birthday card. They were instructed to “do whatever you need to do'' to print out the card using the SSK that they had purchased.
Success was defined as successfully printing the birthday card while adhering to ALL provided instructions
The instructions were intentionally vague to avoid giving hints to participants and evaluate how effectively the SSK guided users
Instruction card in the SSK
Part 4: Post Test Comprehension Interview
After completing the usability test, the individual participant and I had a debrief interview to reflect on their experience.
The focus was on understanding whether participants grasped the concept of Instant Ink and its functionality throughout the study. This includes distinguishing between Instant Ink and the SSK offer, as well as comprehending the purpose of the SSK.
Furthermore, the debrief aimed to capture participants’ overall sentiments about the experience, allowing them to express any remaining questions or concerns they might have had.
Findings
The 20 tests yielded numerous findings. To distill the data down to the actionable insights, I compared the hypothesized, user success journey line against the observed user journey line, based on trends identified during the study.
Other critical, high-impact findings that did not qualify as “majority trends" were also shared out.
The test unveiled a frustrating holistic user experience. Although the study aimed to assess the usability of the SSK box, it revealed significant challenges in both usability and comprehension. These insights shed light on the factors contributing to the low Instant Ink conversion rates from the SSKs and the high rate of SSK cartridge returns.
Due to NDA, I will not be sharing the specifics of my findings here.
Impact & Moving Forward
While the findings may not have been the successes we hoped for, this test yielded several valuable wins.
First, it provided clear directions on enhancing the user experience, marking a significant breakthrough. Despite prior iterations of the SSK box, this test was the first to comprehensively understand the holistic user experience, allowing us to prioritize our efforts effectively.
This understanding led to iterative designs for the SSK, encompassing enhancements in both its physical packaging and online experience. In the improved 2.0 iteration of the box and study, we introduced a glue dot to the instruction card on the ink cartridges, aiming to increase awareness of the card. The adjustment successfully raised card awareness by 80%. Currently a 3.0 box and test is in the planning stages.
Another triumph was the establishment of metrics to measure the impact of our design changes on the SSK, covering both physical packaging and the online Instant Ink enrollment experience. This study also introduced a control experience line.
And finally, beyond the tangible outcomes, this project was insanely fun and insightful for my team. It not only built trust among various business stakeholders but also served as a platform for testing and deriving clear, actionable insights for the SSK.