HP Instant Ink’s Subscription Starter Kit:
End-to-End, Moderated Study
Case Study Summary:
This case study focuses on an end-to-end (E2E), moderated study conducted on HP Instant Ink's pilot Subscription Starter Kit (SSK), aimed at understanding its low conversion rate and high return rate. As the sole UX Researcher on the Instant Ink Design Team, I orchestrated an in-person study to evaluate users' experiences with the cartridge packaging and set up experience. This piece will cover the research challenge, approach, and the outcome of my work.
Impact: As a result of my E2E study, I uncovered key usability and comprehension insights in the SSK experience, establishing a baseline user experience that enabled my team to make data-driven design iterations and measured the impact of each change. This study provided a breakthrough in understanding users’ comprehension of Instant Ink, and their expectations when shopping for and replacing ink cartridges. In the second iteration of this study, we saw an 80% improvement in awareness of the SSK’s instruction card.
This case study is NDA protected so I won’t be going over the exact findings from my study.
HP Instant Ink Background
HP Instant Ink is a printing subscription service, where in exchange for limiting how much you print every month, HP will manage your ink supplies for you. As the sole UX Researcher on the Instant Ink Design Team, I played a crucial role by supporting various projects and spearheading multiple research initiatives.
What is the Subscription Starter Kit (SSK)?
HP Instant Ink's Subscription Starter Kit (SSK) is an ink cartridge kit customers can buy in brick-and-mortar stores that introduces customers to Instant Ink. The kit contained 2 Instant Ink cartridges and an instruction card that included a set-up PIN. Users must enroll into Instant Ink to use the cartridges, and must redeem the PIN or else they would have to buy another subscription on top of the SSK.
Although the SSK was available for purchase in several retail stores, no testing had been done prior to the product’s release. As a result, the SSK experienced a low Instant Ink conversion rate and a high return rate.
My design team and I were asked to support the SSK, and upon reviewing the low conversion and high return rates, we knew we needed to understand users’ first print experience with the SSK.
Front and back of the SSK cartridges
Research Goal
Our research goal was simple: What were users’ experience when trying to print with their SSK?
It is important to note that stakeholders and I agreed that purchase intent would be out of scope for this study. However participants can share it unprompted.
Instead, the study would focus explicitly on the box’s usability.
Research Approach
In order to understand users’ experience trying to set-up and print with their SSK cartridges, I decided to run an end-to-end, moderated study.
The Instant Ink SSK study consisted of 4 parts:
Part 1: Participant background interview
A brief interview was conducted to gain insights into participants’ current ink supply management practices, preferred stores, and their familiarity with retail setups.
This pre-interview serves as a primer, aiming to understand individual participants’ experiences and act as an ice breaker
Part 2: External Box Usability
This section aimed to learn how effective was the SSK box in communicating its contents, service offering, and how well did it set expectations for installing the Instant Ink cartridges. Participants interacted with both a control box and the prototype box (the SSK) and walked me through how they would approach these each of the boxes in a brick-and-mortar store.
I gave participants a control box to recall their typical in-store ink cartridge shopping experience. This was to ensure that the unique packaging design of the SSK, which might induce social-desirability bias, was considered in comparison.
Price points were withheld initially to minimize the focus on purchase intent, but shared after the first walkthrough to see how participants would interact differently.
Participants were quizzed on their recall of box details after the walkthrough so I could evaluate how much information they had absorbed.
I hypothesized that the amount of information participants absorbed “in store” would heavily impact their set-up and first print experience.
Part 3: Get to First Print
We knew from existing research that users typically set up (or replace) their ink cartridges when they need to print something out. Otherwise the cartridges will go into their supply drawer.
Participants were told they had purchased the SSK in-store to print out a birthday card. They were instructed to “do whatever you need to do'' to print out the card using the SSK that they had purchased.
Success was defined as successfully printing the birthday card while adhering to ALL provided instructions
The instructions were intentionally vague to avoid giving hints to participants and evaluate how effectively the SSK guided users
Instruction card in the SSK
Part 4: Post Test Comprehension Interview
After completing the usability test, the individual participant and I had a debrief interview to reflect on their experience.
The focus was on understanding whether participants grasped the concept of Instant Ink and its functionality throughout the study. This includes distinguishing between Instant Ink and the SSK offer, as well as comprehending the purpose of the SSK.
Furthermore, the debrief aimed to capture participants’ overall sentiments about the experience, allowing them to express any remaining questions or concerns they might have had.
Recruitment & Research Ops:
Recruitment:
For participant selection, I recruited 24 prospective Instant Ink customers who:
Mirror the age distribution of our current customer demographic
Own an HP printer
Felt at least somewhat positive about their household subscriptions
Have previous experience purchasing ink from brick-and-mortar stores.
This approach aligns with our current and prospective user base, while still taking into account a diverse range of perspectives.
Research Ops Execution:
Additionally, in order to facilitate this test, I was responsible for the following (Research) Operation tasks:
Budget Oversight: Ensuring the entire test budget remained below $15k, including participant recruitment fees, prototype expenses, and equipment room costs
Contract Management: Coordinating and supervising the Statement of Work contract negotiations between HP and our recruitment agency
Equipment Coordination:
Orchestrated the acquisition of necessary equipment to build the prototypes and establish the test environment
Build all prototypes and create an optimal test environment, which had to be set up and torn down daily
Ensured the availability of suitable recording equipment
Conducting multiple dry-runs of the test to verify all equipment was functioning properly
Privacy and Security: Developed 24 participant tester profiles that align with HP’s security policies while safeguarded participants’ PII
Managing participants: Ensured participants’ arrival at HP’s San Diego campus, guaranteeing their comfort during test sessions, while adhering to HP’s security protocols
Although setting up an in-person test is always challenging, it was not a new monster to me. A noteworthy milestone in my journey at HP was spearheading the first, in-person, moderated test since the COVID-19 pandemic, which established a replicable framework.
Analysis & Findings
To understand my 20 participants’ experience, I first defined the different touch points in the end-to-end test, which became the framework for my deductive coding. Then I reviewed themes surrounding comprehension and usability at each touchpoint, which included participants’ thoughts and behaviors as they proceeded with their assigned task.
My inductive coding focused on comprehension on Instant Ink (the subscription, rather than the box) and user sentiment as participants went about their task.
After the analysis, I then built out a user journey map based on the majority trends that I observed at each touch point. I also mapped each touchpoint to a sentiment level based on my participants’ sentiment.
Other critical, high impact findings that did not qualify as “majority trends” were also shared out, such as several users reaching squatter websites
To distill the data down to actionable, impactful insights, I compared the observed user journey line against the hypothesized journey line stakeholders originally believed customers would experience. This helped stakeholders understand where did the experience deviate furthest from the intended plan and what caused it to deviate. This also allowed us to align on which touchpoint was our top priority to address.
The test unveiled a frustrating holistic user experience in both usability and comprehension. Due to NDA, I will not be sharing the specifics of my findings here.
Impact & Moving Forward
While the findings may not have been the successes we hoped for, this test yielded several valuable wins.
First, it provided clear directions on enhancing the user experience, marking a significant breakthrough.
My hypothesis that the amount of information participants absorbed “in store” heavily impacted their set-up and first print experience was true. This turned out to be true. Iterations to the box’s designs were made based on my findings.
In the improved 2.0 iteration of the box and study, we had the box’s new design call out the instruction card on the front, and used a glue dot to stick the instruction card on the ink cartridges, aiming to increase awareness of the card. The adjustment successfully raised card awareness by 80%.
Another triumph was the establishment of an experience baseline to measure the impact of our design changes on the SSK, covering both physical packaging and the online Instant Ink enrollment experience.
And finally, beyond the tangible outcomes, this project was insanely fun and insightful for my team. It not only built trust among various business stakeholders but also served as a platform for testing and deriving clear, actionable insights for the SSK.