| Page Properties | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Document summary Insert brief summary here
This is a document containing the findings/results of the usability test for the cash management product carried out in September 2024.
Objectives
As part of our ongoing efforts to improve the user experience and align our products with strategic goals, we have identified the System Usability Scale (SUS) score as a key performance indicator (KPI) for user satisfaction. Additionally, we will be tracking two other critical usability metrics: time on task and success rate. These metrics will provide a comprehensive view of how well our product supports primary user goals across web and mobile platforms.
The purpose of this usability test is to The primary objective of this unmoderated usability test is to evaluate the product's usability, focusing on three key metrics: SUS score, time on task, and success rate. This test will establish a usability benchmark for the product , which will serve as a foundation for and inform future improvements.
Success metrics
SUS score
Time on task
Success rate
Test plan
...
Delete this info panel, this is only for guidance.
...
Test plan |
|---|
...
...
Test plan
...
Attach the original test plan for reference - insert link
Core user journeys
| Note |
|---|
Delete this info panel, this is only for guidance.
|
[insert text here]
Persona
Core journey
Rationale
Name of persona + description
Name of the core journey + description
Why this journey was selected + how you selected this journey
Name of persona + description
Name of the core journey + description
Why this journey was selected + how you selected this journey
Name of persona + description
Name of the core journey + description
Name of persona + description
Name of the core journey + description
Why this journey was selected + how you selected this journey
Name of persona + description
Name of the core journey + description
Why this journey was selected + how you selected this journey
Name of persona + description
Name of the core journey + description
Why this journey was selected + how you selected this journey
Name of persona + description
Name of the core journey + description
Why this journey was selected + how you selected this journey
Test environment
| Note |
|---|
Delete this info panel, this is only for guidance. Outline the approach you took - wether it was moderated or unmoderated test, remote or in-person, which platform it was performed on (mobile, web, POS), interactive prototypes or testing account, etc. |
[insert text here]
Tasks
| Note |
|---|
Delete this info panel, this is only for guidance. Provide details on the tasks participants were asked to complete. Ensure tasks align with real-world scenarios that users of the product would encounter. |
[insert text here]
...
Core journey
...
Tasks
...
View history & print receipt
...
A customer that recently made a transaction of N15,000.00, has come to ask you for a re-print of that receipt. Proceed to give the receipt
Note: imagine this to be a POS terminal
...
[journey name]
...
[journey name]
Participants
Participant information
...
Number of participants
...
Indicate the total number of users involved (5-8 participants per journey).
...
Demographics
...
Include relevant demographic information such as age range, gender, location, experience level, job role, familiarity with the product, or other factors depending on the target audience.
...
Selection criteria
...
Describe the criteria used for selecting participants. Were they existing users, new users, or a mix of both?
Participant profiles
| Note |
|---|
Delete this info panel, this is only for guidance. If applicable (otherwise delete section), provide a few short participant profiles to offer context - see examples below. |
...
Participant A
...
Chinedu
...
Age
...
35
...
Occupation
...
SME owner (fashion retail)
...
Tech savviness
...
Moderate
...
Fintech usage
...
Uses mobile banking apps (GTBank, Access Bank) for daily transactions and a POS terminal for customer payments. Prefers platforms that offer quick, reliable transfers and clear transaction histories.
...
Primary devices
...
Android phone (Tecno Camon 16)
...
Key insights
...
Chinedu heavily relies on his POS terminal for business operations and uses fintech apps to track transactions. He is often frustrated by network downtime with POS systems and values transparency in fees. Easy-to-read transaction reports and the ability to quickly reconcile daily sales are critical for his business.
...
Participant B
...
Halima
...
Age
...
27
...
Occupation
...
Market trader
...
Tech savviness
...
Low
...
Fintech usage
...
Uses mobile money apps like OPay and PalmPay for personal savings, transfers, and accepting payments from customers. Not particularly tech-savvy, but uses these services due to their ease of access and lower transaction fees compared to traditional banks.
...
Primary devices
...
Android phone (Infinix Hot 10)
...
Key insights
...
Halima needs simple, straightforward fintech solutions with minimal steps to complete transactions. She values reliability, especially when receiving payments from customers. She finds features like SMS confirmations and immediate access to funds critical for trust in the service.
...
Participant C
...
Tunde
...
Age
...
40
...
Occupation
...
Corporate accountant (mid-sized firm)
...
Tech savviness
...
High
...
Fintech usage
...
Uses digital banking platforms and business fintech solutions for managing payroll, supplier payments, and invoicing. Regularly interacts with multiple banking platforms for business transactions.
...
Primary devices
...
Windows laptop, iPhone 13
...
Key insights
...
Tunde requires fintech solutions that streamline business transactions and offer integration with accounting software. He values features like automated invoicing, bulk payments, and detailed reporting, but he finds it frustrating when banking platforms lack API integrations with their business software. Efficiency and robust security features are critical for business operations.
...
Participant D
...
Amaka
...
Age
...
22
...
Occupation
...
NYSC corp member (service year)
...
Tech savviness
...
High
...
Fintech usage
...
Primarily uses personal banking apps like Kuda and ALAT for saving, budgeting, and personal transfers. Also utilises fintech apps for peer-to-peer transfers and mobile payments.
...
Primary devices
...
Android phone (Samsung A32)
...
Key insights
...
Amaka prefers fintech apps that offer budgeting tools and savings features. She values gamification and rewards systems, such as cashbacks and referral bonuses, to increase engagement. Her main frustration comes from long transaction processing times or unexpected app downtimes. Security and user-friendly onboarding are important factors for her trust in the app.
...
Participant D
...
Babatunde
...
Age
...
50
...
Occupation
...
POS operator
...
Tech savviness
...
Moderate
...
Fintech usage
...
Relies heavily on mobile money apps like Paga and Moniepoint to run his POS business, facilitating cash withdrawals, deposits, and transfers for customers. He also uses traditional banking apps for reconciling the daily balance.
...
Primary devices
...
Android phone (Itel A56)
...
Key insights
...
Babatunde’s biggest concern is the reliability of the fintech app or platform, especially in high-traffic areas. He prefers platforms with low transaction failure rates and quick settlement times. Instant customer support is crucial when POS systems encounter issues, and he values platforms with lower transaction fees.
Recruitment process
| Note |
|---|
Delete this info panel, this is only for guidance.
|
[insert text here]
Usability metrics and results
Success rate definition
| Note |
|---|
Delete this info panel, this is only for guidance. Success criteria:
Measurement:
|
SUS score
SUS is a standardised questionnaire used to measure the usability of a product, using a 10 questions 5-point scale to generate a score between 0 and 100, with higher indicating higher usability.
The average SUS score is typically 68. A score of 68 or higher is generally considered acceptable or better than average. Above 70 is good, and above 80 is excellent.Test environment
A remote moderated test which was performed on mobile with Figma prototypes.
Tasks
Core journey | Tasks |
|---|---|
Cash Requester Onboarding | Imagine you are a POS agent who needs cash to perform daily transactions, Using this interface, register yourself as a cash requester. |
Request for cash | Imagine you are a POS agent who needs cash to perform daily transactions, Using this interface, request for cash and also assume that you have gotten to the pick up location so you can go through the process of picking up the cash. |
Cash Supplier Onboarding | Imagine you are a business owner who always has excess cash and you need the cash mopped up easily. Using this interface, register yourself as a cash supplier. |
Attend to the requester | Imagine the requester has gotten to your location to pick up cash. Using this interface, attend to the requester. |
Participants
Participant information
Number of participants | 7 participants per journey. |
|---|---|
Selection criteria | Participants were selected based on the familiarity with the manual cash mop up process to ensure they represent the target audience. |
Recruitment process
Recruitment channels: Whatsapp groups where the requester and the suppliers are currently using for the manual mop up process.
Usability metrics and results
Success rate definition
Cash requester onboarding: This task is successful if a user can onboard themselves as a cash requester. and get to the dashboard.
Request for cash and Pick it up: This task is successful if a user can request for cash and and as well go through the process of picking up the cash.
Cash supplier onboarding: This task is successful if a user can onboard themselves as a cash supplier and get to the dashboard.
Attend to the requester: This task is successful if a user can go through the mop up process and get to the transaction completion page.
SUS score
...
...
Time on task
The time it takes participants to complete each journey. Shorter times generally indicate better usability. This metric helps to measure efficiency.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
Success rate
[designer to define success for each journey]
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
Results
...
Delete this info panel, this is only for guidance.
SUS score
The average SUS score was [insert score], with [brief interpretation: e.g., this score falls within/above/below the industry benchmark].
Analysis: the SUS score reveals [positive/negative] aspects of user satisfaction, particularly related to [insert key findings].
Time on task
The average time to complete key tasks was [insert time].
Analysis: tasks such as [task 1] took longer than anticipated due to [insert reason], indicating potential usability challenges.
Success rate
The overall success rate was [insert percentage]. This indicates that [insert percentage of participants] were able to successfully complete the core tasks without assistance.
...
Cash requester onboarding: 54 seconds.
Request for cash: 4 minutes 19 seconds.
Cash supplier onboarding: 2 minutes 11 seconds
Attend to requester: 2 minutes 43 seconds.
The time taken criterion is not accurate because most participants paused to attend to customers, and internet issues occurred while they were testing, which prevented Useberry from loading tasks on time. Additionally, they were explaining the screens to us (moderators) to give us insight into their thought process, as it was a moderated test.
Success rate
Following the criteria listed above for success rate
Cash requester onboarding: Successful
Request for cash: Successful
Cash supplier onboarding: Successful
Attend to requester: Successful
Results
Core journey | SUS score | Time on task | Success rateResults/outcome | Comments |
|---|---|---|---|---|
[journey name] | e.g. 60 | 1m 30s | % | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey |
Post-test qualitative questionnaire feedback
| Note |
|---|
Delete this info panel, this is only for guidance. Post-test questionnaire: After completing the tasks, participants will be asked additional targeted questions to gather specific feedback on their experience. These questions will be presented BEFORE the SUS questionnaire, for example:
These questions will help uncover specific areas of concern or satisfaction that may not be fully captured by the SUS score alone. |
Cash requester onboarding | 86.5 | 54 seconds | Successful | |
|---|---|---|---|---|
Request for cash | 86.5 | 4 minutes 19 seconds. | Successful | |
Cash supplier onboarding | 87.5 | 2 minutes 11 seconds | Successful | |
Attend to the requester | 87.5 | 2 minutes 43 seconds. | Successful |
Post-test qualitative questionnaire feedback
Participants provided additional insights through open-ended post-test questions:
Common pain points | List any recurring issues, frustrations, or confusion that participants mentioned. |
|---|---|
Positive feedback | Summarise any positive comments about the product or specific features. |
User suggestions | Insert suggestions for improvement or areas where users struggled but offered solutionsThe product is slow {Useberry was taking time to load, hence the frustration and the need for this comment} |
User suggestions | They want a case where they can identify as both requester and supplier. |
Recommendations for improvement
Based on the test findings, the following recommendations are proposed to improve the product’s usability:
Design changes
[Task/feature]: revise [UI component, workflow, etc.] to streamline the user journey and reduce time on task.
[Task/feature]: clarify instructions on [specific screen/interaction] to improve success rates.
Quick wins
Minor adjustments to [insert feature or task] that can be implemented without major development effort.To fix the UX writings to communicate the processes or flows better
Long-term recommendations
Consider a more comprehensive redesign of [specific user journey], especially focusing on [insert issue]a scenario where a user is both a requester and a supplier.
Conclusion
Summary
The usability test revealed that while the product performs adequately in most areas, there are specific pain points related to [specific feature]. By focusing on these areas, we can significantly improve the user experiencemost of the users understand the ways to navigate through the flows, there is still a need to revisit the flow for when a requester gets to the supplier’s location and this can only be improved when we get accurate data or insights when the product launches a pilot test with real life data as some of these flows are still assumptions.
Next Steps
The design team will implement the recommended changes and prepare for a follow-up usability test to measure improvements in SUS, time on task, and success rate.
...