Document summary
This document outlines a comprehensive plan/guide towards testing and conducting a System Usability Scale (SUS) for the current savings product which is essential to quantitatively assess its overall ease of use, identify specific usability issues, and gather user feedback to inform targeted improvements for a more intuitive user experience.
Objectives
As part of our ongoing efforts to improve the user experience and align our products with strategic goals, we have identified the System Usability Scale (SUS) score as a key performance indicator (KPI) for user satisfaction. Additionally, we will be tracking two other critical usability metrics: time on task and success rate. These metrics will provide a comprehensive view of how well our product supports primary user goals across web and mobile platforms.
The purpose of this usability test is to establish a usability benchmark for the product, which will serve as a foundation for future improvements.
Success metrics
SUS score
Time on task
Success rate
Test plan
Core user journeys
Personal savings
Persona | Core journey | Rationale |
---|---|---|
Keneth Etuk | Create Flexible Savings. | This is an important flow for user to be able to save, without this flow user cannot save or earn interest. |
Inemesit udo | Create Locked Savings. | This is an important flow for user to be able to save, without this flow user cannot save or earn interest. |
Anthony | Create Fixed Savings. | This is an important flow for user to be able to save, without this flow user cannot save or earn interest. |
Hussein | Top-up Savings. | This is an important flow for user to be able to boost their savings goals and interest earnings. |
Vincent | Withdraw from Savings. | This is an important flow for user to be able to transfer funds from their savings. |
Taiwo | Withdraw from Savings. | This is an important flow for user to be able to transfer funds from their savings. |
Abubakar | Manage Savings. | This is an important flow for user to be able to adjust, and optimize their savings and goals for effective financial planning. |
Favour | Cancel Savings. | This is an important flow for user to be able to close their savings and access remaining funds. |
Adesina | Break Savings. | This is an important flow for user to be able to withdraw from locked savings before maturity, often incurring a penalty. |
Maryam | Rollover Savings. | This is an important flow for user to be able to transfer funds from a maturing account to a new one. |
Business Savings
Persona | Core journey | Rationale |
---|---|---|
Hafsat | Create Flexible Savings. | This is an important flow for user to be able to save, without this flow user cannot save or earn interest. |
Michael | Create Locked Savings. | This is an important flow for user to be able to save, without this flow user cannot save or earn interest. |
Mfonido | Create Fixed Savings. | This is an important flow for user to be able to save, without this flow user cannot save or earn interest. |
Joseph | Top-up Savings. | This is an important flow for user to be able to boost their savings goals and interest earnings. |
Daniel | Withdraw from Savings. | This is an important flow for user to be able to transfer funds from their savings. |
Bola xchange | Withdraw from Savings. | This is an important flow for user to be able to transfer funds from their savings. |
Abubakar | Manage Savings. | This is an important flow for user to be able to adjust, and optimize their savings and goals for effective financial planning. |
Haminat | Cancel Savings. | This is an important flow for user to be able to close their savings and access remaining funds. |
Adesina | Break Savings. | This is an important flow for user to be able to withdraw from locked savings before maturity, often incurring a penalty. |
Maryam | Rollover Savings. | This is an important flow for user to be able to transfer funds from a maturing account to a new one. |
Adetunji Kabir | Create SAYT. | This is an important flow for user to be able to transfer funds from a maturing account to a new one. |
AOL lukash innovation | Withdraw from SAYT. | This is an important flow for user to be able to transfer funds from a maturing account to a new one. |
Labake Ayeni | Manage SAYT. | This is an important flow for user to be able to transfer funds from a maturing account to a new one. |
Test environment
Interactive figma prototypes of the product was the product was tested on mobile and desktop devices in a remote and unmoderated setting. However testers had the liberty to ask questions when required.
Tasks
Core journey | Tasks |
---|---|
Create flexible savings | A user with interest of creating a flexible savings plan, has to navigate to savings sections, select a flexible savings, Enter details of savings, review terms and condition and then create plan. |
Create locked savings | A user with interest of creating a locked savings plan, has to navigate to savings sections, select a locked savings, Enter details of savings, review terms and condition and then create plan. |
Create fixed savings | A user with interest of creating a fixed savings plan, has to navigate to savings sections, select a fixed savings, Enter details of savings, review terms and condition and then create plan. |
Topup savings | A user with interest of toping up a savings plan, has to navigate to savings sections, select savings plan, click on topup and enter amount, enter transaction pin and confirm. |
Withdraw from savings | A user with interest of withdrawing from a savings plan, has to navigate to savings sections, select savings plan, click on withdraw and enter amount, enter transaction pin and confirm. |
Manage savings | A user with interest of adjusting a savings plan, has to navigate to savings sections, select savings plan, click on manage and update plan and confirm. |
Cancel flexible savings | A user with interest of cancelling a flexible savings plan, has to navigate to savings sections, select savings plan, click on the kebab menu, click on cancel flexible plan, enter transaction pin and confirm. |
Break Savings | A user with interest of breaking a savings plan, has to navigate to savings sections, select savings plan, click on the kebab menu, click on break savings, select reason of breakage, enter transaction pin and confirm. |
Create SAYT | A user with interest of creating a save as you transact savings plan, has to navigate to savings sections, select SAYT savings, Enter details of savings, review terms and condition and then create plan. |
Withdraw from SAYT | A user with interest of withdrawing from SAYT plan, has to navigate to savings sections, select savings plan, click on withdraw, enter transaction pin and confirm. |
Manage SAYT | A user with interest of adjusting a SAYT plan, has to navigate to savings sections, select savings plan, click on settings button, update plan and confirm. |
Participants
Participant information
Number of participants |
|
---|---|
Demographics |
|
Selection criteria |
|
Participant profiles
Delete this info panel, this is only for guidance.
If applicable (otherwise delete section), provide a few short participant profiles to offer context - see examples below.
Participant A | Chinedu |
---|---|
Age | 35 |
Occupation | SME owner (fashion retail) |
Tech savviness | Moderate |
Fintech usage | Uses mobile banking apps (GTBank, Access Bank) for daily transactions and a POS terminal for customer payments. Prefers platforms that offer quick, reliable transfers and clear transaction histories. |
Primary devices | Android phone (Tecno Camon 16) |
Key insights | Chinedu heavily relies on his POS terminal for business operations and uses fintech apps to track transactions. He is often frustrated by network downtime with POS systems and values transparency in fees. Easy-to-read transaction reports and the ability to quickly reconcile daily sales are critical for his business. |
Participant B | Halima |
---|---|
Age | 27 |
Occupation | Market trader |
Tech savviness | Low |
Fintech usage | Uses mobile money apps like OPay and PalmPay for personal savings, transfers, and accepting payments from customers. Not particularly tech-savvy, but uses these services due to their ease of access and lower transaction fees compared to traditional banks. |
Primary devices | Android phone (Infinix Hot 10) |
Key insights | Halima needs simple, straightforward fintech solutions with minimal steps to complete transactions. She values reliability, especially when receiving payments from customers. She finds features like SMS confirmations and immediate access to funds critical for trust in the service. |
Participant C | Tunde |
---|---|
Age | 40 |
Occupation | Corporate accountant (mid-sized firm) |
Tech savviness | High |
Fintech usage | Uses digital banking platforms and business fintech solutions for managing payroll, supplier payments, and invoicing. Regularly interacts with multiple banking platforms for business transactions. |
Primary devices | Windows laptop, iPhone 13 |
Key insights | Tunde requires fintech solutions that streamline business transactions and offer integration with accounting software. He values features like automated invoicing, bulk payments, and detailed reporting, but he finds it frustrating when banking platforms lack API integrations with their business software. Efficiency and robust security features are critical for business operations. |
Participant D | Amaka |
---|---|
Age | 22 |
Occupation | NYSC corp member (service year) |
Tech savviness | High |
Fintech usage | Primarily uses personal banking apps like Kuda and ALAT for saving, budgeting, and personal transfers. Also utilises fintech apps for peer-to-peer transfers and mobile payments. |
Primary devices | Android phone (Samsung A32) |
Key insights | Amaka prefers fintech apps that offer budgeting tools and savings features. She values gamification and rewards systems, such as cashbacks and referral bonuses, to increase engagement. Her main frustration comes from long transaction processing times or unexpected app downtimes. Security and user-friendly onboarding are important factors for her trust in the app. |
Participant D | Babatunde |
---|---|
Age | 50 |
Occupation | POS operator |
Tech savviness | Moderate |
Fintech usage | Relies heavily on mobile money apps like Paga and Moniepoint to run his POS business, facilitating cash withdrawals, deposits, and transfers for customers. He also uses traditional banking apps for reconciling the daily balance. |
Primary devices | Android phone (Itel A56) |
Key insights | Babatunde’s biggest concern is the reliability of the fintech app or platform, especially in high-traffic areas. He prefers platforms with low transaction failure rates and quick settlement times. Instant customer support is crucial when POS systems encounter issues, and he values platforms with lower transaction fees. |
Recruitment process
Delete this info panel, this is only for guidance.
Participants were selected based on specific demographic criteria to ensure representation of the target audience.
Recruitment channels included [insert recruitment methods: online panels, customer database, etc.].
Screeners were used to ensure participants met the following criteria:
- regularly interact with [specific technology/product. type]
- other criteria related to the specific test such as being a current customer, not being a customer, using a competitor’s product, etc.
Our target
Usability metrics and results
Success rate definition
SUS score
SUS is a standardised questionnaire used to measure the usability of a product, using a 10 questions 5-point scale to generate a score between 0 and 100, with higher indicating higher usability.
The average SUS score is typically 68. A score of 68 or higher is generally considered acceptable or better than average. Above 70 is good, and above 80 is excellent.
Time on task
The time it takes participants to complete each journey. Shorter times generally indicate better usability. This metric helps to measure efficiency.
Personal savings
Create flexible savings: Users are expected to complete this journey within 3 minutes.
Create locked savings: Users are expected to complete this journey within 3 minutes.
Create fixed savings: Users are expected to complete this journey within 2 minutes.
Break savings plan: Users are expected to complete this journey within 2 minutes.
Cancel savings plan: Users are expected to complete this journey within 2 minutes.
Top up savings: Users are expected to complete this journey within 2 minutes.
Withdraw from savings: Users are expected to complete this journey within 2 minutes.
Manage savings: Users are expected to complete this journey within 2 minutes.
Rollover savings: Users are expected to complete this journey within 2 minutes.
Business savings
Create flexible savings: Users are expected to complete this journey within 3 minutes.
Create locked savings: Users are expected to complete this journey within 3 minutes.
Create fixed savings: Users are expected to complete this journey within 2 minutes.
Break savings plan: Users are expected to complete this journey within 2 minutes.
Cancel savings plan: Users are expected to complete this journey within 2 minutes.
Top up savings: Users are expected to complete this journey within 2 minutes.
Withdraw from savings: Users are expected to complete this journey within 2 minutes.
Manage savings: Users are expected to complete this journey within 2 minutes.
Rollover savings: Users are expected to complete this journey within 2 minutes.
Create SAYT plan: Users are expected to complete this journey within 2 minutes.
Pause SAYT plan: Users are expected to complete this journey within 1 minutes.
Change SAYT withdrawal frequency: Users are expected to complete this journey within 1 minutes.
Withdraw from SAYT savings: Users are expected to complete this journey within 2 minutes.
Success rate
Personal savings
Create flexible savings: this journey is considered successful when success rate is above 68%.
Create locked savings: this journey is considered successful when success rate is above 68%
Create fixed savings: this journey is considered successful when success rate is above 68%
Break savings plan: this journey is considered successful when success rate is above 68%
Cancel savings plan: this journey is considered successful when success rate is above 68%
Top up savings: this journey is considered successful when success rate is above 68%
Withdraw from savings: this journey is considered successful when success rate is above 68%
Manage savings: this journey is considered successful when success rate is above 68%
Rollover savings: this journey is considered successful when success rate is above 68%
Business savings
Create flexible savings: this journey is considered successful when success rate is above 68%
Create locked savings: this journey is considered successful when success rate is above 68%
Create fixed savings: this journey is considered successful when success rate is above 68%
Break savings plan: this journey is considered successful when success rate is above 68%
Cancel savings plan: this journey is considered successful when success rate is above 68%
Top up savings: this journey is considered successful when success rate is above 68%
Withdraw from savings: this journey is considered successful when success rate is above 68%
Manage savings: this journey is considered successful when success rate is above 68%
Rollover savings: this journey is considered successful when success rate is above 68%
Create SAYT plan: this journey is considered successful when success rate is above 68%
Pause SAYT plan: this journey is considered successful when success rate is above 68%
Change SAYT withdrawal frequency: this journey is considered successful when success rate is above 68%
Withdraw from SAYT savings: this journey is considered successful when success rate is above 68%
Results
Personal savings
Core journey | SUS score | Time on task | Success rate | Comments |
---|---|---|---|---|
Rollover savings | 72 | 2m 57.4s | 83% | Journey seems easy to understand by users. |
Create fixed savings | 70.3 | 382m 56.5s | 71% | Result was skewed by a user that spent too much time answering SUS questionnaire. |
Create locked savings | 71.3 | 3m 25.1s | 60% | 40% of participant dropped off during prototype test |
Create flexible savings | 69.7 | 37m 20.8s | 60% | 33% of participant dropped off during prototype test |
Break savings plan | 70.7 | 5m 15.2s | 70% | The average time spent on the journey is a lot, a place to improve |
Cancel savings plan | 76.3 | 2m 45.3s | 60% | The low success rate may be attributed lots of drop-off at different points during the test. |
Top up savings | 78.5 | 1m 29.3s | 71% | flow was simple and straightforward based on feedback |
Withdraw from savings | 67.1 | 2m 39.8s | 88% | High success rate, but SUS score suggests journey can be improved |
Manage savings | 75.5 | 2m 7.3s | 83% | The high success rate and SUS score suggests journey was easy and straightforward for users to achieve their goals |
Business savings
Core journey | SUS score | Time on task | Success rate | Comments |
---|---|---|---|---|
Rollover savings | 70 | 1m 35.6s | 75% | Flow looking straightforward and easy for users based on feedback |
Create fixed savings | 78 | 1m 51.4s | 83% | Journey seems smooth, no negative feedback |
Create locked savings | 72.9 | 5m 18.1s | 75% | Journey was smooth, but the time on task can be improved |
Break savings plan | 77.1 | 1m 39.7s | 100% | Journey seems smooth, no negative feedback |
Create flexible savings | 75.8 | 102m 32.7s | 71% | Some testers were stuck at some point during the test, leading to them spending too much time to eventually achieve the flow |
Cancel savings plan | 85.5 | 1m 26.9s | 100% | Journey was easy for users, no negative feedback |
Top up savings | 78.1 | 2m 1.9s | 100% | Journey was easy for users, no negative feedback |
Withdraw from fixed savings | 84.2 | 1m 30.6s | 67% | Journey seems smooths, drop offs at the start and prototype sections highly skewed the results for the success rate |
Withdraw from locked savings | 78.8 | 1m 12.1s | 86% | Flow looking straightforward and easy for users based on feedback |
Withdraw from flexible savings | 72.9 | 3m 59.3s | 75% | Journey was easy for users, no negative feedback |
Manage savings | 77.5 | 1m 23.2s | 100% | Journey was easy for users, no negative feedback |
Create SAYT plan | 76.4 | 2m 34.7s | 78% | Journey was easy for users, no negative feedback |
Pause SAYT plan | 72.5 | 32m 46.3s | 75% | A tester was stuck at the over section during the test, leading to them spending too much time to eventually complete the flow |
Change SAYT withdrawal frequency | 67.1 | 1m 56.0s | 100% | The poor SUS score indicates the journey needs to be improved |
Withdraw from SAYT savings | 85.5 | 1m 28.2s | 56% | The low success rate stems from high drop off rate after the instructions section of this test |
Post-test qualitative questionnaire feedback
Participants provided additional insights through open-ended post-test questions:
Common pain points | Some participants complained about difficulty finding some buttons, especially when it come to breaking or canceling a savings plan. Participants also mentioned that the “copy” used in some places were not clear enough. |
---|---|
Positive feedback | Some participants talked about how short the journeys are, they can create a savings plan very quickly, top up and withdraw from a plan quickly. |
User suggestions | A suggestion that stands out was the use of better “copy” to convey messages clearer |
Recommendations for improvement
Based on the test findings, the following recommendations are proposed to improve the product’s usability:
Design changes
Overall look and feel of the product needs to be improved to give the product better aesthetics which influences customer decision while using the product.
Declutter congested sections and components to help customers make decisions better and faster, which improve success rates.
Quick wins
Update the “copy” to covey information better..
Long-term recommendations
Consider a more comprehensive redesign of the create savings plan journey, especially focusing on reducing the number of steps needed to create a savings plan.
Conclusion
Summary
The usability test revealed that while the product performs adequately in most areas, there are specific pain points related to savings plan creation journey and copy used in overall in the product. By focusing on these areas, we can significantly improve the user experience.
Next Steps
The design team will implement the recommended changes and prepare for a follow-up usability test to measure improvements in SUS, time on task, and success rate.
Add Comment