| Page Properties | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Document summary Research for SUS and gaining quick wins
This research summary presents the results of the usability testing conducted on the sales network tools.
Objectives
As part of our ongoing efforts to improve the user experience and align our products with strategic goals, we have identified the System Usability Scale (SUS) score as a key performance indicator (KPI) for user satisfaction. Additionally, we will be tracking two other critical usability metrics: time on task and success rate. These metrics will provide a comprehensive view of how well our product supports primary user goals across web and mobile platforms.
...
SUS score
Time on task
Success rate
Test plan
Test plan | https://docs.google.com/document/d/1BAltDfoqDzOkX4O4fPh0PQemcIkFQiZgjvZArWvVlS8/edit?usp=sharing Sales Network Tool Test Plan |
|---|
Core user journeys[insert text here]
Persona | Core journey | Rationale |
|---|---|---|
BRM & PRM (the primary point of contact within the sales network between business owners/marketers and the organization) | Onboarding After Receiving Activation LinkWhy this journey was selected + how you selected this journey | It is the first contact of the BRM with the platform (They wouldn’t be able to sell products without onboarding) |
Onboarding a Business Owner (A major part of why they use the platform, to onboard businesses) | Why this journey was selected + how you selected this journeyOne of the primary reasons for getting onboarded is to get more Business Owners(BOs) onboarded (Survey with users showed these core journey) | |
Selling and Assigning Cards to Business Owners and Customers + descriptionWhy this journey was selected + how you selected this journey | They sell cards to BOs and customers to increase market traction and ultimately their commissions | |
Monitoring Business Owners Performance and Placing PND on Non-performing Business Owners + descriptionWhy this journey was selected + how you selected this journey | They monitor BOs to make sure they keep up to the expected target | |
Taking Courses via Dashboard + descriptionWhy this journey was selected + how you selected this journey | They take courses to stay in tune and updated with the latest so as to perform optimally | |
BRM (the primary point of contact within the sales network between business owners/marketers and the organization) | Ability to Cash Out Commissions + descriptionWhy this journey was selected + how you selected this journey | BRMs need to get their commissions/rewards for personal consumption |
SC & SPO (those who oversee the main contact within the sales network between business owners/marketers and the organization, specifically the managers of BRM/PRM) | Initiating the Onboarding Process of a BRM + descriptionWhy this journey was selected + how you selected this journey | They are to initiate the onboarding process of BRMs that will perform and increase market traction |
Monitoring BRM/PRM Performance and Placing PND on Non-performing BRMs/PRMs | They monitor BRMs/PRMs + descriptionto make sure they keep up to the expected target | |
Ability to Cash Out Commissions + description | They need to get their commissions/rewards for personal consumption | |
Selling Cards to BRMs | They sell cards to BRMs/PRMs to increase market traction and ultimately their commissions | |
SC (those who oversee the main contact within the sales network between business owners/marketers and the organization, specifically the managers of BRM/PRM) | Assigning POS to BRMs | SCs are to assign POS terminals to BRMs |
Test environment
The tests were unmoderated, we allowed the users navigate these journeys themselves, remote and recorded. The tests were done with the present working platforms (mobile and web). Some technical issues didn’t allow for the use of test environment and prototypes.
Tasks
| Note |
|---|
Delete this info panel, this is only for guidance. Provide details on the tasks participants were asked to complete. Ensure tasks align with real-world scenarios that users of the product would encounter. |
[insert text here]
Core journey | Tasks | |
|---|---|---|
View history & print receipt | A customer that recently made a transaction of N15,000.00, has come to ask you for a re-print of that receipt. Proceed to give the receipt Note: imagine this to be a POS terminal | |
[journey name] | [journey name]Core journey | Tasks |
Onboarding After Receiving Activation Link | As BRM/PRM, after receiving the activation link via mail, follow the flow to be onboarded | |
Onboarding a Business Owner (A major part of why they use the platform, to onboard businesses) | As a BRM/PRM, from your dashboard onboard a Business Owner/Personal Customer | |
Selling and Assigning Cards to Business Owners and Customers | As a BRM/PRM, from your dashboard, sell and assign cards to your BOs and customers | |
Monitoring Business Owners Performance and Placing PND on Non-performing Business Owners | As a BRM/PRM, from your dashboard, monitor the performance of your downlines and place a PND on non performing downlines | |
Taking Courses via Dashboard | As a BRM/PRM, take courses from the dashboard | |
Ability to Cash Out Commissions | As a BRM, show how you cashout your commissions | |
Initiating the Onboarding Process of a BRM | SC/SPO should log in and initiate the onboarding process of a new BRM/PRM | |
Monitoring BRM/PRM Performance and Placing PND on Non-performing BRMs/PRMs | SC/SPO should monitor the performance of downlines and place a PND on non performing downlines | |
Ability to Cash Out Commissions | SC/SPO should follow the dashboard on how to cashout their commissions | |
Selling Cards to BRMs | SC should from their dashboard, sell cards to your BRMs | |
Assigning POS to BRMs | SC should from their dashboard, assign POS terminals to BRMs |
Participants
Participant information
Number of participants | Indicate the total number of users involved (5-8 participants per journey). | |
|---|---|---|
Demographics | Include relevant demographic information such as age range, gender, location, experience level, job role, familiarity with the product, or other factors depending on the target audienceDemographics | The participants, ranging in age from 25 to 40, include both male and female individuals. The group consists of a combination of new partners unfamiliar with the processes and existing users. |
Selection criteria | Describe the criteria used for selecting participants. Were they existing users, new users, or a mix of both? |
Participant profiles
...
Delete this info panel, this is only for guidance.
...
Partners selected for the test were both new and existing users. |
Participant profiles
Participant A | ChineduComfort |
|---|---|
Age | 3525 |
Occupation | SME owner (fashion retail) Personal Banking Relationship Manager |
Tech savviness | Moderate |
Fintech usage | Uses mobile banking apps (GTBank, Access Bank) for daily transactions and a POS terminal for customer payments. Prefers platforms that offer quick, reliable transfers and clear transaction histories. |
Primary devices | Android phone (Tecno Camon 16Samsung) |
Key insights | Chinedu heavily relies on his POS terminal for business operations and uses fintech apps to track transactions. He Comfort is a new user who was recently onboarded to work as a PRM. She is often frustrated by network downtime with POS systems and values transparency in fees. Easy-to-read transaction reports and the ability to quickly reconcile daily sales are critical for his business. and error messages during onboarding process of a marketer. |
Participant B | HalimaFrancis |
|---|---|
Age | 27 28 |
Occupation | Market traderBusiness Relationship Manager |
Tech savviness | LowHigh |
Fintech usage | Uses mobile money apps like OPay and PalmPay for personal savings, transfers, and accepting payments from customers. Not particularly tech-savvy, but uses these services due to their ease of access and lower transaction fees compared to traditional banks. |
Primary devices | Android phone (Infinix Hot 10) |
Key insights | Halima needs simple, straightforward fintech solutions with minimal steps to complete transactions. She values reliability, especially when receiving payments from customers. She finds features like SMS confirmations and immediate access to funds critical for trust in the service.Francis interacts with business owners on a daily basis and requires easy and smooth interface in order to adequately manage his downlines |
Participant C | TundeRabiu |
|---|---|
Age | 40 |
Occupation | Corporate accountant (mid-sized firm) State Coordinator |
Tech savviness | High |
Fintech usage | Uses digital banking platforms and business fintech solutions for managing payroll, supplier payments, and invoicing. Regularly interacts with multiple banking platforms for business transactions. |
Primary devices | Windows laptop, iPhone 13 |
Key insights | Tunde requires fintech solutions that streamline business transactions and offer integration with accounting software. He values features like automated invoicing, bulk payments, and detailed reporting, but he finds it frustrating when banking platforms lack API integrations with their business software. Efficiency and robust security features are critical for business operations.Rabiu manages a network of BRMs and is particular about tracking performance |
Participant D | AmakaGabriel |
|---|---|
Age | 2231 |
Occupation | NYSC corp member (service year) State Product Officer |
Tech savviness | High |
Fintech usage | Primarily uses personal banking apps like Kuda and ALAT for saving, budgeting, and personal transfers. Also utilises fintech apps for peer-to-peer transfers and mobile payments. |
Primary devices | Android phone (Samsung A32) |
Key insights | Amaka Gabriel prefers fintech apps that offer budgeting tools and savings features. She values gamification and ease for managing his network of PRMs. He values proper reporting and the rewards systems, such as cashbacks commissions and referral bonuses, to increase engagement. Her His main frustration comes from long transaction processing times or unexpected app downtimes. Security and user-friendly onboarding are important factors for her trust in the app. |
Participant D
Babatunde
Age
50
Occupation
POS operator
Tech savviness
Moderate
Fintech usage
Relies heavily on mobile money apps like Paga and Moniepoint to run his POS business, facilitating cash withdrawals, deposits, and transfers for customers. He also uses traditional banking apps for reconciling the daily balance.
Primary devices
Android phone (Itel A56)
Key insights
Babatunde’s biggest concern is the reliability of the fintech app or platform, especially in high-traffic areas. He prefers platforms with low transaction failure rates and quick settlement times. Instant customer support is crucial when POS systems encounter issues, and he values platforms with lower transaction fees.
Recruitment process
...
inconsistent reporting and error blockers while using the tool. |
Recruitment process
Participants were selected based on
...
their partner user type to ensure representation of the
...
Sales Network.
Recruitment channels included
...
customer database
...
Screeners were used to ensure participants met the following criteria:
- regularly interact with [specific technology/product. type]
- other criteria related to the specific test such as being a current customer, not being a customer, using a competitor’s product, etc.
...
and social media groups.
Screeners implemented include:
Participants regularly interact with the sales network tools.
Participants are current partners.
Usability metrics and results
Success rate definition
...
Delete this info panel, this is only for guidance.
Success criteria:
A task is considered successful if the participant completes it as intended within the defined parameters, without assistance, and within a reasonable time frame.
Define clear success criteria for each task, such as reaching a specific page, completing a transaction, or filling out a form correctly.
Measurement:
...
SUS score
SUS is a standardised questionnaire used to measure the usability of a product, using a 10 questions 5-point scale to generate a score between 0 and 100, with higher indicating higher usability.
...
The time it takes participants to complete each journey. Shorter times generally indicate better usability. This metric helps to measure efficiency.
...
.
Onboarding After Receiving Activation Link: users should/are expected to complete this journey within 10 minutes.
Onboarding a Business Owner: users should/are expected to complete this journey within 4 minutes.
Selling and Assigning Cards to Business Owners and Customers: users should/are expected to complete this journey within X 3 minutes.[insert journey name]
Monitoring Business Owners Performance and Placing PND on Non-performing Business Owners: users should/are expected to complete this journey within 2 minutes.
Taking Courses via Dashboard: users should/are expected to complete this journey within X minutes.[insert journey name]2 minutes.
Ability to Cash Out Commissions: users should/are expected to complete this journey within 1 minute.
Initiating the Onboarding Process of a BRM: users should/are expected to complete this journey within 6 minutes.
Monitoring BRM/PRM Performance and Placing PND on Non-performing BRMs/PRMs: users should/are expected to complete this journey within X 3 minutes.
[insert journey name]Ability to Cash Out Commissions: users should/are expected to complete this journey within 1 minute.
Selling Cards to BRMs: users should/are expected to complete this journey within X 3 minutes.[insert
journey name]Assigning POS to BRMs: users should/are expected to complete this journey within X 4 minutes.
Success rate
[designer to define success for each journey]
[insert journey name]Onboarding After Receiving Activation Link: this journey is considered successful when BRM/PRM is able to get to the dashboard after validation.
Onboarding a Business Owner/Customer: this journey is considered successful when BRM/PRM is able to onboard a business owner/customer.
Selling and Assigning Cards to Business Owners and Customers: this journey is considered successful when BRM/PRM is able to sell and assign cards on the platform.
Monitoring Business Owners Performance and Placing PND on Non-performing Business Owners: this journey is considered successful when [specify].[insert journey name]when BRM/PRM is able to view the performance of downlines and place PND when necessary.
Taking Courses via Dashboard: this journey is considered successful when BRM/PRM is able to take a course.
Ability to Cash Out Commissions: this journey is considered successful when [specify].[insert journey name]BRM is able to cashout successfully.
Initiating the Onboarding Process of a BRM: this journey is considered successful when SC is able to raise a request for a BRM onboarding successfully.
Monitoring BRM/PRM Performance and Placing PND on Non-performing BRMs/PRMs: this journey is considered successful when [specify].[insert journey name]SC/SPO is able to view the performance of downlines and place PND when necessary.
Ability to Cash Out Commissions: this journey is considered successful when BRM is able to cashout successfully.
Selling Cards to BRMs: this journey is considered successful when [specify].[insert journey name]SC/SPO is able to sell cards on the platform.
Assigning POS to BRMs: this journey is considered successful when [specify]SC is able to successfully assign a terminal/POS to a BRM.
Results
...
Delete this info panel, this is only for guidance.
...
| Panel | ||
|---|---|---|
| ||
SUS score The average SUS score for SC/SPO was 79.9, this score falls within the industry benchmark. Analysis: the SUS score reveals positive aspects of user satisfaction, particularly related to how users use the platform for carrying out day to day activities. Analysis: the SUS score reveals [ positive /negative] aspects of user satisfaction, particularly related to [insert key findings]to how users use the platform for carrying out day to day activities. Time on task The average time to complete key tasks was [insert time]4 minutes. Analysis: some tasks such as [task 1] took longer than anticipated due to [insert reason], indicating potential usability challengestook longer to finish mainly because of technical issues such as API failure and we had to redo some tasks. Success rate The overall success rate was [insert percentage]72%. This indicates that [insert percentage of participants] some users were able to successfully complete the core tasks without assistance. Analysis: certain tasks such as [specific task] had a lower success rate due to [insert reasons], while others were completed with high efficiencythat had a bit of technical know how took more time such as assigning terminals. |
Core journey | SUS score | Time on task | Success rate | Comments |
|---|---|---|---|---|
[journey name] | e.g. 60 | 1m 30s | % | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey | [journey name] | Brief key feedback or observation for this journey |
Post-test qualitative questionnaire feedback
...
Delete this info panel, this is only for guidance.
Post-test questionnaire:
After completing the tasks, participants will be asked additional targeted questions to gather specific feedback on their experience. These questions will be presented BEFORE the SUS questionnaire, for example:
Was there anything confusing or frustrating about the sign-up process?
How easy or difficult did you find the sign-up process?
Did you encounter any difficulties in reading or understanding the text or labels during the tasks?
Were there any features or elements that stood out as particularly helpful or problematic?
Did you experience any challenges with the colour scheme or visual elements?
Do you have any additional comments or suggestions for improving this experience?
...
Onboarding After Receiving Activation Link | 75 | 09m 24s | 70% | Drop off from API failure (BVN validation) |
|---|---|---|---|---|
Onboarding a Business Owner (A major part of why they use the platform, to onboard businesses) | 42.5 | 04m 17s | 85% | |
Selling and Assigning Cards to Business Owners and Customers | 90 | 04m 08s | 79.5% | Experience can be better |
Monitoring Business Owners Performance and Placing PND on Non-performing Business Owners | 50 | 09m 13s | 75% | There are metrics they want displayed on the table shown |
Taking Courses via Dashboard | 100 | 01m 12s | 100% | |
Ability to Cash Out Commissions | 80 | 0m 34s | 100% | |
Initiating the Onboarding Process of a BRM | 90 | 10m | 90% | |
Monitoring BRM/PRM Performance and Placing PND on Non-performing BRMs/PRMs | 57.5 | 03m 08s | 89% | There are metrics they want displayed on the table shown |
Ability to Cash Out Commissions | 60 | 01m 29s | 100% | |
Selling Cards to BRMs | 57.5 | 03m 34s | 68% | |
Assigning POS to BRMs | 73.5 | 05m 57s | 56% |
Post-test qualitative questionnaire feedback
Participants provided additional insights through open-ended post-test questions:
Common pain points | List any recurring issues, frustrations, or confusion that participants mentionedCertain data they need to be able to be more productive isn’t being displayed and low internet bandwidth won’t load pages properly. |
|---|---|
Positive feedback | Summarise any positive comments about the product or specific featuresSome features such as cashout and placing PND are easier to navigate. |
User suggestions | Insert suggestions for improvement or areas where users struggled but offered solutionsData required should be displayed and optimize performance of the platform. |
Recommendations for improvement
Based on the test findings, the following recommendations are proposed to improve the product’s usability:
Design changes
[Task/feature]: revise [UI component, workflow, etc.] to streamline the user journey and reduce time on task.
[Task/feature]: clarify instructions on [specific screen/interaction] to improve success rates.
Quick wins
Minor adjustments to [insert feature or task] that can be implemented without major development effort.
Long-term recommendations
Consider a more comprehensive redesign of [specific user journey], especially focusing on [insert issue]Remove the claim profile screen upon login for PRMs, as it is irrelevant to their partner type.
Change the landing screen for SPOs, replacing the Loans section with a more relevant option for their role.
Implement a unified user interface for SCs and SPOs to eliminate the use of the Legacy UI.
Revise and rename features on the PRMs dashboard to reflect their specific roles and responsibilities.
Update dashboard terminology to better align with partner roles, such as replacing "Add Aggregator" with "Add BRM/PRM."
Quick wins
Reevaluate the success and failure screen messages to ensure they clearly communicate the intended message and remove any unnecessary screens.
Long-term recommendations
Streamline the approval stages involved in onboarding BRMs/PRMs to improve efficiency.
Consider using a different provider for liveness and face verification, as the current provider, Metamap, requires users to turn their heads during the process.
Conclusion
Summary
The usability test revealed indicated that while the product performs adequately in most areas, there are specific pain points related to [specific feature]. By focusing on these areas, we can significantly improve the Sales Network Tools effectively support partners in managing their networks, several pain points need attention. These include lengthy processes, such as onboarding business owners, approval flows for BRM/PRM onboarding, concerns about the visibility of network performance, and frequent downtimes during critical tasks. Addressing these challenges will lead to significant improvements in the overall user experience.
Next Steps
...