Onboarding SUS report

Objectives

As part of our ongoing efforts to improve the user experience and align our products with strategic goals, we have identified the System Usability Scale (SUS) score as a key performance indicator (KPI) for user satisfaction. Additionally, we will be tracking two other critical usability metrics: time on task and success rate. These metrics will provide a comprehensive view of how well our product supports primary user goals across web and mobile platforms.

The purpose of this usability test is to establish a usability benchmark for the product, which will serve as a foundation for future improvements.

Success metrics

• SUS score

• Time on task

• Success rate

Test plan

Core user journeys

Test environment

Platform: The tests were conducted on Useberry, as well as web dev/staging apps.

Method: Combination of in-person moderated and remote unmoderated tests. Interactive prototypes were set up on useberry for the unmoderated tests, as well as the SUS questionnaires. Web testing accounts were used for the in-person moderated tests.

Tasks

Participants

Participant information

Recruitment process

• Participants were selected based on their likelihood to interact with the flow on a day-to-day basis. For each flow, participant demographies represent the ideal users.

• Recruitment channels included: State Coordinators (SCs), State Product Officers (SPOs) and Business Relationship Managers (BRMs).

• Screeners implemented include:

  • Participants regularly interact with the onboarding flow

  • Participants have never used the onboarding flow

  • Participants are current customers

Usability metrics and results

Success rate definition

SUS score

SUS is a standardised questionnaire used to measure the usability of a product, using a 10 questions 5-point scale to generate a score between 0 and 100, with higher indicating higher usability.

The average SUS score is typically 68. A score of 68 or higher is generally considered acceptable or better than average. Above 70 is good, and above 80 is excellent.

Open

Time on task

- The time spent on each task is inconclusive because a lot of participant were multi tasking during this test

• Personal Onboarding: users should/are expected to complete this journey in 4 minutes.

• Business Onboarding: users should/are expected to complete this journey within 4 minutes.

• BRM/PRM onboarding BO: users should/are expected to complete this journey within 7 minutes.

• Marketer/PRM onboarding BO: users should/are expected to complete this journey within 4 minutes.

• Account upgrade up to level 3: users should/are expected to complete this journey within 4 minutes.

• Soft onboarding: users should/are expected to complete this journey within 1 minute.

• Account upgrade to registered business: users should/are expected to complete this journey within X minutes.

• Forgot passcode: users should/are expected to complete this journey within <1 minutes.

Success rate

• Personal Onboarding: this journey is considered successful when the personal customer has successfully created a personal account.

• Business Onboarding: this journey is considered successful when when the business owner has successfully created a business account.

• BRM/PRM onboarding BO: this journey is considered successful when when the BRM has successfully onboarded a Business Owner.

• Marketer/PRM onboarding customer: this journey is considered successful when when the Marketer/PRM has successfully onboarded a personal customer.

• Account upgrade: this journey is considered successful when the customer has successfully upgraded their account to a level 2 and 3 account.

• Soft onboarding: this journey is considered successful when the customer has successfully imported their existing business/ personal profile to their new personal/business account.

• Account upgrade to registered business: this journey is considered successful when the business owner has successfully upgraded their account to a level 4 account.

• Forgot passcode: this journey is considered successful when a personal customer or business owner has recovered the passcode to their account.
  
  

Results

Post-test qualitative questionnaire feedback

Participants provided additional insights through open-ended post-test questions:

Recommendations for improvement

Based on the test findings, the following recommendations are proposed to improve the product’s usability:

Design changes

• Getting rid of the CBN walkthrough screens or shorten it, because nobody reads it. (When i asked participants why they didn’t read it, they said it was too lengthy, and as long as we didn’t force them to read it, they were going to skip it.)

• Unifying the progress bars. It’s very inconsistent in the onboarding flow. Some parts of the flow have the progress sliders, while others have the progress steppers

Quick wins

• Minor adjustments to the photo capture screens. None of the participants read instructions during this test.

• Using visual instructions for the photo capture screens. Users don’t read instructions and they can’t get past photo capture.

• Reassess the profile creation success screens, as well as the KYC primers to communicate the right message, remove any unnecessary screens.

Long-term recommendations

• Improving the account upgrade process. Currently, when users upgrade their accounts, we show them that their account has been upgraded even when we’re still running some background checks. This leads to a lot of confusion and accounts being PND’d.

Conclusion

Summary

The usability test revealed that while the product performs adequately in most areas, there are specific pain points around the length of the onboarding process and account upgrade. By focusing on these areas, we can significantly improve the user experience.

Next Steps

The design team will implement the recommended changes and prepare for a follow-up usability test to measure improvements in SUS, time on task, and success rate.