...
Usability metrics and results
Success rate definition
| Note |
|---|
Delete this info panel, this is only for guidance. Success criteria:
Measurement:
|
SUS score
SUS is a standardised questionnaire used to measure the usability of a product, using a 10 questions 5-point scale to generate a score between 0 and 100, with higher indicating higher usability.
The average SUS score is typically 68. A score of 68 or higher is generally considered acceptable or better than average. Above 70 is good, and above 80 is excellent.
...
SUS score
SUS is a standardised questionnaire used to measure the usability of a product, using a 10 questions 5-point scale to generate a score between 0 and 100, with higher indicating higher usability.
The average SUS score is typically 68. A score of 68 or higher is generally considered acceptable or better than average. Above 70 is good, and above 80 is excellent.
...
Time on task
The time it takes participants to complete each journey. Shorter times generally indicate better usability. This metric helps to measure efficiency.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
[insert journey name]: users should/are expected to complete this journey within X minutes.
Success rate
[designer to define success for each journey]
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
[insert journey name]: this journey is considered successful when [specify].
Results
| Note |
|---|
Delete this info panel, this is only for guidance. SUS score The average SUS score was [insert score], with [brief interpretation: e.g., this score falls within/above/below the industry benchmark]. Analysis: the SUS score reveals [positive/negative] aspects of user satisfaction, particularly related to [insert key findings]. Time on task The average time to complete key tasks was [insert time]. Analysis: tasks such as [task 1] took longer than anticipated due to [insert reason], indicating potential usability challenges. Success rate The overall success rate was [insert percentage]. This indicates that [insert percentage of participants] were able to successfully complete the core tasks without assistance. Analysis: certain tasks such as [specific task] had a lower success rate due to [insert reasons], while others were completed with high efficiency. |
An overall SUS score was calculated as the questionnaire was issued after all the tasks were completed by each test participant. These are the results:
Business Banking App SUS score: 34.5
Personal Banking App SUS score: 37.5
...
Core journey
...
Time on task
...
Success rate
...
Comments
...
Check status of all accounts that are attached to the user
...
12s
...
100%
...
View balances
...
7s
...
100%
...
View transaction history
...
8s
...
100%
...
Initiate a funds transfer operation
...
15s
...
100%
...
Initiate a bill payment initiation
...
9s
...
100%
...
Initiate card management flow
...
26s
...
100%
...
Initiate savings product flow
...
33s
...
100%
...
View and initiate KYC level upgrade
...
47s
...
60%
...
Initiate POS product flow
...
10s
...
100%
...
Most BOs were familiar with this
...
Copy account number to add money to the account
...
12s
...
100%
...
All users were able to complete this task effectively
...
Edit App settings
...
44s
...
60%
...
Change password / passcode
...
1m 48s
...
80%
...
Hide and Reveal Bank Balance
...
56s
...
10%
...
Only one user managed to complete this task successfully, and she spent nearly a full minute on the task before she found it. Others gave up after searching with no progress.
...
Change between account number and phone number
...
37s
...
20%
...
Only two people completed this. And that was because they had spotted the action previously while looking around the settings.
...
Edit transaction limit, View account limit
...
43s
...
60s
Post-test qualitative questionnaire feedback
Participants provided additional insights through open-ended post-test questions:
...
Common pain points
...
List any recurring issues, frustrations, or confusion that participants mentioned.
...
Positive feedback
...
Some users mentioned they liked the way the app looked.
Recommendations for improvement
Based on the test findings, the following recommendations are proposed to improve the product’s usability:
Design changes
[Task/feature]: revise [UI component, workflow, etc.] to streamline the user journey and reduce time on task.
[Task/feature]: clarify instructions on [specific screen/interaction] to improve success rates.
Quick wins
...
There were no predefined time on task measurements to measure against, so an average was calculated (per task) once all tasks had been completed by the participants.
Success rate
A task is considered successful if the participant completes it as intended within the defined parameters, with or without assistance (as many of the partipants needed some degree of direction to get things done), and within a reasonable time frame. Success rate will be calculated as the percentage of participants who successfully complete each task as defined by the instruction.
Results
An overall SUS score was calculated as the questionnaire was issued after all the tasks were completed by each test participant. These are the results:
Business Banking App SUS score: 34.5
Personal Banking App SUS score: 37.5
Suceess rates and time on task figures are detailed in the table below
Core journey | Time on task | Success rate | Comments |
|---|---|---|---|
Check status of all accounts that are attached to the user | 12s | 100% | |
View balances | 7s | 100% | |
View transaction history | 8s | 100% | |
Initiate a funds transfer operation | 15s | 100% | |
Initiate a bill payment initiation | 9s | 100% | |
Initiate card management flow | 26s | 100% | |
Initiate savings product flow | 33s | 100% | |
View and initiate KYC level upgrade | 47s | 60% | |
Initiate POS product flow | 10s | 100% | Most BOs were familiar with this |
Copy account number to add money to the account | 12s | 100% | All users were able to complete this task effectively |
Edit App settings | 44s | 60% | |
Change password / passcode | 1m 48s | 80% | |
Hide and Reveal Bank Balance | 56s | 10% | Only one user managed to complete this task successfully, and she spent nearly a full minute on the task before she found it. Others gave up after searching with no progress. |
Change between account number and phone number | 37s | 20% | Only two people completed this. And that was because they had spotted the action previously while looking around the settings. |
Edit transaction limit, View account limit | 43s | 60s |
Post-test qualitative questionnaire feedback
Participants provided additional insights through open-ended post-test questions:
If you could make this app better for yourself, what would you improve
Is there any feature you really enjoy
Common pain points |
|
|---|---|
Positive feedback |
|
Recommendations for improvement
Based on the test findings, the following recommendations are proposed to improve the product’s usability:
General
The SUS test, while fantastic for many other products and mediums seemed to have confused a number of the test participants. Some of them mentioned that it seemed like the were being asked the same question twice and struggled to answer the questions without some sort of additional prodding. A more nuanced testing scale may be required in the future.
Design changes
Edit App Settings: redesign this page to enable users find and edit items quicker. Half the time spent on this task for most participants was on locating the CTA required to complete their task.
Use clearer instructions, and more colloquial english to describe desired actions across the app.
Lean heavily on the use of visual cues similar to what exists on other apps to reduce the cognitive load on users when they try to locate action points and decipher desired behaviours.
Quick wins
Relocating the “Hide/Show balance” CTA to somewhere that’s closer to the balance
Long-term recommendations
Consider a more comprehensive redesign of [specific user journey]the dashboard for business users, especially focusing on [insert issue]simplifying the interface to present a cleaner, easier to understand version for users
Translation of the app to other languages because a considerable volume of revenue comes from areas where English cannot be understood by users.
Conclusion
Summary
The usability test revealed that while the product performs adequately below average in most areas, and there are specific pain points related to [specific feature]to a locating where to complete a majority of actions. By focusing on these areas, we can significantly improve the user experience.
...