What are the problems in today’s product delivery process?
Our existing product process is failing to consistently deliver high quality product in an agile manner.
These problems can be summarised as below:
Our product’s functionality do not consistently exceed customer’s needs
Our product UX is shit
Our product engineering quality is not great
Most product team members are disconnected from business realities hence empathy is lost
We are only as good as our products and how our customers love us, so we have to change or die
Diagnosing these problems
Why does our product’s functionality not consistently exceed customer needs?
Business leaders today define functionalities that need to be worked on and pass down to technical product managers verbally or through documentation of various degrees of quality.
Technical product managers are then responsible for translating these requirements into working software by working with EA and engineering teams.
Business leaders are responsible for running the whole business line often have many other responsibilities aside product management. They do not have the full bandwidth to talk to customers, study data, experiment, innovate, work with user experience design, work with compliance, risk, finance and every other internal role needed to be consulted.
The lack of a deliberate and careful product discovery process stemming mostly from lack of bandwidth is what has led to products that are are just functional and rarely exceed customer’s expectations.
Why is our product UX shit?
Most of our user experience design team do not design anything today, they just paint.
They often do not own the user journey, test it, mostly lack technical depth in human interaction design and are sometimes disconnected from the product development process. Vetting designs has been the responsibility of the business leader or even TPMs rather than with customers themselves.
Why is our product engineering quality is not great?
We lack mature software development practices that should include unit tests and automated functional tests
Engineering teams do not have complete ownership and observability of production. This role has been solely given to the technical support teams
Engineering teams do not know the limits and scale ratings of their services
Engineering teams are sometimes not given clear specifications that can allow them design the systems to work appropriately and allow for change in the long term
Engineering teams spend valuable time building products that do not provide value to customers, and have not been not validated with customers. This means they needlessly incur technical debt, and consumes their time so they do not have capacity to work on strategically important projects
Business product teams are not prioritising paying technical debts.
Why are most product team members disconnected from business realities, meaning empathy is lost?
Most teams get handed instructions and deadlines with some briefing of the reason for the initiative from their business leaders.
TPMs and engineers often participate in turning these instructions into working products but do not participate in the risk and rewards of the outcomes.
Many other functions are often not aware of the initiatives and only hear by happenstance either through the grapevine or from customers.
The lack of ownership of business outcomes hence often make the teams disconnected and lose business and customer empathy.
Solving these problems
How will our product’s functionality consistently exceed customer needs?
The process of producing products that customers will love is not by accident. It comes from understanding the customer through qualitative and quantitative analysis of customer’s needs and behaviours, superimposed with internal considerations from functions such as growth, compliance, risk, market share, sustainability, etc.
Business leaders with medium to large scopes do not have the bandwidth for thorough analysis of these considerations. To do this properly, Business Product Managers, shortened to ‘Product Managers’ will need to fill this role.
Product managers will now be responsible to achieving the organisation’s product objectives by co-owning KPIs that are assigned to them together with their product teams.
They will be working with the various other functions to meet these objectives by launching initiatives with them that will cause changes to these KPIs to reach the goals set. Essentially, we are elevating the role of product management.
How will our UX no longer be shit?
Once product managers have clarity on what needs to be built, the responsibility of crafting this into an awesome, well thought through user experience is on the UX Designer role.
Working from a position of thorough understanding of human interface design, they will make protoype designs which they will go test with customers and ensure the customers don't need to think to use the product.
They will decide using existing design framework the layouts, actions, animations, nudges, layering etc. that ensures customers find our product intuitive and easy to use.
UX designers will no longer be painters but owners of user’s experience through embracing world-class design systems, feedback and iteration through qualitative and quantitative analysis.
How will our product engineering quality be better?
We will now mandate automated functional testing, fuller unit test coverage, static code analysis etc. Engineers will now be responsible for automating functional tests while working with QA analysts
If you build it, you own it. There will be automated measurements of service level KPIs and the results will affect team appraisal
Engineering teams will be expected to load test and rate their services to know their production limits
Improved product management will ensure requirements are clear and minimise changes in the product specification. Deliberate product iterations will afford engineering to also deliberately evolve the services to meet business objectives
Engineering teams will no longer spend time building products that do not provide value to customers, or are not validated with customers. This will reduce the volume of tasks they are required to complete, allowing them to focus on the most important work
The Product Manager directed teams will now have a complete view of all the initiatives that need to be worked on from growth to compliance and to technical debt. This will allow the teams to balance the need building new features with cleanups along the way.
How will product team members will be connected to business realities and gain empathy?
Everyone in teams and sub-teams will now be appraised on the basis of the performance of their teams and sub teams as a community.
Team and sub-team KPIs are visible to everyone. With this visibility and community appraisal, we expect everyone to be accountable to each other and band together to reach business objectives.
More updates to our processes
1. KPIs will measure performance
What is changing: The performance of every part of the business will be measured using KPIs.
Why we are doing this: To allow the company to identify problems in real time, and be able to fix them quickly and efficiently. It will also allow Moniepoint to focus resources on where they are most needed.
How it will be done: Each business unit, team, sub-team and individual will have KPIs.
These KPIs will be measured and will be the basis by which we measure performance on an ongoing basis. Subjective assessments will be removed as far as practicable.
2. Product Managers will gather all potential initiatives, and prioritise to meet KPI goal(s)
What is changing: Product Managers will gather and thoroughly validate all possible initiatives from every part of the business.
Why are we doing this: To ensure we make objective prioritisation decisions, to most effectively achieve our KPI goal(s).
How it will be done: Product Managers will gather initiatives from:
Growth (who will come up with initiatives that impact revenue)
Compliance
Engineering (technical debt)
Finance
QA (bug fixes)
Customer Service
Market and competitive analysis
Others (as applicable)
Product Managers will use data and customer research to validate these initiatives.
If an initiative involves just that business line, it is covered by that Product Manager, with no need for involvement from other business lines.
If an initiative cuts across multiple business lines, it is referred up to the CPO. The CPO then co-ordinates the whole project, involving each of the business lines required.
With a full set of initiatives for their business line- all of which are validated and based on accurate data- the Product Manager and Business Leader will prioritise initiatives to most effectively achieve their business lines' KPI goal(s).
3. New potential business lines will be run as lean startups
What is changing: New potential business lines that Moniepoint is exploring will be validated and run in the early stages by a Product Manager, who acts as a mini-CEO.
Why are we doing this: Each business line costs $1m-$3m to run. Staffing a business line before the product has sufficient traction is a deeply inefficient way to allocate capital.
Moniepoint will allocate capital to a smaller number of validated products that it has high conviction on. It will do this through high-quality early development work by a Product Manager.
How it will be done: Product Managers who work directly for the CPO may be assigned to research and validate new potential business lines.
In this capacity they act as mini-CEOs. The Product Manager fulfils all the roles of that business line in an acting capacity. If the business line is validated by customer research and is deemed deserving of further investment, the roles that the Product Manager is filling are backfilled in order of priority.
If the new business line is a big, important area for Moniepoint, a Business Leader will be brought in. otherwise, the Product Manager will continue to run the business line.
4. Deep customer research will determine what we build
What is changing: Customer research will be much deeper and more thorough than has been conducted to date. Products will be tested and validated prior to being handed to Engineering teams to build.
Why we are doing this: To ensure Engineering resource is only used on building product that we know will give maximum value to customers. Cutting wasteful development will result in a significant increase in efficiency in Engineering teams, and reduce the amount of technical debt incurred.
How it will be done: The business leader and the Product Manager will prioritise initiatives.
The Product Manager is expected undertake market and customer research to work out what we should build to deliver the maximum impact- in the way that brings the most value to the customer. The research will be both qualitative and quantitative- gathered through internal and external data, and from speaking to real customers.
The research will inform what to build, market positioning, and the complete product UX and UI (created through prototype testing and customer feedback prior to Engineering build).
5. Product design will become a centre of excellence at Moniepoint
What is changing: We are elevating the expectations of the UX Design teams. They will completely own the experience of a user
Why we are doing this: A crucial element for our product to be the best in the market is for it to have the best UX and UI.
To make our brand the most prominent in Nigeria, UI must be high-quality and applied consistently across all products and marketing materials. World-class UI will also have a major impact on the customer experience and perception of Moniepoint.
How it will be done: We are hiring a world-class UI/UX designer to ensure exceptional design systems are in place for both UX and UI.
The expert will create the standards, processes and design systems to be applied across the company, but they will also support designers in business units to adopt the design systems.
The experts will also work closely with designers embedded in business units to upskill them.
More details in Organisation Structure section below.
6. Product Managers will be empowered, and the voice of the customer
What is changing: The customer is going to become deeply important in every facet of Moniepoint. Product Managers will be the voice of the customer, and will be crucially important to each Business Unit achieving their goals.
Why we are doing this: Only by truly understanding our customers and creating products that deliver them value will we be able to move faster than the competition, and achieve our North Star Goal. For this reason Moniepoint is intentionally shifting to becoming truly customer-centric.
How it will be done: Product Managers will be the voice of the customer. They will:
Deeply understand the market
Deeply understand the competitive landscape
Be close to the customer
Spend time to really understanding the customer’s experience
Be expected to represent the customers interests in all internal interactions.
Product Managers will have responsibility for running the research and validation for each product initiative, and monitoring its implementation through the build stage. They will be expected to continue to monitor the performance of their products, and work to optimise and manage products through their full lifecycle.
Product Managers- in conjunction with their Business Leader and taking input from all relevant functions including TPM, Compliance, Growth and others- will manage the product roadmap. They will ensure the roadmap is focused on achieving business objectives, by delivering value to our customers.
7. Process and systems implementation will be enforced
What is changing: Moniepoint is putting in place processes and systems which will allow us to achieve our North Star Goal. We will monitor the implementation of these by teams across the company.
Why we are doing this: The processes and systems are put in place to allow Moniepoint to produce products faster and to a higher quality. These systems will not be successful unless they are adopted in practice by teams across the company. Implementation is where many such systems fail.
How it will be done: Implementation and adherence to the process will be monitored and enforced through KPIs and metrics. This is made possible by the process being measurable so it is clear to see how well teams and individuals are adopting the process, and the impact this has on outcomes.
There will be a large number of stakeholders within Moniepoint that will be involved at various points in the Product Development Process. The process will be introduced as part of onboarding and training, and reinforced regularly to all stakeholders.
Teams, Sub teams and overall Organisation structure
Business units are made up of product teams. Product teams are led by Product Managers and have other functions as sub teams. A sub team is a group of one of more people dedicated to a function needed for the team
Teams are given KPIs to own and every initiative they work on is the reach set KPI goals.
Every team is will consist of a Product Manager, one or more engineering sub-teams each having its own TPM and an EA, a growth sub team, user experience sub-team, product operation sub-team, technical support sub-team, finance product control sub-team and compliance sub-team.
The Product Manager, TPM, EAs, Product Operations leaders are all expected to exclusively belong to the team. Other functions can be played by people who are shared with other sub teams. This does not however reduce their accountability towards team and sub team objectives.
Functional leaders as centres of excellence will continue to play their role of improving the state of the art of their functions across the organisation.
Business level structure
Function | Who is Responsible | Divisionally reports to | Functionally reports to |
|---|---|---|---|
Overall leadership | Business leader | CPO | CPO |
Product management | Business leader | Business Leader | CPO |
Technical product management | Program manager | Business leader | SVP TPM |
Quality assurance | Program manager | Business leader | SVP TPM |
Engineering | VP of engineering | Business leader | CTO |
Product design | Business leader | Business leader | Head of UX/UI Design |
Product Growth Management | CCO | Business leader | CCO |
Operations | Head of operations | Business leader | Business leader |
Compliance | Compliance business partner | Business leader | CCO |
Finance | Finance product controller | Business leader | Financial controller |
Team level structure
Function | Who is Responsible | Divisionally reports to | Functionally reports to |
|---|---|---|---|
Overall leadership | Product manager | Business leader | Business leader |
Product management | Product manager | Business leader | Business leader |
Technical product management | Director of Technical product management | Product manager | Program manager |
Quality assurance | Director of Technical product management | Product manager | Program manager |
Engineering | Director or Engineering | Product manager | VP of Engineering |
Operations | Director of Operations | Product Manager | Head of Operations |
Product design | Product manager | Product Manager | |
Product Growth Management | Business leader | ||
Compliance | Compliance business partner | Business leader |
Engineering sub-team structure
Function | Who is Responsible | Divisionally reports to | Functionally reports to |
|---|---|---|---|
Technical product management | Technical product manager | Direct of Technical product Management | Direct of Technical product Management |
Quality assurance | Engineering Architect | Technical product management | Direct of Technical product Management |
Engineering | Engineering Architect | Technical product management | Director of Engineering |
Org charts
Roles in the team structure
The responsibilities of each key role embedded within a Business Unit, and the changes in role relative to today are outlined below:
Business Leaders
Responsibilities: Business Leaders will retain responsibility for achievement of the business units' objectives.
Changes: Business Leaders' role is unchanged.
Product Managers
Responsibilities: Product Managers will have responsibility for:
Initiative research and prioritisation: collecting potential initiatives from across the organisation (particularly Growth) and externally to understand potential initiatives the business line could work on. Validate these initiatives, and prioritise work on them to give the business line the most effective path to achieve KPI goal(s)
Market research: understand market and competitor dynamics, to ensure Moniepoint’s products are consistently the best in the market
Product research: collecting qualitative data from face-to-face customer research to make informed decisions about what to build to deliver the highest impact, given market and competitive dynamics
Managing the creation of prototypes and validation of designs, to ensure that Engineering teams are provided with validated, rigorously tested products to build
Managing the roadmap of products that will be delivered to achieve these Business Leaders' objectives
Some Product Managers will have responsibility for starting new potential business lines and acting as mini-CEOs in early stages of the business lines' development (see here)
Changes: Product Managers will have an elevated role. The profile of candidates will need to be higher than has been the case previously. They will be a crucial element in achieving the objectives of Business Leaders through building the highest-impact products, and managing the roadmap of products that will be delivered to achieve these business objectives.
Being able to do this whilst also running proper testing and validation of products- and managing the full suite of products through the entire lifecycle- requires high-quality individuals.
TPMs
Responsibilities:
Fast, high-quality technical product delivery
Comprehensive and accurate project tracking in Jira
Changes: TPMs will no longer be required to do customer research and validate products with customers prior to build, as this will be the responsibility of Product Managers.
TPMs will have explicit responsibility for acting as ‘scrum master’ to input data and updates in Jira. Having accurate inputs into Jira is a crucial element required for success of the new system.
Product Managers taking a more prominent role with responsibility for market and customer research,a nd delivering validated products that bring value to customers will free up TPMs to focus on efficient and effective product delivery.
Product Designers
Responsibilities:
Create best-in-class products using the UX and UI design systems and frameworks
Ensure the design system is applied consistently to every new product
Continually upgrade the UX and UI of existing products
Changes: Design will be much more intentional and will be expected to have best-in-class UX and UI, consistently applied using central design system and standards from the outset of a product launch.
Designers will work closely with Product Managers to prototype and design products to allow products and designs to be validated with real customers prior to being built. This will result in valuable Engineering resource being focused only on already validated products.
Product Operations
Responsibilities: Establishing, running and optimising operational processes for their business line.
Changes: No changes
Product Growth Management
Responsibilities:
Achieve product growth according to KPIs
Creating all relevant product communications
Engaging relevant stakeholders to design & distribute materials
Deeply understand conversion funnel and growth dynamics
Changes: No changes.
Engineering
Responsibilities: Fast, high-quality software development to enable technical product delivery.
Changes: No changes.
Risk & Compliance
Responsibilities: Ensure that all product development activities adhere to regulatory standards, mitigate potential risks, and maintain the integrity of Moniepoint’s compliance protocols.
Changes: No changes.
Finance
Responsibilities: Financial analysis, ensuring settlement & reconciliation standards are in place and followed, and financial resource allocation (if applicable) for products.
Changes: No changes.
Functional Managers as centres of excellence for the role
Each of the roles outlined above have functional management, as detailed in the Business Unit org chart.
The role of the functional managers for each function is to:
Create world class systems and frameworks, to be adopted by decentralised teams within the Business Units
Ensure the systems and frameworks are used correctly and consistently across all relevant areas of Moniepoint
Mentor, train and continually upskill their functional reports embedded within Business Units
Product delivery process
Phase 1: Request for change
Input: Desire for change to meet some KPI goal(s)
Objective: Gather all potential initiatives the business line could work on, validate them, and determine prioritisation to allow the business line to most effectively achieve the KPI goal(s).
The Product Manager should qualitatively and quantitatively analyse customers, market and growth data, compliance needs, technical debt, finance needs, and other relevant internal or external inputs to be able to accurately list all initiatives that may need to be worked on, and determine what needs to be done or built.
This responsibility falls on a product manager and they will do this by talking to customers, analysing growth data and insights and speak to other sub teams and business units to gain context.
This understanding should lead to hypothesis that they will test to ascertain its workability before they draft the other sub teams for implementing the change.
Outputs:
Clear documented request for change document
KPIs that may need to be introduced to track the performance of initiative
Responsibility: Product Manager
Participants: Customers and other internal sub teams, teams and business and functional units.
Phase 2: UX design
Input:
RFC document containing an explanation of the motivation for the initiative, supporting data and conclusions on how the product will work.
Low fidelity mock-ups of product concepts that capture the value proposition
Objective: Prototype the product concept, test with real customers, iterate and validate
Output:
High fidelity product mock-ups with complete UX and UI design
Responsibility: UX designer
Participants: Product Manager, UI designer, Growth team
Phase 3: Technical Deep-dive
Inputs:
RFC document
High fidelity product mock-ups with complete UX and UI design
Objective: Get complete clarity on the implementation details for the proposed product, document the implementation steps and estimate the work to be done
Output:
Updated tickets to capture the Implementation details and sub tasks
Detailed breakdown of engineering steps into subtasks in each ticket
Story points to be assigned to the ticket
Test cases
Architectural design
Responsibility: TPM
Participants: Product Manager, EA, Engineers, QA, BL, VPE, Security Analyst
Expected time to complete: 2 days
Measurements:
Metric | Motivation | Source |
Story point coverage | To measure the adoption of work estimation in our product management process | Jira |
Acceptance criteria coverage | To encourage the addition of acceptance criteria by the PMs | Jira |
Functional Test Coverage | To encourage the addition of relevant test cases to Jira tickets | QMetry via Jira |
Phase 4: Sprint Planning
Input:
Backlog of products that have gone through deep dives and have been estimated
Backlog of bugs & technical debt that have been discussed and estimated
Objective: Determine the sprint goal, i.e. the work to be delivered at the end of the sprint and each product team member’s role
Output:
List of tickets to be worked on during the sprint
Assignment of tickets to engineers
Responsibility: Business Leader
Participants: Product Manager, TPM, EA, Engineers, QA, BL, VPE
Expected time to complete: 1 day
Measurements:
Metric | Motivation | Source |
Team and sprint health | To help measure the flow of work within the team and potentially predict the delivery speed of a team | Jira |
Notes
Team and sprint health is a combination of different metrics to help answer the questions: how much work are we doing? Where are we spending most of our time e.g fixing bugs, new initiatives etc? Are there bottlenecks or inefficiencies that could pose a risk to the delivery of the initiatives? Is the team overwhelmed? Are certain resources within the team under-utilised or over-utilised? How much are we investing to execute this initiative? etc. When we have clarity on the specific metrics this document will be updated accordingly.
Phase 5: Coding
Input: The products to be implemented and test cases to be automated
Objective: Implement the products according to the agreed design and automate the test cases using our automation test suite
Output:
A working deployable feature
Automated test cases
Responsibility: TPM
Participants: EA, Engineers
Expected time to complete: variable depending on size of project
Measurements Summary:
Metric | Motivation | Source |
Cycle time | To measure how efficient our pipeline is by measuring how long it takes to go from code commit to production | Gitlab,ArgoCD |
Code suggestions | To improve code quality and reduce cycle time | SonarLint |
Time to merge | To measure how long it takes codes to be reviewed and merged - long wait times requires investigation | Gitlab |
Coding time | To know how long an engineer spent implementing a given task | Gitlab |
PR count per contributor | To measure how any PRs is initiated by developer | Gitlab |
Review reaction time | How fast do reviewers respond the merge requests | Gitlab |
PR size | To know how if developers adhere to small PR size best practice | Gitlab |
PR comment count | To measure the toll required to review an engineers code | Gitlab |
Notes
The metrics above are by no means exhaustive, keep in mind that the dashboard tool might have more metrics then listed here and sometimes the metrics might be represented under a different name
Phase 6: Build Pipeline
This is a critical stage comprising a number of quality gates to help ensure the quality of the product feature(s) being shipped. The goal is to pass the modified code base through series of quality checks and ensure that the code base passes all of these checks before it can proceed to the next stage. On successful build, the feature is deployed to a dev environment for further checks if it’s a service, if it is a library it is pushed to the relevant package registry for distribution. On failure, the pipeline fails allowing the team or engineer responsible for the changes being verified to fix the failure.
Input
Committed or merged code
Objective: build the modified code base, pass it through series of quality checks and ensure that the code base passes all of these checks before it can proceed to the next stage
Output:
A working deployable package (docker image, packages etc)
Responsibility: TPM
Participants: TPM, EA, Engineers, QA, BL, VPE
Expected time to complete: 3 days
Pipeline Quality Gates:
Gate | Measurements | Source |
Unit Test | Unit test coverage | Cobertura, Jasmine |
Static code Analysis | Unit test coverage | SonarQube |
Container Analysis | Container vulnerabilities | Trivy |
Functional Test | Automated functional test coverage (API) | Cypress/Playwright |
Load Test | Latency score Test Result (successful if the agreed thresholds are maintained otherwise it is considered a failed test) | Jmeter |
Security Test | API Vulnerabilities | Astra(TBD) |
Notes
Executing some of the quality gates e.g functional test, load test and security test will require spinning up a fresh and clean environment, seeding that environment with necessary data before running the tests. We have agreed to adopt Garden as an orchestration tool to implement this process. You can find the documentation and template on how to include this in your projects /wiki/spaces/MAE/pages/1168244934
Phase 7: Production Deployment
The target is to have a completely automated process where the outputs of the build stage is deployed to production environment without human intervention. However, I acknowledge that this will require work (for instance having feature flags) and time before we get there, so for now we will include a human element by retaining our current deployment flow, where EAs or anyone with the right authority can modify the Gitops file to add the new release version before they are picked by ArgoCD for deployment. From there, we can work towards the ideal scenario as we grow more confident in our pipelines. That said this ideal scenario will remain an actively tracked target for all teams.
Input
Release tag
Objective: Deploy the tested service by simply modifying Gitops file
Output:
A running service without defects
Responsibility: TPM
Participants: EA, Engineers, QA, BL, VPE
Expected time to complete: 1 day
Measurements:
Measurements | Motivation | Source |
|---|---|---|
Deployment status | To help us evaluate the outcome of a deployment. This will feed into measuring the reliability of our pipelines | Argocd rollout |
Error rate | To measure the impact of of the deployment on error rate | Newrelic |
Latency | To measure the impact of of the deployment on latency | Newrelic |
throughput | To measure the impact of of the deployment on throughput | Newrelic |
Resource Usage | To measure the impact of of the deployment on resources | Newrelic |
Deployment Frequency | This is a derived metric that helps evaluate the speed and reliability of our pipeline - being able to deploy fast shows our pipeline is reliable and fast enough to push changes | argocd |
Change failure rate | This is also a derived metric which measures the reliability of our pipeline - a low change failure rate implies having a reliable pipeline that prevents defects from getting to production | newrelic, pagerduty, argocd |
Notes
This assumes that every team uses Gitops flow for deploying changes. Therefore it is now compulsory for every team to implement Gitops deployment strategy -including (Canary or blue/green deployment) for all your services. This is will be an actively tracked target for every team. Reach out to David Irivbogbe for guidelines on how to add this to your project.
Phase 8: Production Monitoring
In this phase the services running on production are continuously monitored in order to proactively identify performance bottlenecks, defects, anomalous behaviours etc.
In addition, products in production should be monitored by Product Manager for customer satisfaction, and optimised based on customer needs on an ongoing basis.
Input: Running services
Objectives:
Monitor the performance of our products on production
Monitor and optimise for Customer Satisfaction and Product Market Fit (PMF)
Output: Product optimisations included in sprint queues.
Responsibility: Product Manager
Participants: TPM
Expected time to complete: ongoing
Measurements:
Measurement | Motivation | Source |
|---|---|---|
Error rate | To know the percentage of our requests end up in failure | Newrelic |
Latency | To know how fast we respond to user requests | Newrelic |
throughput | To know much load we are able to handle | Newrelic |
Resource Usage | To know the amount of resources spent on handling the current load | Newrelic |
Apdex | To measure how happy our customers are | Newrelic |
Uptime | To know how available are our services | Freshping |
Mean time to recover | To help us see how fast we recover from failures | Newrelic, pagerduty |
Incidents Count | To know how many incidents we emit; both customer reported and those automatically triggered by monitoring tools | Pagerduty |
Customer Satisfaction | To measure how satisfied the customer is with the product in production | CSAT |
Product Market Fit | To measure whether the product in production suits market needs (has the market changed, and our product is left behind?) | CSAT |
Notes
Incident analysis is a separate topic detailing how we handle the different types of incidents and how they impact teams, for now we will stick with this and evolve accordingly.
Process Implementation
The implementation of the process above is divided into the stages
Visibility
Adoption
Re-evaluation and Continuous improvement
Visibility
We need to implement the framework that will allow us measure the metrics outlined above - with that it becomes easier to know our base state. This will help us achieve three important things
We will see what we need to work on.
We can set realistic targets and timelines
We can measure the impact of our efforts towards meeting those targets.
The framework for visibility is divided into the aspects below:
Aspect | Process Phase | Source | Dashboard Tool | Data Ingestion Mode |
|---|---|---|---|---|
Product Research Visibility | Phases 1 to 2 | |||
Product Management Visibility | Phases 3 to 5 | Jira, Gitlab | Selection in Progress. The options are Hatica, Jellyfish, LinearB. /wiki/spaces/MAE/pages/1320690426 is the RFC for the tool selection | Direct integrations with the sources |
Build Pipeline Visibility | Phase 6 | SonarQube, Jenkins | Grafana and/or Jellyfish,LinearB, Hatica | Webhook integration and/or ETL |
Production Visibility | Phases 7 to 8 | ArgoCD, Newrelic, Pagerduty | Newrelic and/or Grafana | Webhook integration and/or ETL |
The outputs of this framework will be dashboards showing the team level performance across the different aspects.
Adoption
Once we have implemented the needed observability framework, we are then able to track these metrics on a team level, this will hopefully help the teams become self aware and self improving. In addition to that, we can set targets for every team and see their results in real-time. This will help every stakeholder to objectively see the level of adoption in each team and also see the impact of the effort being put into adopting the process by every team.
Based on these data, we can establish a team appraisal cadence through the leaders of each team.
Re-evaluation and Continuous improvement
The goal is to keep improving our processes and make it a well oiled machine where every component is healthy and contributing effectively to the success of the organisation. While at it we will keep evaluating the process using the metrics and making sure we keep evolving and improving the process based on business realities.
Final Notes
This is work in progress do not hesitate to make suggestions!
Add Comment