Building an Operational Quality System for Fast Delivery
Building an Operational Quality System for Fast Delivery
Building an Operational Quality System for Fast Delivery
How I connected grocery operations, data, and courier incentives to improve order integrity
How I connected grocery operations, data, and courier incentives to improve order integrity
How I connected grocery operations, data, and courier incentives to improve order integrity
+0.7 percentage points in order integrity
97–100% basket completeness
+35% increase in HR engagement
1.45 million orders per four-week reporting period.
Context
Context
SberMarket was Russia's largest e-grocery platform, operating across 154 cities in partnership with 120 retailers. In 2021, the platform delivered 24 million orders, reaching more than 216,000 deliveries per day at peak and approximately RUB 58.6 billion in turnover.
At this scale, a delivery is not a single product flow. It is a chain of interdependent operations: a customer places an order, a store prepares it, a picker finds the products, unavailable items may be substituted, a courier collects the order, and the customer evaluates the result. A small failure early in the chain can become a poor customer experience, a refund, a support request, or a lost repeat order.
At this scale, a delivery is not a single product flow. It is a chain of interdependent operations: a customer places an order, a store prepares it, a picker finds the products, unavailable items may be substituted, a courier collects the order, and the customer evaluates the result. A small failure early in the chain can become a poor customer experience, a refund, a support request, or a lost repeat order.
I joined as Product Owner, Operational Quality & Service Design. My focus was to understand how the fulfilment system worked in reality, identify where quality and business value were being lost, and turn those findings into interventions that operating teams could implement and measure.
I joined as Product Owner, Operational Quality & Service Design. My focus was to understand how the fulfilment system worked in reality, identify where quality and business value were being lost, and turn those findings into interventions that operating teams could implement and measure.
The Challenge
The Challenge
The platform had many operational metrics, but teams did not always share a clear view of which ones mattered most or how they influenced each other.
The platform had many operational metrics, but teams did not always share a clear view of which ones mattered most or how they influenced each other.
Different parts of the system were optimized locally:
stores focused on picking and availability;
couriers focused on completing deliveries efficiently;
retailers focused on their operational performance;
product and operations teams looked at customer and platform-level outcomes.
Different parts of the system were optimized locally:
stores focused on picking and availability;
couriers focused on completing deliveries efficiently;
retailers focused on their operational performance;
product and operations teams looked at customer and platform-level outcomes.
The result was a familiar service-design problem: everyone could see a part of the journey, but no one had a sufficiently clear model of the whole system.
The result was a familiar service-design problem: everyone could see a part of the journey, but no one had a sufficiently clear model of the whole system.
My Role
My Role
I owned the initiative end to end:
mapped the order-fulfilment service, including timed operational steps;
conducted field research and worked through real delivery shifts;
built SQL-based diagnostics across cities, retailers, stores, courier cohorts, and delivery formats;
defined the methodology for the platform's Perfect Order Index;
interpreted an ML-based SHAP analysis to understand which fulfilment metrics most influenced retention;
aligned operations, retailer business development, catalog and out-of-stock teams, fraud, and product;
secured budget and drove targeted interventions across stores, couriers, training, and incentive mechanics;
designed and evaluated a controlled courier-incentive experiment.
I owned the initiative end to end:
mapped the order-fulfilment service, including timed operational steps;
conducted field research and worked through real delivery shifts;
built SQL-based diagnostics across cities, retailers, stores, courier cohorts, and delivery formats;
defined the methodology for the platform’s Perfect Order Index;
interpreted an ML-based SHAP analysis to understand which fulfilment metrics most influenced retention;
aligned operations, retailer business development, catalog and out-of-stock teams, fraud, and product;
secured budget and drove targeted interventions across stores, couriers, training, and incentive mechanics;
designed and evaluated a controlled courier-incentive experiment.
The work required influence without formal authority. I was not managing every team involved in fulfilment, so the solution had to be legible and useful to people with different responsibilities and incentives.
The work required influence without formal authority. I was not managing every team involved in fulfilment, so the solution had to be legible and useful to people with different responsibilities and incentives.
Starting with the real service
Starting with the real service
1. Mapping the fulfilment system
I began with the full order journey rather than with a single interface or dashboard. I mapped the time-sensitive stages from order creation to delivery and connected them to the people, systems, and decisions involved at each step.
I began with the full order journey rather than with a single interface or dashboard. I mapped the time-sensitive stages from order creation to delivery and connected them to the people, systems, and decisions involved at each step.
The service blueprint made several dependencies visible:
product availability influenced whether a basket could be completed;
picking speed affected the handoff to the courier;
substitutions could preserve the order but still reduce its perceived quality;
courier behaviour affected the final integrity of the basket;
the delivery format changed the economics and operational constraints of the order.
The service blueprint made several dependencies visible:
product availability influenced whether a basket could be completed;
picking speed affected the handoff to the courier;
substitutions could preserve the order but still reduce its perceived quality;
courier behaviour affected the final integrity of the basket;
the delivery format changed the economics and operational constraints of the order.
2. Finding the metric that connected the system
I then built operational diagnostics in SQL and compared performance across cities, retailers, stores, courier cohorts, and delivery formats. This helped separate isolated symptoms from patterns that repeated across the platform.
I then built operational diagnostics in SQL and compared performance across cities, retailers, stores, courier cohorts, and delivery formats. This helped separate isolated symptoms from patterns that repeated across the platform.
The most useful signal was basket completeness: how complete the customer's basket remained after unavailable products and substitutions were resolved.
Basket completeness connected several parts of the business at once:
shelf availability and catalogue accuracy;
picking speed and store execution;
the quality of substitutions;
courier productivity and delivery handoff;
customer experience and order integrity;
GMV and the likelihood of a customer returning.
Basket completeness connected several parts of the business at once:
shelf availability and catalogue accuracy;
picking speed and store execution;
the quality of substitutions;
courier productivity and delivery handoff;
customer experience and order integrity;
GMV and the likelihood of a customer returning.
It was not the only important metric, but it was a controllable operational lever with a clear relationship to the outcome we wanted to improve.
3. Creating a shared quality model
To make the work actionable, I helped define the methodology for the platform's Perfect Order Index. The goal was to give teams a common way to talk about fulfilment quality instead of relying on disconnected local measures.
To make the work actionable, I helped define the methodology for the platform's Perfect Order Index. The goal was to give teams a common way to talk about fulfilment quality instead of relying on disconnected local measures.
I also interpreted an ML-based SHAP analysis of fulfilment metrics and segmented the results by customer type — including new versus returning customers and promotional versus non-promotional orders. This added an important layer to the decision-making: the same operational defect could have a different business impact depending on who placed the order and under what conditions.
I also interpreted an ML-based SHAP analysis of fulfilment metrics and segmented the results by customer type — including new versus returning customers and promotional versus non-promotional orders. This added an important layer to the decision-making: the same operational defect could have a different business impact depending on who placed the order and under what conditions.
From insight to intervention
From insight to intervention
Once basket completeness emerged as the key lever, the question became: which part of the system could change it in a measurable way?
Once basket completeness emerged as the key lever, the question became: which part of the system could change it in a measurable way?
The answer was not simply “ask couriers to be more careful.” The intervention had to be:
understandable to couriers;
measurable after substitutions were finalized;
connected to a behaviour that could change during a delivery;
compatible with a high-volume operating model;
testable against a comparable control group.
The answer was not simply “ask couriers to be more careful.” The intervention had to be:
understandable to couriers;
measurable after substitutions were finalized;
connected to a behaviour that could change during a delivery;
compatible with a high-volume operating model;
testable against a comparable control group.
We designed a courier-incentive mechanic that linked bonuses to 97–100% post-substitution basket completeness. The threshold turned an abstract quality goal into a concrete operational condition: the courier's reward was connected to whether the delivered basket preserved the customer's order as completely as possible.
We designed a courier-incentive mechanic that linked bonuses to 97–100% post-substitution basket completeness. The threshold turned an abstract quality goal into a concrete operational condition: the courier's reward was connected to whether the delivered basket preserved the customer's order as completely as possible.
This was part of a broader portfolio of interventions across stores, couriers, retailer operations, training, and incentive mechanics. The experiment gave us a focused way to evaluate one of these interventions rigorously.
This was part of a broader portfolio of interventions across stores, couriers, retailer operations, training, and incentive mechanics. The experiment gave us a focused way to evaluate one of these interventions rigorously.
Testing the hypothesis
Testing the hypothesis
Hypothesis
If courier incentives are tied to high post-substitution basket completeness, courier behaviour will improve and the share of orders delivered with an intact basket will increase.
If courier incentives are tied to high post-substitution basket completeness, courier behaviour will improve and the share of orders delivered with an intact basket will increase.
Experiment design
We ran a controlled difference-in-differences experiment across eight cities in a fast-delivery grocery segment processing approximately 1.45 million orders per four-week reporting period.
The design compared the change in order integrity in the pilot group with the change in a comparable control group. This allowed us to account for baseline differences and broader time effects instead of attributing every change after launch to the incentive mechanic.
The design compared the change in order integrity in the pilot group with the change in a comparable control group. This allowed us to account for baseline differences and broader time effects instead of attributing every change after launch to the incentive mechanic.
The experiment also tested an important boundary condition: the intervention was evaluated separately for fast delivery and planned delivery. This mattered because the two formats have different operational rhythms, constraints, and customer expectations.
The experiment also tested an important boundary condition: the intervention was evaluated separately for fast delivery and planned delivery. This mattered because the two formats have different operational rhythms, constraints, and customer expectations.
Results
Results
The fast-delivery pilot produced a confirmed +0.7 percentage-point lift in order integrity.
The effect was not statistically significant in planned delivery. That result was valuable: it showed that the intervention was not a universal improvement to apply indiscriminately. Its effect depended on the operating context for which it had been designed.
The effect was not statistically significant in planned delivery. That result was valuable: it showed that the intervention was not a universal improvement to apply indiscriminately. Its effect depended on the operating context for which it had been designed.
The pilot therefore gave the team three concrete outcomes:
evidence that the courier incentive could improve basket integrity in a mature, high-volume fast-delivery segment;
a clear indication that the same mechanic should not automatically be transferred to planned delivery;
a reusable approach for adapting incentives to different market and operational conditions.
The pilot therefore gave the team three concrete outcomes:
evidence that the courier incentive could improve basket integrity in a mature, high-volume fast-delivery segment;
a clear indication that the same mechanic should not automatically be transferred to planned delivery;
a reusable approach for adapting incentives to different market and operational conditions.
Additional efficiency work
Additional efficiency work
Alongside the quality initiative, I worked within a McKinsey-led business-efficiency team on process investigations. One investigation diagnosed a push-notification fallback-cost problem worth approximately RUB 55 million per year and scoped the technical fix with engineering.
Alongside the quality initiative, I worked within a McKinsey-led business-efficiency team on process investigations. One investigation diagnosed a push-notification fallback-cost problem worth approximately RUB 55 million per year and scoped the technical fix with engineering.
This work reinforced the same principle: operational quality is not separate from product economics. Small system decisions — a fallback route, a notification rule, a substitution policy, or an incentive condition — can create material cost when multiplied by millions of orders.
This work reinforced the same principle: operational quality is not separate from product economics. Small system decisions — a fallback route, a notification rule, a substitution policy, or an incentive condition — can create material cost when multiplied by millions of orders.
Impact beyond the experiment
Impact beyond the experiment
The project changed how operational quality could be managed across the platform:
field research and data analysis were combined instead of treated as separate activities;
teams received a shared quality model and a common language for fulfilment performance;
interventions were prioritized by their potential impact on customer and business outcomes;
incentive changes were evaluated with a control group and by delivery format;
the organization gained a way to adapt operational mechanics to changing market conditions.
The project changed how operational quality could be managed across the platform:
field research and data analysis were combined instead of treated as separate activities;
teams received a shared quality model and a common language for fulfilment performance;
interventions were prioritized by their potential impact on customer and business outcomes;
incentive changes were evaluated with a control group and by delivery format;
the organization gained a way to adapt operational mechanics to changing market conditions.
The most important shift was from reporting operational symptoms to designing an operating system: identify the lever, connect it to behaviour, test the mechanism, and use the result to decide where it should or should not scale.
The most important shift was from reporting operational symptoms to designing an operating system: identify the lever, connect it to behaviour, test the mechanism, and use the result to decide where it should or should not scale.
My impact
My impact
My contribution combined three roles that are often separated inside large organizations:
Service designer: I mapped the end-to-end journey and exposed dependencies across the fulfilment system.
Product owner: I framed the problem, built alignment, secured budget, and drove delivery across teams.
Data-informed operator: I wrote and used SQL diagnostics, worked with experimental design, and interpreted model outputs to connect operational actions to measurable outcomes.
My contribution combined three roles that are often separated inside large organizations:
Service designer: I mapped the end-to-end journey and exposed dependencies across the fulfilment system.
Product owner: I framed the problem, built alignment, secured budget, and drove delivery across teams.
Data-informed operator: I wrote and used SQL diagnostics, worked with experimental design, and interpreted model outputs to connect operational actions to measurable outcomes.
The project also strengthened my ability to lead through influence. The solution depended on teams that did not report to me, so progress came from making the problem concrete, showing how each function contributed to the outcome, and giving stakeholders evidence they could act on.
The project also strengthened my ability to lead through influence. The solution depended on teams that did not report to me, so progress came from making the problem concrete, showing how each function contributed to the outcome, and giving stakeholders evidence they could act on.
Key learnings
Key learnings
Design can operate at system level
The most valuable design work is not always a screen. In a high-volume service, the experience is produced by a network of policies, incentives, handoffs, data definitions, and operational decisions. Mapping that system can reveal opportunities that are invisible at the interface level.
The most valuable design work is not always a screen. In a high-volume service, the experience is produced by a network of policies, incentives, handoffs, data definitions, and operational decisions. Mapping that system can reveal opportunities that are invisible at the interface level.
The best metric is a bridge
Basket completeness became useful because it connected frontline behaviour with customer experience and business results. A metric becomes powerful when different teams can influence it and understand why it matters.
Basket completeness became useful because it connected frontline behaviour with customer experience and business results. A metric becomes powerful when different teams can influence it and understand why it matters.
A non-significant result can be a product decision
The absence of a significant effect in planned delivery was not a failure of the experiment. It prevented the team from scaling a mechanic outside the context where it worked and helped us treat operating formats as distinct products rather than interchangeable channels.
The absence of a significant effect in planned delivery was not a failure of the experiment. It prevented the team from scaling a mechanic outside the context where it worked and helped us treat operating formats as distinct products rather than interchangeable channels.
Research and analytics work best together
Field research showed how the service actually operated. SQL diagnostics showed where patterns repeated at scale. The experiment connected both perspectives to a decision.
Field research showed how the service actually operated. SQL diagnostics showed where patterns repeated at scale. The experiment connected both perspectives to a decision.
What I would show in the final portfolio version
What I would show in the final portfolio version
To turn this narrative into a visual case study, I would include:
a simplified fulfilment service blueprint;
one diagnostic view showing the relationship between basket completeness and downstream outcomes;
the Perfect Order Index structure;
a compact experiment diagram with pilot and control groups;
a results chart showing the +0.7pp lift and the planned-delivery boundary;
one final “system view” showing how research, data, incentives, and measurement formed a continuous loop.
To turn this narrative into a visual case study, I would include:
a simplified fulfilment service blueprint;
one diagnostic view showing the relationship between basket completeness and downstream outcomes;
the Perfect Order Index structure;
a compact experiment diagram with pilot and control groups;
a results chart showing the +0.7pp lift and the planned-delivery boundary;
one final «system view» showing how research, data, incentives, and measurement formed a continuous loop.
Back to all works
Back to top
Back to all works
