Process Alarmistic Tool to Monitor Key Performance Indicators in E-commerce Operations

https://drive.google.com/file/d/1KayeesPChUPYKcmK_fJJq39s6YbQ9hTu/view?usp=drive_link

The highly competitive e-commerce industry is rapidly growing, which brings challenges in handling not only logistics services, such as warehousing and shipping, but also payments processing and fraud detection. Businesses need to monitor key performance indicators (KPIs) to achieve operational excellence, but this increasing complexity leads to the need for better resource and time allocation to find and correct any issues. Tools have been developed to monitor operational KPIs and support decision-making, providing insights through, e.g., drill-down analysis, anomaly detection, target comparison, amongst others. However, there is still not a standardised approach for monitoring KPIs in the context of e-commerce marketplaces, which are websites that sells goods online from a variety of providers. One of the main luxury fashion companies, Farfetch, operates an online marketplace and currently lacks a systematised way to prioritise which KPIs’ behaviours should be paid attention to on a weekly basis. To create this standardised approach, a tool was developed which, through a Slack App, informs stakeholders about the critical KPIs they should focus on for the marketplace. It does so by providing insights based on three main criteria: 1. determining if a KPI has significantly deviated from its target; 2. if it has exhibited recent outlier behaviour; 3. if its actual has fallen within the prediction interval calculated by its corresponding forecasting model. To determine the best forecasting model for each KPI, both exponential smoothing models and Prophet were tested with cross-validation. The selection was based on the Mean Absolute Scaled Error (MASE). The outputs of each of the criterion are red, yellow and green lights, which are categories differentiated by adjustable thresholds. By combining the results of the aforementioned criteria, a KPI’s behaviour is classified as an "alarm" or an "attention", being that an “attention” is a less severe version of an "alarm". If the KPI worsened compared to last week, it is classified directly as either “bad alarm” or “bad attention”. For good alerts, i.e., KPIs which improved and which were classified as “alarm” or “attention”, a different division is made: they are categorised as “recovering” if they are still off target and “over-performer” if they are already on target. When a KPI is neither of these, if the forecasted value for the following week is significantly off target, the alert is “future attention”. The “MAD-Delta” approach is also introduced to detect which dimension groups are the best and worst contributors for each KPI as a whole, depending on whether it improved or worsened, respectively, in comparison to the previous week. To evaluate the tool’s accuracy, thirty scenarios which were categorised as different alert types were presented to two stakeholders, who independently classified them. The results show substantial agreement with the tool’s classifications, highlighting its quality, whilst also having led to the implementation of improvements in the tool. However, variations in stakeholder perceptions underscored the challenges of creating a unanimous classification system.