Institutionalized a test and learn philosophy for a specialty retailer
What We Did: Enabled FP&A managers to design, measure tests and devise go-to-market strategies
The Impact We Made: Provided decision makers with “one version of the truth” and increased profitability by 12%
Summary – In depth understanding of customer behavior
To increase profitability during the housing downturn, a large retailer wanted to experiment various strategies from pricing and marketing to store remodeling. Mu Sigma developed an enterprise level system to design tests and provided business groups with a unified framework to measure the impact. This collaborative platform helped finance teams tweak go-to-market strategies and make better informed decisions leading to a better understanding of customer behavior and a 12% rise in profitability.
About The Client – A specialty retailer
The client is one of the world’s largest retailers and specializes in home improvement and construction products and services. Mu Sigma worked with the finance team to develop a test and learn system that provided feedback on changes in go-to-market strategy implemented by Merchandising, Marketing and Store Operations.
The Challenge – Inability to measure efficiency of store strategies
During the economic crisis and housing downturn of 2010, store profitability declined dramatically. Changes to assortment, pricing and operational strategy were being considered as part of eff-orts to improve profitability and revenue per square foot at existing stores. To determine which of these strategies would prove most effective, many different tests needed to be designed and evaluated before chain wide roll out. It was critical for the retailer to have a unified framework in place that could measure the impact and provide decision makers with “one version of the truth”.
The Approach – A test and learn system
Tests were designed in collaboration with the client’s merchandising, pricing, store operations and marketing teams. Opportunity assessment, duration of a test and changes to current strategy were discussed in collaborative forums with stakeholders to align with organizational goals.
- Test and control groups were identified by evaluating similarity in sales patterns, location, competition and demographics
- Statistical tests were performed to ensure that test groups were representative of the population
- An ensemble of predictive models was used to isolate lift due to events while accounting for seasonality, economic trends, weather, promotions and holidays
- Impact of events on sales, profitability and customer satisfaction was continuously measured at various levels- customer, item/SKU, store and market level
Deep dive analysis on test results were performed and the root causes of exceptions were uncovered. Discussions with finance and relevant business teams were conducted to finalize go-to-market strategy.
The entire algorithm was developed in a modular fashion and was automated to design and measure over 200 tests a day. An R-Hadoop stack was leveraged along with a HTML 5 user interface to provide stakeholders with a system to design tests, measure and institutionalize a culture of low-cost experimentation.
The Outcome – Increased store profitability
- The test and learn framework was highly scalable; End-to-end process automation successfully tested and measured impact for over 2000 events
- A web-based application was built to centrally manage event design and deliver easy-to-understand reporting for executives
- Identified opportunities worth $150M from positive testing and highlighted risks worth $70M from negative testing
- Enabled the client to test various strategies across merchandising, pricing, store operations and pricing, understand their impact prior to an organization roll out, thereby increasing profitability by 12%
- Institutionalized a test and learn philosophy and provided a unified approach to measuring the impact of events across business functions