Embedded Automatic Model Training and Forecasting in an Enterprise Software Application
How can the process of Knowledge Discovery in Databases be automated, competitive and reliable? One approach is to focus on a narrow vertical market application, with known data sources and data feeds. Then you can automate the Exploratory Data Analysis (EDA) and Preprocessing phases.
But how do you automate the selection of training data? Can the enterprise application be installed and configured at a variety of clients without a Senior Knowledge Discovery Engineer? How can you minimize "worst case" results of such a system when used by a business user going through their normal business role? How can you deeply investigate and model "business values" (i.e. things that can get an end user promoted or fired) into the core of the data mining algorithms?
This talk answers these questions and more. The patent-pending application, ELF, is an enterprise application in the retail supply chain vertical market. Before the development of this system, one enterprise application was used to lay out a weekly newspaper flier three weeks before the sales event, which in turn fed data into a replenishment application.
The replenishment application kept products on the store shelves, with a minimal amount of over stock and under stock. The pain point was that the retail buyer would have to manually estimate the the sales lift, or the multiplier increase in sales, for every item for every store. While human expertise can be great, it isn't as scalable when applied to a sales event with 1,000 - 4,000 items on sale in 6,000 stores.
ELF (Event Lift Forecasting) would import data from a planned event and automatically analyze and forecast the lift for each store-item combination. Data elements used included pricing, placement in the flier, store geography and demographics, seasonality, and product hierarchy.
The resulting ELF system produced a 8-30% reduction in over and under stock costs, which is very significant in terms of the low profit margins in the supply chain industry.
Greg Makowski is a Principal Consultant of Golden Data Mining, in Los Altos, California. Since 1992, he has deployed over 70 data mining models for clients i n targeted marketing, financial services, supply chain, e-commerce, and Internet advertising in North America, South America and Europe.
He has applied a variety of data mining algorithms during these engagements and has experience using SQL, SAS, Java, and areas of Cloud Computing. Greg has eight years of experience in Product Management and over six years of experience working with start ups.