Companies make a major mistake when they select one forecasting system and simply trust that everything in that system will work as advertised. Instead, forecasting should be prototyped in another competitive solution to make sure that the results match.
Background
As you can see in this post…
http://www.scmfocus.com/demandplanning/2010/07/prototype-environment-and-background/
Having a prototyping environment which can allow a company to simulate different forecasting approaches, (hierarchical approaches, improve forecast accuracy,
formulate planning business processes and test planner workloads and model
design trade-offs.) before locking in on a design for a bigger, less flexible system like APO.
I have decided to perform prototyping in a product called Demand Works Smoothie in order to troubleshoot a problem DP implementation. This is important because SAP DP is not a good prototyping environment, and the way the system is currently configured is not working properly. Thus I needed a reliable external system to get the forecasting down to where my client needed it. After reviewing a number of products, I picked Demand Works Smoothie (heretofore known as just “Smoothie”) in order to do this job because of its strong ability to do best fit forecasting (called Expert selection in Smoothie), the fact that it has resident within it the forecasting models that I needed to test and the fact that DemandWorks has knowledge able support. It is important to point out that DemandWorks can be used for a lot more than I am describing here, however, this is the need that I originally selected Smoothie for. To find out all the different things that Smoothie can do, please refer to their webpage.
Data Storage and Manipulation
One of the most important factors for a good prototyping environment is having a fast and easy to manipulate data back-end. This is where SAP DP really falls down. SAP DP’s data backend is so onerous that it takes quite a lot of time to make changes to it, and in my view is one of the poorest areas of functionality in the entire SAP APO suite. (see details on this topic here.)
http://sapplanning.org/2009/05/09/the-worst-functionality-in-scm/
Data management in Smoothie is very easy. It read both databases and spreadsheets. Up until this time I have only connected it to spreadsheets for data uploads. However, the spreadsheet that is uploaded to Smoothie is very easy to manage. Essentially, all the different items that are to be uploaded are different tabs in the spreadsheet. This allows a different model to be saved as a simple spreadsheet, and one spreadsheet model can hold anywhere from 2 tabs (if only basic forecasting is performed) to quite a bit more that that for different measures, or if supply planning is being done (Smoothie does both demand and supply planning, with the Smoothie SP being an extension or add on to the Smoothie forecasting application).
The Smoothie manual describes its data management in the following way:
Almost all data in Smoothie is sorted at the base of the model. The model’s base is the level that the data is imported into. You can easily identify the model’s base level by using the Tree driller and clicking on the plus sign next to the folder.
The dot indicates that you are working with a base level item. Folders indicate that you are working with an aggregated of items (i.e. a hierarchy)
Smoothie does things differently depending upon whether you are at the base level or working with an aggregation. For example forecast model changes made in Visual Forecasting are handled differently depending upon whether you are working with a detail item or an aggregation. - Smoothie Help
Testing
The steps were pretty simple. I needed an extract of my customer’s data per SKU. As my client only forecasted to one location, that decreased the SKU number significantly. Secondly I decided to remove the very low demand items from the product database. Very low demand was less than 3 items per year. That reduces the database somewhat. Thirdly, I need to get only those items for which there were no manual overrides. I wanted to compare a strict statistical forecast in Smoothie to one in SAP DP. If I pulled products which has been manually overwritten I would havecommingledthe forecasting effectiveness of individual forecasters with the actual statistical method used. I did not want to do that, so I excluded manually altered forecasts. This would now be an apples to apples comparison.
Next I decided to simply forecast using Expert mode in Smoothie and the system selected the forecasts.
I then exported from Smoothie and this provided me with a file which showed the method chosen for every product. It also provided me with several different forecast error measurements. The question was whether Smoothie could produce forecasts that were of higher quality than the forecast produced by DP. This should be relatively easy as the SAP DP implementation that I was evaluating was primarily going out on one model.
Flagging Unforecastable Items
It is an illusion to think that all items in a product database can be forecasted. Items of high variability and intermittancy, at some point cannot be forecasted, and need to be placed on consumption based logic in supply planning. Thus these product types do not need to be in the planning engine (although they do need to be in the S&OP engine, and this can be accomplished by generating two feeds, one from the forecasting engine for forecastable products, and one from the supply planning engine providing non-forecastable products). In determining which products should not have a forecast, I decided to use MAPE, MASE and the R-squared (which in Smoothie is a correlation between the statistical forecast and the naive of mean averaged over time.) In this way I was able to segment my client’s items into forecastable and unforecastable products. This was helpful in clearing out a lot of the product database from the forecasting engine that was simply not adding any value being there. To find out more about determining forecastability see this post.
http://www.scmfocus.com/demandplanning/2010/06/forecastable-non-forecastable-formula/
{ 1 trackback }