Fresh Perspective.

Lasting Results.

Print

Three Kinds of Estimating Error

Overview

Conceptually, there are three major types of estimating error. These include quantity errors, rate errors, and errors of omission. Most companies underestimate how much these errors are costing them. Those that ignore best practices for estimating typically fall victim to the winner’s curse – repeatedly “winning” money-losing jobs.

While companies suffer to varying degrees from each type of error, reducing these errors requires an integrated approach that addresses the causes of all three types of error. It is quite common to see companies improve their pretax profit percentage by 2-5% by improving these systems.

Implementation is difficult and requires thoughtful design and training prior to rollout. Many organizations have failed because of piecemeal implementation or proceeding before staff have an adequate understanding of the tools and how they fit into the big picture.

Introduction

Congratulations! You won that big job. What’s more, you’ve got a foot in the door with a customer that your most hated competitor had a virtual lock on. But now the pain starts. The job is actually happening and it is getting stuck in the system. Purchasing noticed that some key materials were missed on the estimate. You know that you are going to lose money. Then what actually happens to the bank account and the financials after it is done is even worse than you expected. You notice a dent in the financials for the months that the order went through that is bigger than would have been explained by the missed materials alone. What happened? There are several potential sources of error in the estimate.

The Winner’s Curse

All companies in make-to-order environments, including construction, business services, and job shop manufacturers, have to ensure that they are charging the right price for their work, or they could end up losing money on jobs or “leaving money on the table”. As the old saying about the Winners Curse goes:

Whoever makes the biggest mistake wins the job.

Some of the most sophisticated estimators are very large construction companies. Despite their highly advanced systems with fully implemented best practices, some of them still gross up their bids above their target net profit percentage to counteract the winner’s curse. They even have a name for this practice: “bid shading”. The fact of the matter is that all estimating systems include some error. The question is how much!

The Value of Accurate Estimating

Everyone understands that it is important to be an accurate estimator. It is difficult, however, to understand exactly how important. As everyone knows, sometimes you are high and you win anyway, and sometimes you are just a little low, but it is just enough to win what turns out to be a profitable job.

The following graphic shows broadly each of the four scenarios that can come up in the estimating process when an accurate estimator is competing against an inaccurate estimator. The accurate bidder only has bids that fall in the dark blue range, while the inaccurate bidder falls into both the dark and light blue ranges.

Scenario 1: The inaccurate bidder has bid way too low and lost money.

Scenarios 2 & 3: Both bidders fall in the range of the accurate bidder. Each of them wins roughly half the time and makes some money. About half the time, each of the two bidders can be expected to make more than their target profit percentage.

Scenario 4: The inaccurate bidder bids very high. This leaves open the possibility of winning a job at any level of profitability for the accurate bidder. 

Conceptually, what is shown above is common sense. In order to test this assumption and try to quantify the estimate, Bellrock built a simulation of three companies competing 20,000 times for a piece of work in a pure cost-plus environment. The only difference was the variability of their estimates around their target price. The result showed that the optimum profitability improved by about half the reduction in error. For example, if a company dropped their error range from 10% to about 5%, that would increase their profits by about 2.5% on sales. Obviously, this is a significant figure. On $10M in business, 2.5% is $250,000.

Estimating Simulation – what else did it show?

The estimating simulation showed a number of conclusions – some of which were obvious and some of which were not. We can objectively say that the following are facts, not opinion.

  • Getting better at estimating is not a zero-sum game. If all players get better at it, everyone makes more money.
  • It is sometimes more profitable to reduce your markup as you become a better bidder in order to maximize profits (you need less padding when you get more accurate and win more profitable jobs).
  • Still, your estimating accuracy (relative to your competition) is important. As others improve their estimating process, it almost always exposes a company’s estimating processes that have large cost variances by reducing the number of high profit jobs they can win (Are you certain your competitors are not implementing best practices? Have they done it already?)
  • The more competition a company faces, the more its profits are increased by improving estimating accuracy.

The Three Kinds of Error

Having identified that improving the accuracy of estimates is important; the next step is to identify the three sources of error. From there, processes can be devised to control those errors. While all sources of error are related, there are different methods and techniques required to control each of the three types.

  • Quantity Errors. These are errors in the amount of various items, such as identifying the amount of labor or the amount of a given material or service, such as shipping. This is the most important source of error since it not only impacts price, it also impacts the way that operations are evaluated – usually based on job variances.
  • Rate Errors. This is most important for labor. Getting the quantity right and then multiplying by the wrong number can result in drastic errors on ALL quotes that use the affected rate.
  • Errors of Omission. This is the most painful sort of error. Estimating almost always has a human element. Forgetting something such as a particular piece of material, the shipping cost, or an entire operation can lead to a severe case of the winner’s curse.

For the purposes of this paper, we will review the methods used to reduce each of the three types of errors at a very high level only. The specific detailed of application of these concepts will vary within each industry and (to a lesser extent) within each company. For further information, you are encouraged to consult the resources listed at the end of this paper.

Controlling Quantity Error

There are two key types of quantity error: errors on labor and errors on materials, with labor hours typically being the most difficult to estimate. This can be controlled by creating systems for estimators to use in working up their hours. Such a system could identify 1) subsystems with standard numbers of hours, or 2) key factors that have the greatest effect on the total hours for the job. The hours (or setup and run speeds) should be documented, and formats created to work up the right quantities.

For example, in custom cabinetry, the number of drawers can usually be used to estimate the painting time, cutting time and assembly time for the drawers, rather than breaking it down further. The key factors are corners and edges, and the number of corners and edges is typically the same as the size of the drawer changes.

A combination of certain key metrics could also be used to improve the quality of estimation. As an example, in underground piping – the number of joints, diameter of the pipe and total length could be combined to help in improving estimation.

Material quantities can also often be improved. Be certain to have standard approaches to deal with the different scenarios in a way that best fits your company. Consider how case lots or minimum sizes will be figured into the cost, and consider adding a standard waste factor that may be unique to specific operations or products.

Finally, creating a database of past jobs is also advised, as past data can help in improving estimation, since it gives the organization an internal benchmark. Capturing data from past jobs and creating a usable database is not an easy task, however. Systems need to be put in place to capture and update the database on an ongoing basis.

Controlling Rate Errors

Rates are perhaps the most difficult aspect of estimating for a company to set if they do not currently have a system to do so. The process typically starts with creation of a fiscal budget. Costs are applied to each department (or the whole company if a single rate is being calculated), and the number of estimated hours that will be used by that department in a year are spread over all the hours to develop a fully loaded cost rate. This can be a daunting task, as the implications of being wrong can be significant.

Note that there are widely varying systems that can be employed which lead to different rates for particular pieces of equipment or different labor rates for different departments. As an example, consider a construction and environmental company engaged in measuring water quality and doing underground and over-ground construction. For each of its departments, there should usually be a separate labor rate, as skill levels (pay rates) required for each job are different.

Selecting the complexity of the system should be based upon the company’s existing skill set in developing and applying different rates and upon how much true costs vary by process in practice (e.g. widely varying wage rates for different staff or large, expensive pieces of equipment).

ABC: New Methodology in Controlling Rate Errors

As the market has become more and more competitive, the need to improve Job Costing accuracy and eliminate Rate Errors has increased. The biggest problem in doing an accurate job costing is the allocation and assignation of indirect and support resource costs. Activity Based Costing (ABC) was developed to provide more accurate ways of assigning the indirect and support resource costs to activities, business processes, products, services and customers to improve the accuracy of costing.

ABC systems recognize that many organizational resources are required not for physical production of units of product, but to provide a broad array of support activities that enable a variety of products and services to be produced for a diverse group of customers. The goal of ABC is not to allocate common costs to products—the goal is to measure and then price out all the resources used for activities that support the production and delivery of products and services to customers.

A major advantage of using ABC is that it avoids or minimizes distortions in product costing that result from arbitrary allocations of indirect costs, unlike more traditional line item budgets which cannot be tied to specific outputs. As an example, ABC might assign IT costs to departments that actually use them or it might assign the cost of preparing invoices, processing payment, and collections to each order rather than adding them to hours or allocating them as a percentage of sales. Note that a common finding in ABC systems is that small orders are less profitable than managers previously believed.

ABC systems should normally be implemented only if there are staff on hand that have an understanding of cost accounting concepts.

Controlling Errors of Omission

Errors of omission are prevented by different methods in different contexts. Consider creating standardized forms and checklists for various types of jobs or components, and have them include all of the likely components, so that they cannot be forgotten. Also, maintain a database of standard estimates that have been thoroughly reviewed for accuracy to use as templates in creating new estimates. In this fashion, only the modifications can include item errors. Using details from past jobs is also a helpful tool in ensuring that there are no omission errors.

Feedback and Improvement

A good estimating system closes the loop. Accuracy improves over time if it includes feedback from purchasing, production, and/or the field to improve it. Creating a feedback loop also helps prevent new staff that is training on the new system from replicating the same error repeatedly while no one is watching.

Feedback from purchasing should generally occur as soon as a missing item or quantity error is noticed, so that it can be integrated into the quote and possibly included in other parts of the work order. Operations should provide feedback on errors as soon as they are noticed as well. Finally, a review of the variances on the job should be completed and causes figured out. If they are controllable errors (for example, a piece of equipment broke down), then they generally do not suggest a problem with the estimating system. It is not enough to say that these aspects of feedback should occur. That such feedback is required must be clearly defined in position descriptions. An individual or team should be tasked with performing the variance reviews, and a process should be developed for uncovering the causes of variances.

Otherwise, despite good intentions, there will usually be a lack of follow through that leads to a stagnating estimating system. This usually means not just a lack of improvement, but also decays in the accuracy of the system, as the nature of the jobs being estimated and the internal processes being used by staff changes over time.

Reducing Error Isn’t the Whole Story

The best estimating system is thoroughly integrated into measurement for operations. See our white papers, “Measurement for Manufacturing”, “Measurement for Construction”, and “Measurement in Services” to learn more about how the great information being created by estimating can be used to hold Operations accountable for delivering peak efficiency.

Implementation Guidelines

In our experience, there are a number of things that can be applied to improve the quality of estimates and prevent errors.

Reduce all three kinds of error at the same time. Most important is that the effort should be applied to reduce all three errors together rather than trying to improve only one category of error at a time. The reason for this is that changing one area affects the others. For example, if a company starts to miss fewer things on the estimates and, by getting more accurate at setting hours, increases the average labor hours, it is possible that the labor rate should be reduced to stay competitive.

How far you should go depends on where you start. If your organization has no practical rates and an undeveloped system that is dependent on an experienced estimator, it is probable that the implemented system will be of only moderate accuracy and complexity, since there is a risk of failed implementation if you reach too far. If, however, you have a good system already, aim for a great system – the returns are significant.

Design, then train, then implement, in that order. It is important to start out with a fundamentally developed strategy (plan).

Include checks for larger jobs. The largest jobs should typically include a review by an experienced estimator before a price is given to a customer. Set a clear point at which an estimate must be reviewed and include an initial or other record of that person to create accountability.

Keep some flexibility in the system. Estimators and sales people have a lot of experience and often have a gut feel for how much something will cost to make or how much a competitor will charge. These things should still be considered. The system provides a cost and a target price, but that is not the last word on the right price for each job.

Be prepared to run into a few things that don’t seem to make sense. Sometimes the numbers will seem nonsensical. For example, Bellrock once had a client that found that its negotiated rates for service contracts were actually below its fully loaded cost. In the end, they decided to keep the negotiated hourly rate the same and increased the markups applied to material for those customers. It turned out that the new material charges weren’t that far off what many of its competitors were charging. As you can imagine, profits went up considerably.

Check before the full switch. Errors sometimes find their way into the new design. To prevent this from happening, either estimate jobs both the old way and the new way for a period, or re-estimate old jobs using the new system. Seek to understand why the numbers are different. It may be an error in the new system or the new system may be showing you the better numbers that accuracy can produce.

Next Steps

Changing the estimating system can seem overwhelming. It is not about doing just one thing better – it is about doing a lot of things better in a integrated way that brings together systems for identifying and tracking hours and variances, hourly rates, machine rates, labor utilization and so on. To begin, consider these questions:

  • What has kept us from improving in this regard in the past? Is it expertise? Is it time? Is it our organization’s ability to get things to stick?
  • What are the potential benefits to our company?
  • How will this system link to sales?
  • How will this system link to production?
  • How will this system link to finance?
  • How would we structure the design and implementation initiative?
  • Can we do this ourselves?
To learn how other business leaders are making meaningful advances in this area contact Tara Landes at (604) 345-0424, or by email at [email protected]

Resources

  • ADVANCED MANAGEMENT ACCOUNTING – Robert S Kaplan and Anthony A Atkinson
  • Cost and Effect – Robert S Kaplan and Robin Cooper
  • http://www.bls.gov/oco/ocos006.htm (Overview of cost estimating labor market)