Why Gartner and Forrester Do Not Want TCO Calculated

Executive Summary

  • The last thing most IT analysts like Gartner want is an accurate TCO.
  • This article explains why Gartner prefer an underestimated TCO.

Why IT Analysts Don’t Want TCO Calculated

Gartner, the IT analyst company, was one of the first entities to market around the idea of TCO. However, Gartner never did much more than stay at the conceptual level. The likely reason for this is fairly simple: if Gartner had actually calculated the TCO for the applications it reviewed, it would have given all the wrong answers. I quote from the SCM Focus Press book Gartner and the Magic Quadrant: A Guide for Buyers, Vendors and Investors.

How Gartner Pushes Customers to the Most Expensive Solutions

Gartner also tends to push buyers toward more expensive solutions. This higher expense is not only for software but also for services. For instance, I work as an SAP consultant, and SAP consultants are some of the most expensive resources in IT. One aspect of the cost is the hourly billing rate, which is high. Another aspect is that SAP software is complicated to install, so SAP projects tend to be quite long.

Not only does Gartner not disclose its income from vendors for technology advisement services, but also does whatever it can to prevent buyers from knowing that it takes any income from software vendors. When someone who knows how the system works brings up this issue, Gartner points to the fact that they have an ombudsman, so none of the quite significant revenues they get from vendors has any influence on their ratings. However, this ombudsman follows none of the rules of a traditional ombudsman, and as the complaint process is completely hidden from view, the ombudsman cannot be considered an objective arbiter. This, in fact, was the exact charge in the lawsuit brought about by a software vendor named ZL Technologies, as explained in the following quotation.

Hypothetical Interest in TCO

Hypothetically, IT analyst firms should be good candidates for performing TCO analysis. Gartner, the largest IT analyst firm in the world, is actually credited with first introducing the concept of TCO to enterprise software. However, as has been discussed, most IT analyst firms have major conflicts of interest because they sell consulting services to vendors. They also take considerably more money from the large vendors in comparison to the smaller vendors—and not coincidentally larger vendors tend to charge more for their software, more for their consulting, and so on. The upshot is that the TCOs of large vendors will be higher. Since the vast majority of IT analyst firms take the most money from vendors with the highest TCOs, they have little incentive to provide detailed TCO analyses, as doing so would probably
cause them to lose some of their software vendor consulting revenues. Also, different IT analyst firms have different interpretations of TCO. Gartner, for instance, tends to be “pro-TCO,” while Forrester tends to be “anti-TCO,” proposing that most companies cannot aspire to TCO successfully and that simpler approaches should be used. Forrester also points out that, according to their surveys, only about twenty-two percent of purchasing companies use any type of TCO in their decision-making.

Is TCO Even Possible?

Several entities disagree on whether or not a full TCO analysis should be a goal. In this section, we will review the concerns leveled frequently at TCO. One of the
entities—which could be labeled as anti-TCO and which is influential in the area of enterprise software decision-making—is Forrester, the IT analyst firm.

“To really implement TCO-based analysis it takes a comprehensive and continuously updated catalog of asset inventory, in-service dates, agreedupon
operating cost rates for activities, and a scheme to divide shared costs among the constituent business processes that use them. For most firms, this is a pipe dream viewed either as a waste of resources in a futile quest for achievement or too intimidating to even begin. Forrester recommends a more expedient and realistic financial approach that can be just as effective but much simpler to calculate—relative cost of operations (RCO). RCO can be a middle-ground solution, moving far beyond acquisition-cost-only analysis, while being more achievable than a full-blown TCO.” — Forrester

I agree with Forrester’s assertion that companies rarely use TCO, but I do not see why performing a complete TCO analysis is beyond most companies if they are properly advised. We have performed a wide variety of TCO analyses for software categories and make these analyses available for a fee at our sub-site. Because we take no money from vendors, we can legitimately say that our TCO analyses do not have a bias. One could always propose other non-financial biases—and these are possible and also come down to whether you consider exposure to a topic to be a bias or to be knowledge (babies, after all, are completely unbiased and open-minded)—however, if one analyzes the major problem with respect to objectivity in analysis in not only the enterprise software arena, but also other areas such as financial advisement —consistently the problem is financial bias. This is true to such a degree that once financial bias is removed one has taken care of the vast majority of the problem. Forrester is not alone in their concerns regarding TCO. The white paper Rethinking TCO takes a similar view of TCO as Forrester. Rethinking TCO points out some of the limitations of performing a meaningful TCO analysis when they state:

The large amount of variability in product complexity and comprehensiveness from one enterprise vendor is an important limitation on the potential for using TCO in multi-vendor analysis.”

However, this type of calculation will be performed either implicitly or explicitly (with an actual TCO); therefore, it is difficult to see how the approach of not performing the analysis is better than performing a TCO, especially considering what is at stake. In fact, merely performing the analysis—imperfect as it may be—puts
a company in a more analytical frame of mind during their selection process.

Customized Versus Noncustom TCO

“…different companies within the same industry may have significantly different business processes expressed in the same software product, or may have extensively customized a standard software package in order to gain some degree of competitive advantage. Indeed, this variability in use often represents a key strategic value for the enterprise software package: by using a standard software product in a nonstandard or customized fashion, many customers hope to gain competitive advantage over other companies in their industries that may be using the same or similar software.”

That is all true. However, this is where the implementation experience of individuals performing the TCO comes into play. If they are sufficiently experienced, they
can increase or decrease the cost of a particular item through their knowledge of how much customization can be expected on the project. Furthermore, a noncustomized TCO value probably would not make a lot of sense. It appears that this is an argument against non-customized TCO calculations.

Implementation Costs as a Percentager of TCO

“The implementation process is another major factor that adds to the problems with using TCO as a vendor selection tool. Implementation costs figure as one of the largest single expenses in enterprise software, and yet they are neither standardized nor consistent from one vendor to the next or one implementer, or implementation, to the next.”

While true, this can easily be accounted for in the TCO estimation, so it is difficult to understand the exact concern. Yes, implementation times can be adjusted for TCO estimations. And that is not the end of it; maintenance is quite different from one software vendor to the next and we adjust the maintenance costs per software vendor.
We account for all of these differences in the TCO estimation. For example, if an implementation is from a large consulting company, we adjust the estimate accordingly. If the implementation involves a large brand-name software vendor, we again extend the implementation timeline. For example, when we develop estimates for software vendors like SAP (which is neither designed to be easily implemented or easily used), we impose higher costs than any other software in the enterprise market. SAP takes longer to install, which means implementation costs are high, as are maintenance costs. This is why it is so important to check that the software functionality is reliable and usable; not only is the application more effective, the cost of the implementation is reduced, thus reducing its TCO and increasing its ROI. This goes back to a central theme of this book: without quantification through something like a TCO process, reliability and usability may be presumed to be equivalent between various software applications, when, in fact, they are quite different. Strangely, no one seems to bring up the issues that arise if costs that are assumed to be equivalent end up being quite different, making it difficult to make good decisions.

Gartner’s Fake Ombudsman

Gartner currently has an employee act as an ombudsman to handle disagreements. The conflict of interest is self-evident in the way ZL’s concerns were summarily dismissed with little supporting evidence. There is a crying need to establish an impartial ombudsman similar to those found in public media, in order to ensure purchasers that they are receiving impartial analysis.

My conclusion, which is the exact same conclusion of every director or vice-president of marketing at a best-of-breed vendor with whom I have discussed this topic, is that payments from vendors influence Gartner’s ratings substantially. Not one best-of-breed vendor I have ever spoken to thought Gartner was an unbiased source of information on software vendors. Not one.

Conclusion

The system is rigged in favor of vendors that pay the most and the vendors that can afford to pay Gartner the most are those with the highest license fees. Therefore, one of Gartner’s main roles is to guide buyers to both the most expensive TCO software solutions in the market, as well as to the most expensive consulting companies. Gartner could not perform an honest TCO analysis for any of the software categories that it covers without alienating its biggest vendor customers. Therefore, Gartner has no interest in TCO beyond discussing it as a general principle.

Custom TCO Estimates and Consulting

  • Want Help with TCO for your Business?

    It is difficult for most companies to estimate TCO without outside advice. Vendors and consulting companies do not want their customers to know what they TCO is. Getting TCO advice from consulting companies leads to underestimated TCO. We do offer remote unbiased multi-dimension TCO estimation.

    This article is free, we do not answer questions for free. Filling out this form is for those that have a budget. If that describes you, just fill out the form below and we'll be in touch asap.

References

TCO Book

TCO3

Enterprise Software TCO: Calculating and Using Total Cost of Ownership for Decision Making

Getting to the Detail of TCO

One aspect of making a software purchasing decision is to compare the Total Cost of Ownership, or TCO, of the applications under consideration: what will the software cost you over its lifespan? But most companies don’t understand what dollar amounts to include in the TCO analysis or where to source these figures, or, if using TCO studies produced by consulting and IT analyst firms, how the TCO amounts were calculated and how to compare TCO across applications.

The Mechanics of TCO

Not only will this book help you appreciate the mechanics of TCO, but you will also gain insight as to the importance of TCO and understand how to strip away the biases and outside influences to make a real TCO comparison between applications.
By reading this book you will:
  • Understand why you need to look at TCO and not just ROI when making your purchasing decision.
  • Discover how an application, which at first glance may seem inexpensive when compared to its competition, could end up being more costly in the long run.
  • Gain an in-depth understanding of the cost, categories to include in an accurate and complete TCO analysis.
  • Learn why ERP systems are not a significant investment, based on their TCO.
  • Find out how to recognize and avoid superficial, incomplete or incorrect TCO analyses that could negatively impact your software purchase decision.
  • Appreciate the importance and cost-effectiveness of a TCO audit.
  • Learn how SCM Focus can provide you with unbiased and well-researched TCO analyses to assist you in your software selection.
Chapters
  • Chapter 1:  Introduction
  • Chapter 2:  The Basics of TCO
  • Chapter 3:  The State of Enterprise TCO
  • Chapter 4:  ERP: The Multi-Billion Dollar TCO Analysis Failure
  • Chapter 5:  The TCO Method Used by Software Decisions
  • Chapter 6:  Using TCO for Better Decision Making

How It Works

How It Works

Each TCO calculator is self-service allowing you to continually change different elements in order to see the impact on costs. They are designed to adjust to the specific project factors such as the number of users, the general level of customization, the number of post go live adjustments to the application, etc..

The TCO calculators can improve your ability to plan your purchase.

How It’s Unique

How It’s Unique

Our self-service calculators have been developed through detailed analysis verified by many years of project experience combined with all the available research – all in order to develop a series of uplifts to costs based upon inputs. The formulas used are nuanced, and do not simply “scale” in direct proportion with changes to the inputs.

  • Our TCO calculators are designed to scale to any sized implementation and different levels of implementation complexity and customization.
  • We offer a true TCO by estimating internal costs (such as the time spent by internal resources on implementation and support) as well as the external costs. In comparison with the very limited TCO studies that exist on enterprise software, our TCO calculations are easily the most comprehensive.
  • Having performed this analysis for many applications, we have brought key observations between these applications as well as between various software categories.

What Is Included

What Is Included

Each package is a combination of two analyses. The first analysis is the interactive TCO Calculator which is provides a total TCO based upon the individual costs of software costs, hardware costs, implementation costs, maintenance costs as well as Lifetime Improvement Costs (the costs of the estimated improvements and adjustments to the application over its lifetime). Both these individual component costs as well as the aggregate of all the costs constantly change given your input to the calculator.

What It Is

What It Is

This offering provides buyers with the detailed information they need to for both the total cost of ownership of a single application, as well as the comparative total cost of ownership between multiple applications. This is the only self service TCO calculator that exists on the Internet, and it is available currently for 57 of the most well known enterprise software applications. This calculator receives input from you and automatically adjusts the costs so that they are customized for your intended way of implementing and using the software.

Transforming a Complex Analysis into a Simple Cost Breakdown

Even though all of the calculations behind each TCO calculator are complex and have been extensively tested and validated, they are easy to use. All that you need is basic information about your project such as the number of projected users, whether the implementing is more simple or more complex, etc..

Each package covers a single application including a comprehensive total cost of ownership analysis that takes into account the following costs:

  • Software Costs
  • Hardware Costs
  • Implementation Costs
  • Maintenance Costs
  • Lifetime Improvement Costs