Don't Forget the "e" in IT

Jerry Krawczyk, Director, Information Technology Services, The Pennsylvania State University

Universities face constant budgetary stress driven by increasing costs and pressure to control tuition increases; state institutions are also experiencing reductions in state support. At the same time, university IT departments are seeing increasing demands for additional IT services that are easily and readily accessible and driven by the institution's educational and business needs. These include demands for learning management systems, collaboration environments, online instruction, MOOCs, and so on.

As technologists, we tend to focus predominately on the technical or IT side of services. However, an equally important side exists: the "e" in IT — that is, applying economic analysis to IT portfolios and investments as part of decision-making, governance, and strategy processes.

Economic analysis might entail one or more of the following:

• Cost analysis: A formal estimate of projected cost over the useful life of the proposed solution, including direct costs, indirect costs, and future escalations.
• Cost-effectiveness analysis: A cost analysis of alternative solutions with similar outcomes or benefits.
• Cost-benefit analysis: A methodology for comparing economic pros and cons by quantifying benefits.
• Business-case analysis: A structured proposal — often including one or more of the previous analyses — that presents the logical business argument for undertaking a particular course of action, be it a proposed IT investment, service change, or service retirement.

As I now describe in more detail, given constrained budgets and increasing demand for IT services, the "silent e" should not be forgotten.

The Value Proposition: Why?

Why should IT leaders focus additional effort on developing a sound cost-benefit analysis and associated business-case analysis? As I now describe, doing so is a wise investment for three key reasons.

Decision Making and Demonstrating Value

University IT services are traditionally viewed as a cost center or overhead expense. It has thus been challenging to demonstrate their value to the "business."¹ A well-crafted business-case analysis, including a comprehensive cost-benefit analysis, will illuminate not only an IT service's costs (which can be used for budgeting purposes) but also the benefits that help identify its business value. Correctly conducting a cost-benefit analysis lays the foundation for improved IT visibility and transparency, and it reinforces the value IT brings. Also, with the growth in cloud-based IT services, a business-cases analysis is necessary to make objective "make vs. buy" or "on-premises vs. cloud" decisions. A recent Gartner report, One More Time: This is How You Express Costs in Business Value Terms,² echoes this idea.

It's a Best Practice

The Information Technology Infrastructure Library (ITIL) and the Project Management Body of Knowledge (PMBOK) are widely accepted sources for best practices within their respective disciplines and are being embraced throughout higher education. Both ITIL and PMBOK refer to the need for and use of a business-case analysis as part of a project's or service's initial strategy and decision-making process.

• Within the ITIL Service Strategy volume is an entire process, called financial management for IT services,³ that focuses on cost-modeling considerations. In addition, ITIL inputs a cost-benefit analysis or a more comprehensive business-case analysis to other ITIL Service Strategy and ITIL Service Design processes.
• Within the PMBOK, one input to the initiating process group (which includes activities to kick off a project) is the business case: "The business case or similar document describes the necessary information from a business standpoint to determine whether or not the project is worth the required investment."⁴

Measurement and Monitoring

Developing an economic analysis is not only a best practice, it also provides a foundation for monitoring project accomplishments and assessing service value. For example, PMBOK devotes an entire chapter to project cost management and outlines the benefits, processes, and techniques. It is impossible to do an earned value (EV)⁵ analysis without working from a comprehensive cost analysis. From a service management perspective, a well-developed cost-benefit analysis is needed to demonstrate whether investing in a new service or changing an existing service can actually deliver the expected business value.

The Challenges

Given that IT leaders have endorsed economic analysis as a value and a best practice, why aren't more organizations conducting comprehensive business-case analyses? The most common arguments I hear against doing a cost-benefit analysis are that "we can't estimate the costs or benefits" or "it will be too hard to do." In my opinion, these are fallacies. Although developing a cost-benefit analysis can be time consuming and quantifying benefits can be challenging, going through the process produces higher quality and more comprehensive planning.

Part of the challenge lies in the IT community's logical, scientific, and risk-averse culture. After all, developing a robust economic or cost-benefit analysis is part art and part science — and the art aspect makes engineers and technical IT staff a bit uncomfortable. Developing a cost-benefit analysis requires that we predict the future (the art side of the equation); this entails risk and uncertainty — attributes not always welcomed during strategic decision-making activities. However, as I describe later, ways to "embrace the uncertainty" exist.

Additionally, if an IT leader is set on a solution and has not taken the time to assess its true costs and benefits, conducting an objective cost-benefit analysis might lead to an awkward conversation with stakeholders if that analysis does not support the chosen approach. The last thing IT leaders want to hear is that their decision was wrong.

Finding representative data and information is also challenging. Assessing labor expenses, for example, can be difficult if a university does not track staff time dedicated to a service, project, or activity. Assessing and quantifying benefits is similarly challenging. However, the need to invest time and resources into finding or developing relevant data — such as cost estimating relationships (CERs) — should not be a deterrent.

Recommendations for Moving Forward

Following are a set of thoughts and recommendations, based on my personal experience, to help bring the "e" into your IT.

View Cost Estimating/Analysis as a Competency

Within IT's technical world, having the appropriate expertise and experience is critical for success. As with many IT disciplines, economic analysis and cost analysis is a discipline itself and should be recognized as such. For example, technical certifications by Microsoft, Cisco, and so on are often sought out and recognized. Organizations adopting IT service management will seek or develop staff members with certifications in ITIL (such as ITIL Foundations through the ITIL Expert Certification). Likewise, organizations focused on project management often seek staff with the appropriate professional certifications (such as Project Management Professional). The same should be true with economic and cost analysis.

The International Cost Estimating and Analysis Association (ICEAA) is a governing body in cost estimating and analysis. The ICEAA has published the Cost Estimating Book of Knowledge (CEBoK) and award candidates the Certified Cost Estimator/Analyst (CCEA) professional certification upon successfully demonstrating professional experience and passing the CCEA exam. After receiving the CCEA, members must maintain professional development units (PDUs) through training and other activities.⁶

Establish Threshold and Triggers

Given the time and effort involved, not every IT investment decision needs a full-blown business case or a cost-benefit analysis. However, as part of an overall governance process, it is important to establish a policy or trigger as to when certain economic analyses are required.

For example, for low-risk, low-dollar decisions, conducting a simple cost analysis should be all that is required for budget approval and authorization to proceed. For medium-risk, medium-dollar decisions that might involve alternative solutions or delivery methods (such as "on-premises" versus "cloud"), conducting a cost-effectiveness analysis might suffice if the anticipated benefits are expected to be the same. For high-risk, high-dollar IT investment decisions, a business-case analysis and a supporting cost-benefit analysis should be part of the decision-making process. Funding or approval requests without a business-case analysis should be rejected. In order to be effective, the governing body, with budgetary authority, must establish a policy identifying the thresholds of risk and dollar amounts, associated cost analysis and ensure the policy is enforced.

Embrace Uncertainty

If there is one thing I'm certain about, it is that a point-estimate — such as, the five-year cost of ownership will be $500,000 — is almost always wrong. As I noted earlier, estimating costs and benefits is partially an art. So, when developing a cost-benefit analysis, you must embrace the estimates' uncertainty. For example, in addition to developing the individual estimates, you should also develop an associated or risk distribution. Tools and spreadsheet add-ons exist that make applying this methodology easy to utilize and manage. Two such tools I've had success with are @Risk and Crystal Ball, both of which use Monte Carlo simulation⁷ and produce estimates with associated ranges. They also identify the variables or drivers that most influence the results.

Most people tasked with economic decisions understand that the further out a decision is, the less accurate the projections will be. Attempting to make future costs pinpoint-accurate calls into question the entire cost analysis. However, identifying the drivers provides an area of focus for future risk mitigation. So, while a point estimate is almost always wrong, a probability estimate is almost always right (within the margin of error).

Leverage Cost-Estimating Best Practices

The Internet has numerous references and information on best practices and methodologies. In my experience, the U.S. Department of Defense has some of the best material available.⁸

Some best practices worth highlighting include the following:

• Develop a comprehensive work breakdown structure (WBS) with all aspects of the project or service, including one-time acquisition, recurring/maintenance costs, training and travel expenses, and fully burdened labor.
• Invest the time to define the requirements and analysis scope, gather data (direct and indirect elements), develop CERs where data doesn't exist, and document all of the assumptions.
• Develop appropriate financial measures and thresholds that can be used during the project or service lifecycle to demonstrate whether the proposed business value is actually realized.

Conclusions

I resonate with the Gartner report, Run IT as a Business Using Six Pillars of Effective IT Financial Transparency,"⁹ which emphasizes the need for IT budgeting, investment planning, chargeback/showback, benchmarking, cost optimization and performance metrics. Leaders and decision makers must recognize the value that an economic analysis will bring to the decision-making process. They can then integrate the economic analysis into existing processes and identify consultants (or develop the staff) with the right set of competencies, balanced between economic analysis techniques/methodologies and understanding IT. Although developing a cost-benefit analysis or a broader business-case analysis requires an investment in time and resources, having such robust and well-articulated analyses is critical for making strategic IT investment decisions.

1. Phil Goldstein, Richard Katz, and Mark Olson, "Understanding the Value of IT," EDUCAUSE Quarterly, vol. 26, no. 3, 2003, pp. 14–18.
2. Jim McGittian, Kurt Potter, and Michael Smith, One More Time: This is How You Express Costs in Business Value Terms, research report G00260050, Gartner Group, March 28, 2014.
3. David Cannon, "ITIL Service Strategy," ch. 4.3, ITIL 2011 Edition, The Stationary Office, 2011.
4. A Guide to the Project Management Body of Knowledge (PMBOK Guide) 5th edition, Project Management Institute, 2013.
5. Ibid, ch. 7.4.2.1; earned-value management is a methodology that combines scope, schedule, and resource measurements to assess project performance and progress.
6. I'm a former holder of CCEA certification and made the mistake of not keeping my certification credentials current.  While I still retain the knowledge and experience, in order to be recertified I would need to complete the certification process.
7. Monte Carlo simulation is a problem-solving technique that lets you approximate the probability of certain outcomes by running multiple trial runs, called simulations, using predefined independent variable distributions.
8. The DON IT Resource, "DON IT BCA and Abbreviated BCA Templates," US Department of the Navy, May 31, 2013; and US Army Cost Benefit Analysis Guide [http://asafm.army.mil/Documents/OfficeDocuments/CostEconomics/guidances/cba-gd.pdf], Office of the Deputy Assistant Secretary of the Army, April 24, 2013. See also GAO Cost Estimating and Assessment Guide and Department of Defense, Cost Analysis Guidance and Procedures [http://www.dtic.mil/whs/directives/corres/pdf/500004m.pdf] (DOD 5000.4-M).
9. Jim McGittigan, Run IT as a Business Using Six Pillars of Effective IT Financial Transparency, research report G00238696, Gartner Group, April 9, 2013.