Strategies for Recovering the Costs of the Campus Data Network Copyright CAUSE 1994. This paper was presented at the 1993 CAUSE Annual Conference held in San Diego, California, December 7-10, and is part of the conference proceedings published by CAUSE. Permission to copy or disseminate all or part of this material is granted provided that the copies are not made or distributed for commercial advantage, that the CAUSE copyright notice and the title and authors of the publication and its date appear, and that notice is given that copying is by permission of CAUSE, the association for managing and using information technology in higher education. To copy or disseminate otherwise, or to republish in any form, requires written permission from CAUSE. For further information: CAUSE, 4840 Pearl East Circle, Suite 302E, Boulder, CO 80301; 303449-4430; e-mail info@cause.colorado.edu Strategies for Recovering the Costs of the Page 1 Michael Hrybyk Campus Data Network CAUSE93 Strategies for Recovering the Costs of the Campus Data Network Michael Hrybyk The University of British Columbia University Computing Services Central Networking Michael.Hrybyk@ubc.ca 1. Introduction This paper explores the methods an institution might use to recover costs associated with an internal campus data network. Choosing and implementing such a strategy is becoming more important as data networks grow in size and complexity. Such growth brings increasing operational and management costs. Pricing network services is an extremely delicate task. If not done correctly, network growth could be stifled, or, alternately, over built and misallocated. 2. Background Approximately two years ago, the University of British Columbia changed the way service units derived their operating budgets. Instead of relying on central funding from the President's office, the service units would be paid for services rendered by individual schools, departments, and other functional groups on campus. This represented a major departure from prior budgetary policy. Luckily, the transition was to be staged rather than occurring overnight. The term "cost recovery" has become synonymous with this type of decentralized budgetary scheme. That is, selected support services recoup their costs from other groups within and outside of the campus community. Institutions are beginning to explore cost recovery for data networking for several reasons. First, it introduces an element of market forces into the decision-making process with regard to allocation of information technology resources. Second, it holds the promise of bringing costs under control, and increasing efficiency. Expenditures are made only if the users of related services demand them. Whether these advantages accrue in reality lies outside of the realm of this paper, but is an extremely relevant topic. The UBC Computing and Communications unit was given the mandate to operate using decentralized funds as soon as possible. The telecommunications group was already cost-recovered. Telephone users paid monthly and long distance fees, which included internal staff and overhead costs. This example is often cited internally at UBC as a successful cost-recovered organizational unit. Other units within C&C were given the task of becoming cost- recovered. In particular, the MIS development group started operating on a contractual arrangement with clients. The Academic Systems group started to charge for UNIX computer usage. Use of the modem pool was billed at $1.20 per hour to individual users. Well over 50% of the C&C budget is currently derived from cost recoveries and decentralized funding sources. The Central Networking group, with a mission of designing, monitoring and managing the campus backbone, was given the task of cost-recovering its activities as well. Unfortunately, little was known as to how to proceed in this area. Augustson, in Gillespie(1989) proposed three criteria for determining whether computing services should be cost-recovered. First, the service should be strategically important to the university and its competitive position. Second, the service should be emerging, immature, or innovative. Finally, the service should be viewed as infrastructure. By these criteria, data networking would seem to be a perfect candidate for central funding. The UBC Data Networking Task Force (Tom et al(1992)) took a slightly different position. The task force noted that by not allocating the costs of the network to users fostered the notion that it was in fact free. Of course, backbone networks are very expensive, and some method for cost allocation as well as market feedback was necessary. The task force suggested that capital costs of the network be taken from central funds, with operating, management, and maintenance costs passed on to individual units. With scant literature on the specific topic of data-networking cost recovery in the university environment, Central Networking began to search for a method of implementation. First, a business plan was formulated, defining a market and proper vehicle for delivery of backbone networking services. Second, an advisory group of departmental network administrators was formed. One task of the advisory group was to outline possible schemes for backbone cost recovery. Finally, Central Networking undertook a survey of academic institutions which indicated that they cost-recovered data networking wholly or in part. The indication was obtained from the CAUSE on-line database at cause.colorado.edu. This paper will be limited to discussion of the survey methods and results, presented in the sections that follow. 3. Methodology A survey questionnaire was developed to determine the different strategies used to cost recover central data networking services. The questionnaire is included in Appendix A. The questionnaire was constructed with two purposes in mind. First, background information regarding the respondent's institution was solicited. The questionnaire asks for the size and type of the network and whether services are provided to the wall plate. Second, and most important, the respondents were asked to describe the method(s) for recovering networking costs, and to what extent the method(s) were utilized (i.e., percentage of budget recovered). The questionnaire was mailed electronically to institutional contacts in the early part of May, 1993. A second, follow-up mailing was done in the last week of the same month. The responses were classed according to types, based on both budget recoveries and method employed. 4. Sample The sample consisted of approximately 90 institutions indicated either partial or full cost-recovery of data networking according to the profile in the CAUSE database. The database was located at the Internet site cause.colorado.edu. The report from the database indicated the institution name, the contact, and whether services were cost recovered. Email addresses for each contact were determined by querying the member database at cause.colorado.edu. Roughly 5% of the institutions listed in the initial report had contacts with invalid or missing email addresses. The questionnaire was mailed to 90 institutions. Responses were obtained from 28, or 31%. 5 responses were not usable. Either the site did not cost recover data networking, or the responses to most questions were in large part missing. Thus, the final sample size was 23. Table 1 contains a summary of the survey responses. It also shows some characteristics of the sample. 54% of the sites had networks comprised of 2000 nodes or more. The average number of networked nodes per institution was 3440. All budgets were $200,000 US or greater. The average number of subnet segments per site was 86, although the variation was quite large. 5. Results Five strategies were found to be used by the respondents in the sample. Each is described below. It should be noted that none of the respondents used any type of per-packet or volume-based charging. Updegrove's Penn State summary, found in Appendix B, makes a cogent case against such a policy. The fact that no volume-based charges were levied by any of the the institutions came as a surprise. Mixed service offerings. This strategy offered the widest variety of products and accompanying charges. Users arranged for simple backbone connections via a wide range of media, or opt for per- node wall plate service. The fee structure was somewhat flexible, depending on the type of service desired. Per-node fee structures. Each node on the network is assessed a monthly or yearly fee. Usually this implies that service is supplied to the wall plate. Although departments might obtain bulk discounts for large number of nodes, the fee charged is essentially fixed, regardless of the service type. One-time Installation Fees. Costs are recovered only when a network port or subnet is installed for the first time. No recurring charges are assessed. Telecom Subsidy. Network service costs are hidden within a cost- recovered telecommunications budget. The income from voice services subsidizes the data network. Computing Services Subsidy. Network service costs are hidden within the general computing services budget. The income from email and cpu utilization charges subsidizes the data network. Table 1 below summarizes the survey results, categorizing respondents by type. Network characteristics are shown. Nodes indicates total number of network endpoints. Segments indicates the number of segments or workgroups within the network. Management shows whether responsibility extends to the communications closet (CC) or the wall plate (WP). Yearly operating budget, in $1000s, is listed. Finally, the percentage of the operating budget that is cost recovered is shown. Within each category, institutions are rank-ordered by network size (node count). Table 1 Cost Recovery Strategies and Network Characteristics Nodes Segmts Mgmt Budget % Rec Comments ($1000s) Mixed Services Univ Wyoming 1400 50 WP 350 100 Univ Texas HSC 1450 70 WP 200 70 Penn State 4200 300 CC 7000 100 UC Davis 4500 140 CC 600 20 Moving to full rcvry Per-node Charges Fort Lewis Coll 134 20 WP 200 22 UC Irvine 4500 180 WP 500 50 % Recvry estimated Harvard 17500 200 WP 1000 100 Fees are 50% Installation Fees Queensland Ins AU 61 WP 1600 10 Macalester 415 6 WP 800 10 Univ Nebraska 1725 76 WP 2900 100 Kent State 2550 8 CC 200 100 Univ Colo HSC 2650 250 WP 2500 100 Univ Kentucky 2900 55 WP 700 100 Univ Arizona 6210 100 WP 2000 10 Univ Georgia 8235 80 WP 1000 10 Eastern Wash 10 Telecom Subsidy West Michigan 837 22 WP 300 100 40% from install fee Grand Valley 870 10 WP 300 70 Baylor 2690 30 WP 2000 100 50% from install fee South Ill Edw 2050 35 WP 300 100 10% from install fee Univ New Hampsh 4100 30 WP 400 100 35% from mixed svcs Computing Services Subsidy Dallas CC 2000 WP 400 100 Uses email fees U Cincinnati 2650 50 WP 500 100 15% from install fee Average 3503 84 1170 69 Std Dev 3671 81 1491 39 6. Analysis Reliability and Validity. The validity of some of the responses is questionable. In particular, the budget figures should be considered carefully. Most respondents had difficulty determining the exact budget for central data networking. The sites with the best budget estimates are most likely the largest ones, which have cost centers dedicated to the data network. This makes it simple to determine the budget size. Second, the budget question did not asked to specify capital versus operational costs. The two were sometimes mixed by some respondents, while others just gave operational costs. Network size estimates probably contain some error, but most seem to provide the figures with confidence. Segmentation estimates vary, as the exact definition of the term is hard to pin down. Sites with bridged networks will report few segments. Still, most of the sites use routed networks, such that the segmentation estimate holds some validity. Charging Model. Clearly, the two most prevalent models were installation charges and service subsidy. 69% of the responses fell into these categories. Only one major university, Harvard, completely recovered costs using per-node charging. Three other major universities used the mixed service model. 7. Discussion It was hoped that the survey would have produced larger variation in the types of solutions. Unfortunately, there were few real attempts to treat data networking as an actual cost center. This is probably due to the newness of the technology, and the propensity to treat the network as infrastructure overhead rather than a service. Using installation fees as the sole basis for cost recovery seems untenable over the long run. In fact, the largest cost over time is more likely the operations and maintenance charges for keeping the network running. If one has to front load these expenses entirely at installation time, the users would be disinclined to join the network, and growth would slow. Installation fees are acceptable if one makes a speculative assumption. Assume that technology is consistently changing, and that by the time the network reaches saturation, a new technology, requiring new hardware and installation schemes, will arise. This guarantees a steady income stream. Unfortunately, this is a difficult assumption upon which to build a source of revenue. The subsidy models (central computing or telecommunications) use older technologies as an income stream to support up the data networking services. This works in the early phase of network startup, but can present problems if the old source of revenue dries up, or the user demand for networking services sharply increases. This scheme also inflates the prices for voice or central computing, making alternative technologies or third parties competitive, and possible erosion of the market. Overcharging on voice to pay for data might lead users to look for alternatives. However, distributed services such as email, directory services, CWIS, etc., might provide a future income stream that might subsidize the basic network infrastructure. However, like the installation fee scheme, the subsidy model is not likely tenable over the long term. 69% of the respondents indicate using the schemes above, which do not hold out promise for the future in a decentralized, cost- recovered environment. Two models, per-node charging and mixed services, are used by Harvard and Penn State respectively. These sites are leading the way in showing how to operate data networks in an atmosphere of decentralized funding. An edited questionnaire response from Penn State's Steve Updegrove is contained in Appendix B. The response from Harvard's Steve King can be found in Appendix C. The mixed services model seems to make the most sense. At UBC, we have asked the network administrators for input on this issue. Most felt that if we had to charge for network services, a market basket approach was the best. This allows the departments the largest amount of flexibility. Some departments may want to install their own LANs to their own specifications (following standards, of course), and may only want a backbone connection. Others, with less technical expertise, might opt for to the wall plate service, with monthly charges per port. This scheme seems to follow the one currently followed on the Internet, whereby one can place (theoretically) unlimited number of nodes behind a router connection. Charges accrue on the basis of bandwidth allocated, and the cost of equipment and management. The mixed service model has another advantage. Providing a market basket of services encourages the organization to broaden the offerings in order to achieve large penetration. One can establish "soft" network services, such as distributed printing, email, and conferencing. All of these depend on the network infrastructure directly, and can contribute funding as overhead. Such "soft" service offerings can foster network growth by adding to the traffic mix, and providing a revenue stream for upgrading as necessary. The mixed model also adapts better to new technologies. One can envision setting up a wireless hub/subnet. How is one to determine the number of nodes behind this hub? Registration is obviously necessary, but the problem of enforcement can be daunting. The per-node model is also usable. It scales the income stream to the size of the network, allowing for orderly capacity planning. However, it only works if the institution has full control to the wall plate. Speaking from experience with users at UBC, clients will balk at paying per-node fees if they are only using a backbone connection to their own custom LANs. If most LANs and requisite internal wiring are under central control (like the telephone system), then per- node charging makes sense. It is related to the way one charges for telephones - i.e., per circuit. In summary, offering a market basket of services, tailored to the needs of the users probably makes the most sense. It allows the user to fashion a LAN to their liking, with the ability to purchase extra services as necessary. Our users at UBC seem to prefer the approach, and Penn State has provided an excellent example as a starting point. 8. References Gillespie, R.G. Chargeback Revisited, in Brian L. Hawkins (ed.), Organizing and Managing Information Resources on Campus, EDUCOM, 1989. Tom, J., Demco, J., Dolchewski, J., Gaudreau, J., Hall, R., McWilliam, D., and O'Reilly, D. Planning for the Data Network: A Discussion Paper. The University of British Columbia, December, 1992. Appendix A Survey Questionaire Dear fellow CAUSE member, We are conducting a survey to determine the various methods of recovering costs for data networking at academic institutions. We decided to survey CAUSE member institutions identified in the CAUSE member database as having implemented a cost-recovery mechanism for data networking. This survey is being distributed to roughly 100 CAUSE contacts from those institutions. The purpose of the survey is to inform our own planning process. We will make public (possibly presenting a paper at the next CAUSE conference) our findings to assist others struggling with the same thorny issues. Please take a few moments to fill in the form below and email it back to me. Since the sample is small, a reasonable percentage is necessary to obtain valid results. Please respond no later than May 14, 1993 to be included in the survey. Hopefully, the form should take no more than 15 minutes to complete. Thanks in advance for your time. Michael Hrybyk University of British Columbia University Computing Services Central Networking hrybyk@netcom.ubc.ca --------------------------------------------------- Data Networking Cost Recovery Questionnaire Name: Title: Institution: Department: 1. Are central data networking services provided (e.g., campus backbone)? Please describe *briefly*. [e.g., network management, DNS, Email ] 2. Please *briefly* describe the campus network: type, topology, protocols. [e.g., inverted fibre backbone, ethernet, 10 based-T LANs, TCP/IP, Appletalk, DECNET ] 3. Please give the number of total network endpoints (computers, peripherals). Break this down by Number of Intel-based PCs/servers: Number of Mac-based PCs/servers: Number of RISC-based workstations/servers (SUN, HP, RS6000, ...): Number of network-attached printers: Other: 4. Please provide the approximate number of network devices (routers, bridges,hubs) managed by your networking organization. 5. How many subnets/workgroups are connected within the campus network? 6. How far does your responsibility for network management and service extend? [ ] To the building communications closet. [ ] To the building floor closets (including risers) [ ] To the wall-plate. [ ] Other: 7. How are data networking costs recovered? Please mark the percentage of the data networking budget covered by each. a. Individual user fees per unit time: % [where such a fee is a charge per user per month] b. Network endpoint fees per unit time: % [where such a fee is a charge per workstation/PC per month] c. Subnet/workgroup fees per unit time: % [where such a fee is charge per LAN per year ] d. Overhead fees charged to other service offerings: % [i.e., internal charges to other computing or network services] e. Lump sum payments by faculties, schools, or departments: % f. One-time network endpoint installation fees: % [where such a fee is a charge per PC network hookup ]. g. One-time LAN installation fees: % h. Per-data-packet or other volume usage fees: % i. Grant funding: % j. Other: % 8. Please *briefly* elaborate on the cost-recovery methods listed in the previous question. In specific, how are central facilities charged versus local LANs? 9. What is the approximate size of the budget for central data networking services? 10. Does your organization cost recover academic and administrative computing? 11. Would you like to receive an electronic copy of the survey results? Please provide an email address for delivery if different from the one in the email header. Again, thank you for your time. Appendix B Case Study Pennsylvania State University The response from Penn State was very thorough. It is presented here in its entirety. From: Steve Updegrove Administrative Director Office of Telecommunications Penn State University The Office of Telecommunications is responsible for video, data, and voice communications services, and is organizationally part of the Office of Computer and Information Systems, which in turn reports to the Provost of the University. The data network is rapidly growing and dynamic within the 23 campuses that comprise Penn State. Our philosophy is to encourage individual colleges and departments to add to their LAN's without centralized "policing" of number of devices We administer a University-wide data backbone service, which includes the responsibility for centralized network management, all facets of providing, maintaining, and perpetuating the functions represented by the various types of equipment and services which are a part of that (including inter- and intra- LATA T-1 and 56K circuits, packet switching nodes, routers, and distribution cabling), and assisting each campus, most colleges, and numerous administrative departments in design of their LAN's. Email and other services used by those who have been networked are provided centrally by other divisions within our parent organization, or within the context of the attached networks. As the client-server model of computing matures, the nature of these are changing, but there will be some form of central data networking services, regardless of how one defines that, provided for a long time to come. Standards of all natures have been defined as strategically desireable. Among these are the use of TCP/IP as the data backbone protocol. However, in response to pressure by DECnet users, a policy has been developed requiring us to continue support of DECnet for at least a period of one year after DECnet Phase 5 is fully announced, providing a transition time for those users to move to the OSI suite. We currently tolerate tunnelling of Appletalk. Even so, periodic spurts of interest arise in supporting Appletalk in the native mode, despite its proprietary, non-standard nature. To date, the additional costs to manage a separate logical Appletalk network have not been justified by the set of Appletalk features that cannot be supported by TCP/IP, but the evaluation is an ongoing one. At the physical level, the backbone is based on FDDI, with some remnants of a past Pronet-80 and -10 being used to meet specific needs, and the pathways to the campuses providing at least a full 56K, and in many cases several times that. (Several campuses are connected via T-1's, which support bandwidth needs for both compressed interactive video as well as data.) Backbone services are provided centrally to a point on each campus, from which either 802.3 or 802.5 (ie., Ethernet or Token Ring) networks can be extended. Cabling is considered as being infrastructure, both among and within buildings, and is generally installed in a physical star within the buildings from a few telecommunication closets. We have adopted the use of Type 2 cabling as a minimum cabling standard, allowing for additional types of cabling (such as fiber or unshielded twisted pair) to be used where conditions warrant. We are currently evaluating the possible adoption of level/category 5 cabling as a potential successor to the use of Type 2, based on it's slightly lower installed cost, and it's similar performance capabilities. Fiber is used between buildings, with over 75% of our major buildings currently connected. At least 6 fibers are installed to each of these, with up to 144 to facilities such as the main Library build- ing. We currently have about 4200 registered addresses, about 1200 of which are for pc's in laboratory environments. There are probably several more (100's) of unregistered machines scattered among the local networks. (Central registration is strongly encour- aged but not required. Although there are certain limitations for unregistered machines, and there is no fee to register, registration is at the discretion of the individual user or department.) There is also a large administrative SNA- based network with a 2000 users or so that is slowly being transitioned to the TCP/IP backbone. We manage 4 class B and 10 class C networks, broken into roughly 300 sub- nets, with an additional 100 workgroups networks being divided within that structure. (These are exclusive of the network at our Hershey Medical Center, which is managed locally, except for name service which is provided centrally for this large multi-router network.) We manage abour 40 routers, 30 bridges, and 40 concentrators/10BaseT hubs. Our responsibility includes everything "inside" the network, with the for- mal demarcation point being the end of the cable attached to the back of the router. With SNMP, we "look" further into the individual LAN's, and in some cases directly into the pc. We offer a cost-recovered service to design, install, and maintain local networks for those who do not have or want to pro- vide that expertise themselves. This applies both at our University Park cam- pus as well as at the other campuses located throughout Pennsylvania. We are also responsible for the management of the centrally installed cabling systems. This is true regardless of whether the cabling has been upgraded to meet minimum cabling specifications or is still previously installed telephone wire. The main areas, in terms of data networks, for which we are NOT responsible are for LAN electronics (concentrators, bridges, servers, communication cards) for individual LAN's which are managed by individual departments. It will take some time to define and align our data networking budget in accordance with this criteria. We do not define a DATA NETWORKING budget per se--there is an overall telecommunications budget which includes data networking services, but which is subdivided by different criteria.) In the spirit of trying to provide something useful, there are a few comments worth offering: --our philosophy is to avoid penalizing anyone based upon which Penn State location they happen to be at, so fees are generally location-independent. --rates are developed based on different services, in turn functions of connection speed, cost of leased facilities, and whether it is useful to stimulate its growth by subsidization with central funds. (Note that there is no specific subsidy of data, voice, or video services by rates generated from either of the other two services.) --When a service is to be fully cost-recovered, we make every effort to include in that rate not only a reasonable amortization period for up-front costs, but also a mechanism to generate both maintenance and replacement (life-cycle) funds. --Two distinct telecommunication budgets are administered--one to account for the use of central funds (generally to support infrastructure, and to support "immature" services) and the other to account for "mature" services, for which all provisioning costs are recovered. --To pay for fringe benefits, office space, and other services used by that portion of the office providing "mature" services, most rates include a 22% overhead to pay for those types of costs. Exceptions are notable for services having high percentages of equipment costs (assessed a half the rate) and for labor costs of networking services within residence halls (billed to the Housing office at cost). --Departments are expected to pay for their LAN from their departmental budget. --The data backbone service is sold at a fee of $3200 one time cost, and a recurring monthly charge, regardless of the number of devices attaced to the LAN, of $275. This pays for the electronics, installation, maintenance, and replacement necessary to continue up-to-date service levels. The connection can be either 802.3 or 802.5, at the department's direction. We also encourage individual departments within buildings to consolidate their needs. In those cases where segregation of the building is warranted (e.g., for security or performance reasons) we will install and manage a "building backbone" consisting of a small network-in-a-box, and bridges to networks attached to it. This also requires one person in the building to assume the role of a "broker" for the building networks. (This is in addition to persons being identified as the network's administrative, technical, and security contacts for that connection. --We also offer an "individual backbone connection" in certain buildings (those in which we can make it financially viable) which is currently billed at $750 one time and $15/month, which is equivalent to past data-switch and SNA service connection costs. In those cases, we provide and manage not only the routers, wiring, and 10BaseT concentrators, but also the card and communications software in the pc itself. IBC's are provided on only individual bases--they may not be used as network connections. --We do not count packets, and in general do not base rates upon usage level, instead using flat rate monthly costs as our preferred model. As noted above, we also encourage autonomous growth of local networks, with registration of addresses being about as close as we get to monitoring that aspect (unless we are retained to manage the network, in which case we do keep close tabs on it). --Because we are not a academic entity, we do not directly submit grant requests, although we oftentimes work closely with those who do for telecommunications related items. The telecommunications budget for this year is slightly over $15M, with 1/6 of that centrally funded, and the remainder cost- recovered. I'd estimate that about 1/4 of the $15M supports "central data networking services" directly, and another 1/4 indirectly (meaning it is shared with other services which bear a portion of the costs of circuits and the like.) Appendix C Case Study Harvard University The response from Harvard is included here, edited for readability. Harvard has by far the largest network of all of the respondents in the survey, and recovers most costs from monthly node fees. From: Stephen J. King Associate Director Office for Information Technology Harvard has installed a 10MB, fiber backbone for the University, a very capital project, and OIT has a high speed data network business unit doing network operations center, LAN support, and consulting The Harvard data network is a distributed architecture, with 7 major nodes, based on TCP/IP and supporting IPX, Appletalk, and Ethernet protocols Number of Intel-based PCs/servers: 6000 Number of Mac-based PCs/servers: 9000 Number of RISC-based workstations/servers (SUN, HP, RS6000, ...): 500 Number of network-attached printers: 2000 7 major hubs or nodes, lots of central monitoring equipment, hardware in over 500 buildings, approximately 200 LANs being connected over time Responsibility extends to the wall-plate. 50% of the budget is recovered from node fees per unit time. 10% comes from workgroup/subnet fees. 10% is derived from installation fees. The rest is made up from overhead and lump sum payments by departments. Note that the entire budget of approximately $1M is cost-recovered.