University/College Information System Structures and Policies: Do They Make A Difference? An Initial Assessment This paper was presented at the 1995 CAUSE annual conference. It is part of the proceedings of that conference, "Realizing the Potential of Information Resources: Information, Technology, and Services--Proceedings of the 1995 CAUSE Annual Conference," pages 2-1-1 to 2-1-9. Permission to copy or disseminate all or part of this material is granted provided that the copies are not made or distributed for commercial advantage. To copy or disseminate otherwise, or to republish in any form, requires written permission from the author and CAUSE. For further information: CAUSE, 4840 Pearl East Circle, Suite 302E, Boulder, CO 80301; 303-449-4430; e-mail info@cause.colorado.edu. UNIVERSITY/COLLEGE INFORMATION SYSTEM STRUCTURES AND POLICIES: DO THEY MAKE A DIFFERENCE? AN INITIAL ASSESSMENT Bruce Rocheleau, Ph.D. Northern Illinois University DeKalb, IL 60115 ABSTRACT: There is a substantial amount of variation in college/university computing structures (e.g., whether the university leadership is consolidated under a chief information officer), computing policies (e.g., the extent to which cost recovery is used), and planning efforts (e.g., the extent to which formal plans are developed concerning information and related policies). Although these policies have been frequently discussed, there has been little systematic research concerning whether these structures and policies have any impact on outcomes such as the extent of access and use made of computing by faculty, staff, and students. This paper draws on the 1994 CAUSE survey to provide an initial assessment of these issues. INTRODUCTION Henry Lucas studied the power and prestige of information service departments in 1984 and found that they had little power and visibility in most organizations.[1] But computing is now becoming a central activity of strategic importance to both universities and businesses. For example, Mara points out that the definition of a user of information technology at Cornell has changed from a hundred or so central office users to over 20,000 members of the university community.[2] If colleges and universities are to attract and keep top notch staff, faculty, and students, they need to serve a full range of users and support the teaching, research, and administration needs of their institutions. To achieve these goals, information managers in colleges and universities need to know what policies are effective. What measures can they take that will help ensure success? For example, in recent years, many universities have spent a great deal of time developing technology plans including strategic, telecommunication, networking, administrative, library, and academic components. Although plans may help to bring about enhanced effectiveness, there is skepticism about the utility of planning because many plans remain on shelves unused. Do universities that construct formal plans (e.g., for networking) have more successful results (e.g., higher percentage of their micros/workstations networked) than those who do not formally plan? Do plans that are updated annually or linked to the budget have more impact? The Chief Information Officer (CIO) has become a familiar position in universities and colleges. Slightly more than 75 percent of the institutions in the 1994 CAUSE survey reported that there was a CIO, though only about 56 percent said that the CIO is recognized "as such" in their organization. CIOs may come under attack if they are not viewed as being effective. For example, a recent article asked the question, "Is Your CIO Adding Value?"[3] In the private sector, several CIOs have been fired and in the public sector, CIOs have recently become the lightning rods for controversy in several states as one former CIO noted:[4] States have to do more with less and they think that technology is going to pull a rabbit out of the hat for them. University CIOs may come under similar pressure as the strategic importance of computing grows. A recent paper noted that the organizational rather than technological challenges have been most difficult and that there still is disagreement about whether centralized or decentralized structures work best in a university setting.[5] Pitkin studied the role of college/university CIOs and found that they differed from their business counterparts because they did not carry out some roles necessary to be an effective executive.[6] The structure and power of the college/university CIO job can vary greatly. For example, some CIOs (about 18 percent) report directly to the Chief Executive Officer (CEO) of their college or university. About 33 percent report to the Chief Administrative Officer, 19 percent to the Chief Financial Officer, and the remainder report to a variety of others. Does it matter whether there is a CIO or whom the CIO reports to? Is a CIO who reports directly to the CEO without any intervening layers of administration more effective? Does it make any difference as far as use of computers in the curriculum whether the head of Academic Computing reports to the CIO? Finally, there has been controversy over what budget and cost recovery policies are most effective in encouraging use of computing by faculty, staff, and students. Thomas M. Schwen, head of Instruction Technology at Indiana University, stated that he was worried about a backlash when campus decision makers found that faculty only made use of a tiny fraction (e.g., 1 or 2 percent) of the capabilities of high-tech classrooms.[7] Do student fees and chargeback systems keep students and professors from using the Internet? There have been reports that high network costs have done so.[8] Similarly, do colleges/universities with ongoing budgets (about 35 percent of our sample) for replacing micros and workstations have faculty and students more involved with computing in the curriculum and the Internet? Which policies, if any, positively influence the spread of academic use of computers. The above questions deserve attention and careful study involving a variety of approaches including case studies, and the employment of experimental and quasi-experimental designs. Our study is exploratory, aimed more at focusing attention on these issues and developing hypotheses than reaching final conclusions about these questions. But the importance of these issues cannot be underestimated. Many people argue that information technology has been slow to permeate the curriculum of colleges and universities. For example, Cotton found that the percentage of courses in which information technology was integrated into the curriculum was 17 percent, no higher than in their kindergarten to high school study.[9] Steger, Williams, McClure, and Smith recently pointed to the dearth of evaluation studies concerning technology expenditures and the need to conduct such evaluations due to the shrinking economic resources available to universities.[10] Can universities modify their policies and structure to improve outcomes? These are the issues that we wish to address here. These are important issues. We hope that this paper will help to stimulate research on this topic. METHODS We began this study with a series of case studies conducted during 1993 concerning the structure and role of information technology at five Midwestern colleges and universities. Interviews were conducted with a variety of information technology staff, and also faculty and administrators at each institution. These case studies led to the devlelopment of hypotheses that, with the permission and assistance of the CAUSE staff, we were able to explore through analysis of the 1994 CAUSE survey which contained several relevant questions. Our CAUSE survey sample used in this paper excludes surveys from 2-year, specialized, and uncoded institutions and the total sample size was 296. The total response rate for the CAUSE survey was approximately 39 percent. There is no way to determine whether, or to what extent, the responding institutions are different from the institutions that did not respond to the survey so caution must be applied to generalizing beyond the respondents to the survey. As shown in Figure 1 below (FIGURE NOT AVAILABLE IN ASCII TEXT VERSION), we studied four major categories of independent variables (Planning activities, CIO- Organizational Structure, Computer Charge Policies, and Resource Allocation Variables) concerning their impact on the outcome variables. A detailed list of variables employed in the analysis is provided in Figure 2. Figure 2 List of Variables Employed in Analysis Planning Activities Institution has overall campus strategic plan Institution has IT Strategic Plan Is IT plan part of overall plan? IT Plan Covers Academic Computing IT Plan Covers Administrative Computing IT Plan Covers Telecommunications IT Plan Covers Library IT Plan Covers Networking IT Plan Linked to Budget IT Plan updated Regularly Composite Measure of Planning Activity (0 to 8) CIO Power Is there a CIO? IS CIO recognized as such? CIO reports directly to CEO Academic Computing reports to CIO Administrative Computing reports to CIO Composite Measure of CIO power (0 to 5) Computer Charge and Budget Policies Are there Student Fees? Chargeback (None, Partial, Full) for Academic Computing Chargeback (None, Partial, Full) for Administrative Computing Is there an ongoing budget for replacing micros/workstations? Environmental Variables Public or Private institution? Type of Institution (Carnegie Class) Size of Institution (FTE's) Index of Resources (Ratio of micros/workstations to FTE's) Resource Allocation Variables: Percent of Faculty with exclusive use of micro/workstation Ratio of micros/workstations to students Ratio of micros/workstations to faculty Ratio of micros/workstations to staff Does Institution provide access in dorms to micros? Percent of micros/workstations that are academic Outcome Variables: Percent of Faculty making use of software in classroom Percent of Faculty making use of computing in curriculum Composite Index of Faculty having access to & Using Email, Gopher, & Web Composite Index of Staff having access to & using Email, Gopher, & Web Composite Index of Students having access to & using Email, Gopher, & Web Percent of Academic micros/workstations networked Percent of Administrative micros/workstations networked Our analysis used a variety of statistical analyses including the use of a series of multiple regressions and partial correlation analyses. These techniques allow us to study the impact of several independent variables (the structures and policies of universities) on outcomes simultaneously while "holding constant" (statistically) the infuence of certain environmental variables such as the size of the institution and its resources. It is important to control for size because, for example, the larger universities have more resources that make it more likely that they will construct formal plans. Also, the general wealth of a university may allow it to look good on outcome measures though its plans and policies may not be especially effective. Although our index of resources was crude, examination of institutions high on this variable confirmed that many wealthy private universities scored among the leaders on this index (e.g., Harvard, University of Pennsylvania, Williams College, and Duke University) which supports our argument that this measure helps to control for institutional resources and wealth. We also studied the impact of other potentially relevant variables such as the Carnegie Classification of the institution. The study has several limitations that mean that we need to view the results as exploratory including the following: 1) Many variables are undoubtedly rough estimates including many of our key dependent variables (e.g., what percent of the staff, faculty, and students have access to and use e- mail, gopher, and the worldwide web). 2) In some of our analyses, there are a substantial number of missing cases for several variables including some outcome measures. 3) Although our analysis examined a large number of potentially important factors, many important variables are undoubtedly missing, especially those measuring the quality of the planning efforts. 4) This is a cross-sectional study and we need longitudinal studies to test for causal relationships among the variables. Despite these limitations, this study addresses significant issues and raises important questions for university officials seeking to discover what are the most effective policies. PRESENTATION OF RESULTS Table 1 summarizes our major findings. One fairly consistent predictor of outcomes was the presence of an ongoing budget for replacing micros/workstations. It was the only predictor of the percent of faculty use of computing in the curriculum and percent of administrative micros/workstations networked. It was also the best predictor of a composite measure of student access to email, gopher, and the Worldwide Web. An organizational structure in which Academic Computing reports to the CIO was the best predictor of the percent of academic micros/workstations networked and a composite measure of faculty and staff access to the Internet. It was the only predictor of the percent of faculty using software in the classroom. It should also be noted that the strength of the above predictors was quite modest, ranging from about 5 to 15 percent of the total variance (out of a possible 100 percent). TABLE 1: OUTCOME VARIABLES BY PREDICTORS (IN THEIR ORDER OF IMPORTANCE)[11] Outcome #1: What percent of faculty make use of computing in the curriculum? Predictor: 1) Institution has an ongoing budget for replacing micros/workstations. Outcome #2: What percent of faculty makes use of software in the classroom? Predictor: 1) Academic Computing Reports to the CIO. Outcome #3: What percent of the academic micros/workstations are networked? Predictors: 1) Academic Computing reports to CIO. 2) Institution has ongoing budget for replacement of micros/workstations. Outcome #4: What percent of administrative micros/workstations are networked? Predictor: 1) Institution has ongoing budget for replacement of micros/workstations. Outcome #5: Composite Measure of Staff Access and Use of Email, Gopher, and Worldwide Web. Predictors: 1. Academic Computing reports to the CIO. 2. There is an ongoing budget for replacing micros/workstations. 3. A composite measure of CIO power (Negative). 4. Institution is public. 5. CIO is recognized as such. Outcome #6: Composite Measure of Student Access and Use of Email, Gopher, and Worldwide Web. Predictors: 1. Institution has ongoing budget for replacement of micros/workstations. 2. Academic Computing is covered in the IT plan. Outcome #7: Composite Measure of Faculty Access and Use of Email, Gopher, and Worldwide Web. Predictors: 1. Academic Computing reports to the CIO. 2. There is an ongoing budget for replacing micros/workstations. In the above analyses, we initially excluded the Resource Allocation variables because we wanted to focus on the impact of the structural and policy issues. When we added the Resource Allocation variables to the analysis such as Percent of Instructors with exclusive use of an institutional micro/workstation and student access to micros in their dorms, we were able to predict better the percent of faculty making use of software in the classroom and in the curriculum though the percent of variance explained still remained modest. SUMMARY AND CONCLUSION We did find some consistent results in our analysis. Generally, the existence of a plan was not a good predictor of outcomes. The existence of an academic plan did help somewhat to increase overall student access to the internet. But, the existence of IT plan that covered academics did not predict use of computing in the curriculum. Likewise, existence of a networking plan was not a predictor of the percent of micros/workstations networked. Neither updating of the plan nor linking it to the budget proved to be predictors in the above analyses.[12] The quality of the plan effort and the nature of the process may be crucial to plan success and we had no way of measuring these aspects of planning efforts. Institutions in which Academic Computing reports to the CIO were more likely to have superior outcomes. Other CIO-related variables were not important. The fact that the CIO reported directly to the CEO did not have any major positive impacts. The computer charge variables turned out to be generally unimportant--student fees had no statistically significant correlations with the student index of access to the Internet nor use in the classroom or curriculum. The environmental variables (FTEs, public or private institution, Carnegie Classification, and our index of resources) had modest relationships with most of the outcome measures. To sum it up, our results were mixed. However, the significance of the ongoing budget and academic computing-CIO relationship suggests that budget and structural factors can have impact. Having a budget to replace micros/workstations is a very concrete activity and appears to have a beneficial impact on networking and use of computing in the curriculum. We checked and discovered that having an ongoing budget was not correlated with our measure of resources and wealth (ratio of micros/workstations to FTEs). Our results also suggest that a structure with CIOs in charge of academic computing has a positive impact on outcomes. As expectations concerning information technology continue to soar, colleges and universities need to put more effort into linking their scarce resources (such as time putting together plans) to bottomline results. Although many of the plans, structures and policies did not have much effect on outcomes, perhaps others (e.g., incentives for faculty to use computers) have more effect? Finally, the above research was done using 1994 CAUSE data. In the last year, there appears to have been a revolution in access to and use of the Internet. Has use of computing in the curriculum had a corresponding change? Can we identify any other policies that encourage good outcomes? We need answers to these questions. ENDNOTES******************************** [1] Henry C. Lucas, Jr. "Organizational Power and the Information Services Department," Communications of the ACM, January 1984, pp. 58-65. [2] Mark Mara, "Implementing Distributed Computing at Cornell University," Paper delivered at the 1993 CAUSE Conference, Boulder, Colorado: CAUSE. [3] Michael J. Earl and David F. Feeny. "IS Your CIO Adding Value?" Sloan Management Review, Spring 1994, pp. 11- 20. [4] Tod Newcome, "The CIO - Lightning Rod for IT Troubles?" Government Technology, October 1995, p. 58. [5] Joseph J. Branin, George D'Elia, and Douglas Lund, "Integrating Information Services in an Academic Setting: The Organizational and Technical Challenge," CAUSE/EFFECT, Fall 1994. [6] Gary M. Pitkin, "Leadership and the Changing Role of the Chief Information Officer in Higher Education," Paper delivered at 1993 CAUSE Conference, Boulder, Colorado: CAUSE. [7] Thomas J. DeLoughry, "Colleges Told to Take 'Disciplined Approach to Technology,'" Chronicle of Higher Education, October 27, 1993, a22-a23. [8] See, for example, Thomas J. DeLoughry, "Unconnected: High Network Costs and Low Interest Keep Many Off the Internet," Chronicle of Higher Education, February 23, 1994, a19-a20. [9] Carole Cotton, "Talking Turkey About "Real Change,"" Paper Presented at the 1993 CAUSE Conference, Boulder, Colorado, CAUSE, p. 5. [10] Susan F. Stager, James G. Williams, Polley Ann McClure, and John W. Smith, "Assessing the Effectiveness of Information Technology," Paper delivered at the 1993 CAUSE Conference, Boulder, Colorado: CAUSE. [11] Note: The relationships are positive unless otherwise noted. [12] Note that we only present here the composite results for the impact on the use of email, gopher, and worldwide web. We did 18 individual analyses and in a few, the updating and linking of the plan to the budget, did appear as predictors.