Managing Data Means Managing Mayhem: The Cultural Revolution Required for Data Administration |-------------------------------------| | Paper presented at CAUSE92 | | December 1-4, 1992, Dallas, Texas | |-------------------------------------| MANAGING DATA MEANS MANAGING MAYHEM: THE CULTURAL REVOLUTION REQUIRED FOR DATA ADMINISTRATION Roger Lurie John Rome John Wasileski Arizona State University Tempe, Arizona ABSTRACT Increasing pressure to gain access to university information is being met by the resistence of old cultural habits. To achieve quality in an increasingly competitive environment, we must foster a revolution aimed at transforming old taboos. The new culture must be supportive of data sharing and information systems as opposed to computer systems. This paper documents the steps being taken at Arizona State University to encourage and enable this transformation. ------------------------------------- Managing Data Means Managing Mayhem: the Cultural Revolution Required for Data Administration THE PROBLEM AND ITS CAUSE "Our data are incarcerated in our systems" was one of the many references to data problems stated by the Arizona State University (ASU) community in the documents provided to applicants for the position of University Data Administrator. The quotation expresses, perhaps better than any other phrase, the utter frustration in obtaining reliable data which was felt by everyone on campus. The Office of the Provost had "...no confidence in anybody's numbers..." since everyone claimed to be drawing their information from the same source but, somehow, the numbers wouldn't agree. This situation came to be because, over the years, much time, effort and money were expended creating very sophisticated transaction systems but very few resources were devoted to designing the information content for those systems. This is not intended to diminish what the designers had accomplished for ASU, they had done a very good job in integrating many of their systems from a process view-point. Rather, it is intended to provide a context within which to see that methodologies used to build systems have not included enough resources to design information as opposed to process. As was observed in "DSS - A Promising Future"[1] we simply fit things to the wrong model of management needs. One Approach to a Solution In recent years, many businesses and some educational institutions have begun to recognize that a more information oriented approach to systems design is required if we are to solve these continually nagging problems. Some organizations, ASU included, established a data administration function whose purpose is to help the organization express and gain control of their informational needs. Originally, these offices were "..techno-clerical; charged with trying to capture and manage data elements as they fell out of applications."[2] Later, as the need for integrated information planning was recognized, data administration became responsible for data models and, subsequently, the data warehouse.[3] After having struggled for several years with the concept, ASU's administration established the office of Data Administration in the summer of 1990. The position reported to the Assistant Vice President for Information Resources Management and its first task was to establish a mission for the office and agree upon first year objectives (please see Attachment A for a list of the objectives.) It is useful to consider the mission statement in light of the 'incarceration' quote above. Arizona State University Data Administration Mission Data Administration seeks to provide more people more access to more information more easily. This succinct statement has been very much an enabling key for ASU's office of Data Administration. In pursuit of our mission, we established the objectives detailed in Attachment A and set about trying to accomplish them. Briefly, the goals state that community support will be nurtured in building a shared information architecture which will include a single data dictionary and a single logical model of ASU's administrative data. Easy to say ... not so easy to do at ASU and, we think, not easy to do anywhere. The fundamental fact is that the culture of any organization is both enabling and growth-limiting. In short, the very habits which are endorsed by an organization also prevent necessary transformations within the organization. Bureaucracy is a specific cultural form which generates a malaise of spirit akin to inertia and, as Toffler noted, once a bureaucratic structure is in place, it resists change of any kind.[4] Bureaucracy is, however, just one of several possible cultures which became accepted as a form of authority during the industrial revolution and was encouraged by military personnel hired into business management positions following the war. Part of the difficulty in changing a culture is that each one establishes mores which are frequently not expressed directly.[5] Many times these become cultural habits which are so ingrained they take the form of a natural law or a religious belief. We have found the following unspoken beliefs on our campus; perhaps they will also be familiar to the reader. Sacred Cows of The Current Culture * Each office which collects data owns the data and can decide unilaterally what to do with it, e.g. change the code structure or meaning on its own. * We've always done it this way. We have a methodology all worked out and there is no need to change it. * The Buckley amendment forbids data access and sharing except for really exceptional cases. This is the true meaning of security. * Transaction processing is the most important part of the business and therefore the most important service we can provide. * Big central machines are the only way to handle the kind of load we have. * Systems are designed to meet the specific needs of a client office. * Programmers can create data elements as they need them. * Only specialists need that sort of capability (information access, etc.). * The important things are the screens the user sees. * Executive information and decision support systems are for upper management only. Many of these "laws" are diametrically opposed to the goals set by data administration. Following them has been quite costly for organizations. A Recipe for Mayhem Whenever offices "own" data they tend to make changes to fit their own needs without considering what ripple effects their changes might have. At one university, a simple code addition to "Ethnic Code" forced more than thirty programming changes in their institutional research office before regularly scheduled reporting could be continued. Central computing organizations cling to old development methods. They claim that queries are too hard to manage on a platform which supports heavy transaction processing and, since "big machines" are the only way to go and "the backlog is so big", information oriented designs will simply have to wait. Add to this the confusion caused by the redundancy created when programmers 'invent' data elements "on the fly", stir in a good measure of not sharing data in the name of security and you have the makings of a first-rate informational mayhem souffle. You can recognize when the mixture is ready by the following signs: ---What the Sacred Cows Cause * Faculty, researchers, and staff are hindered in their work by the daunting process of just getting to needed data. * Systems lack the flexibility to adapt to change. * Staff create independent, incompatible systems and abandon the "corporate" systems to meet their informational needs. * The organization speaks "datababble" (confused mutterings about data, their meanings and uses) which leads to incorrect conclusions. * Management cannot get timely information. It seems clear that, if data administration is to succeed, one of its fundamental jobs must be to help move the organization to a new set of cultural beliefs - beliefs which are more supportive of data sharing. How to Change a Culture ---What ASU's Office of Data Administration did: * Created policies to guide change. * Created a dictionary to foster common understanding. * Provided a solution to the problem of access to data. ASU's office of Data Administration decided on a frontal attack of these problems. We worked within the advisory committee structure to formulate policies for change and simultaneously began working to provide easy access to and a basic understanding of the data. DASC (Data Administration Subcommittee) was formed as an advisory group composed of members of an already existing Administrative Computing Advisory Committee (AdCAC). DASC helped formulate three major policy statements dealing with the fundamental issues of how ASU would address its information environment. The full text of these can be found as Attachment B but the "policy" portion of them is easily summarized. ---The Data Access Policy provides open access to most university information for all faculty and staff in support of university functions. It also provides for restrictions of certain data elements and a process for appealing restrictions. ---The Data Integrity Policy establishes a single logical model for university data and requires that each data element be documented in a single, universal dictionary. ---Finally, the Data Usage Policy specifies "update", "read only", and "external dissemination" of information as three separate categories of usage and gives guidelines for the use of data in each of these. It pays special attention to data which are downloaded and requires both the dictionary value and meaning of elements be preserved. When finally approved, these policies will provide for a sweeping reform of our current practices. The policy on Data Access has been approved by ITAC[6] (Information Technology Advisory Committee) and sent to the Provost. The other two policies are pending action by ITAC. The process has taken nearly two years thus far, has been intricate, and provides clear evidence of the slow motion bureaucratic systems apply to change. Proof of Concept Policies by themselves are insufficient to change established habits. However, policies coupled with actions demonstrating the requirement for the policies while providing needed services can be a powerful combination for cultural change. The policies were being moved through channels, we needed the actions. The actions were initiated through two projects we thought would provide the needed service and focus attention on the need for policies which would fuel a change within the university. The first project was to build a meaningful data dictionary, first as a document and then as an integrated part of our systems. Attachment C shows an example of the work done on each data element while the frame below summarizes the metadata we decided were needed in our environment. ________________________________________ Arizona State University Metadata for Data Element Dictionary Element ID Number Registration Date Element Name Definition Origination office(s) Update office(s) Domain Purpose Usage notes Length Field description Group element/Name Derived Element/Algorithm Release Flag Retirement Date Alias names Language Names Trustee Form(s) Id(s) Screen(s) Id(s) Report(s) Id(s) ________________________________________ Note that the metadata are not fixed, our dictionary is completely flexible and can be adjusted to suit changing needs. For example, we have decided to include new metadata to capture information on the Warehouse (see below) location of elements. It took about six months to complete the definitions (i.e. gather the metadata values) for about 1,000 elements within the student system. The initial version was released near Christmas in a large red three-ring binder. It is affectionately called "Big Red" by those who contributed to it and use it. Though it is now available in a much more convenient form, Big Red is the reference document of choice for definitions and domain values. The Dictionary Project (which is really an on-going process) served two purposes: first, it helped get people within the campus involved (about 20 people, representing all academic and administrative units, formed a Dictionary Task Force for general review with the bulk of work being done by a much smaller subgroup). Second, it helped data administration begin to understand the intricacies of ASU's data. One side-effect was that it "made some noise" on campus and helped focus attention on the larger tasks Data Administration would be undertaking. Our next major project was the design and construction of a Data Warehouse. Following Inmon's[7] ideas that a Data Warehouse is a: * subject oriented * integrated * time variant * non volatile collection of data in support of management's decision making, Data Administration used DEFT, a CASE tool, to design a relational database model of the Student Information System (SIS) optimized for querying. A graphical representation of this model is included as Attachment D. The design team was lead by John Rome from Data Administration and consisted of eleven staff members from Information Technology. The analysis stage was guided by over two hundred questions which client offices provided as difficult to answer under current conditions. Information gathered during the analysis stage demonstrated a need for historical data to provide for longitudinal studies. The Warehouse is populated via extracts from the corporate transactional files and designed to allow for modifications as we learn more about the informational needs of our campus. We can confirm what was observed in "Implementing the Data Warehouse"[8], ... maintenance of a warehouse via extraction and update is a constantly changing process. Historical data are frozen in the data warehouse and the current version provides ten years of history. Implemented in a client/server architecture which uses Sybase on a SUN 630, warehouse client tools like Brio's Data Prism and Data Pivot afford a graphical user interface which is very easy to learn. Many client tools were evaluated in-depth by the project team for the PC/Windows and Macintosh environments. This exploration and evaluation exposed a number of software tools that were in an adolescent state and not ready for delivery to the average end user. We have found Brio Technology's tools very helpful in providing meaningful access to data with minimal training. The project took 116 days from initiation to completion and was announced to the ASU community through an all day open house called "Data Access - A Piece of Cake". Visitors were welcome to eat cake and 'kick the tires' on more than a dozen client tools which we had explored and evaluated[9]. Approximately 250 of the universities faculty and staff attended. The demand for access to reliable data has mushroomed since the implementation of the Data Warehouse project. Requests have come from many university departments including college deans and the Office of the Provost. In one instance, a department was a finalist for a $2 million grant award. The award was contingent upon the availability of data on minority enrollment in the department's programs. After having tried all other possible sources, a very frustrated staff member called to see if the Warehouse might be of some help. Because of its ease of use, an administrative assistant from the department was given brief training and was able to obtain all of the reports needed to meet the criteria for grant approval in less than half a day. We are currently arranging for the Data Warehouse to become a production system which will be expanded to include Human Resources and Financial information all incorporated into the same relational design. The target date for completion is June, 1993. Conclusions Tangible results rather than rhetoric are required to change the culture of an organization. These projects have had substantial effects in that direction. Offices with decentralized, unintegrated systems have initiated discussions to move their data into a warehouse-like environment for sharing. They have also agreed to supply the required metadata for their elements to the dictionary. The flood gates seem to have opened for questions which could not be previously asked. Questions are getting more complicated and are indicating design changes for the warehouse. People now know where the office of Data Administration is located on campus and, perhaps most telling, under conditions of severe budget limitations, the warehouse project has been approved for implementation. Attachment A Arizona State University Information Resources Management Data Administration Objectives for 1990-1991 1. Establish Groups for Coordination and Consensus a. Develop an advisory group(s) for Data Administration policy development. b. Establish a project team to begin planning for an ASU Data Architecture. c. Begin developing positions for data administration staff. 2. Identify Issues Requiring Policy Development a. Identify the issues which require data management policies. b. Prioritize the issues identified above. c. Draft policies for the prioritized issues. 3. Begin the development of ASU's Master Plan for Data a. Conduct a study to discover the hindrances experienced by campus personnel in their efforts to produce information required in their decision making and take steps to ensure that such hindrances are removed as part of the data administration effort. b. Build consensus on what constitutes a data architecture for ASU. [What is in the "Table of Contents"? What do the diagrams look like?] c. Design the format and metadata content for an ASU Data Dictionary. d. Establish a process for data element review and definition. e. Establish standards for data names, capture, encoding, etc. f. Make significant contributions to defining and planning the ASU administrative systems infrastructure. 4. Identify the Key Systems needed for Data Administration a. Create a combined listing of all data elements used within "administrative" system files. b. Identify a data dictionary product and begin planning its implementation. [MISSING ATTACHMENTS B, C, D] NOTES [1] J. I. Penrod and J. S. Wasileski, "DSS - A Promising Future", CAUSE/EFFECT, January 1985, pp. 12-17. [2] William H. Inmon, "A brief history of data administration's amazing growth and development", Database Programming and Design, January 1992, pp. 68-69. [3] Ibid. [4] Alvin Toffler, PowerShift, Bantam Books, New York, November 1990. [5] Ibid. [6] Formerly ICSAC (Information & Communication Systems Advisory Committee). [7] William H. Inmon, "What is a Data Warehouse", Tech Topic vol.1 No. 2, Prism Solutions, Inc., 1992. [8] C. Sue Herman, "Implementing the Data Warehouse", Data Base Newsletter, vol.20 No. 5, Database Research Group, Inc., September/October, 1992. [9] Roger Lurie, "Arizona State University Data Warehouse Client Tool Evaluation Report", Arizona State University, Data Administration, October, 1992.