Networked Delivery of Multimedia Information Copyright CAUSE 1994. This paper was presented at the 1993 CAUSE Annual Conference held in San Diego, California, December 7-10, and is part of the conference proceedings published by CAUSE. Permission to copy or disseminate all or part of this material is granted provided that the copies are not made or distributed for commercial advantage, that the CAUSE copyright notice and the title and authors of the publication and its date appear, and that notice is given that copying is by permission of CAUSE, the association for managing and using information technology in higher education. To copy or disseminate otherwise, or to republish in any form, requires written permission from CAUSE. For further information: CAUSE, 4840 Pearl East Circle, Suite 302E, Boulder, CO 80301; 303449-4430; e-mail info@cause.colorado.edu Networked Delivery of Multimedia Information Robert Brentrup DCIS Project Director Dartmouth College Computing Services 6028 Kiewit Computation Center Hanover, NH 03755-3523 Abstract Multimedia information sources present exciting possibilities for increasingly sophisticated and evocative presentations of material in many fields of education. Widespread use of this technology has implications for the campus computing infrastructure, local network capabilities and the services provided by libraries. Dartmouth College is expanding its campus-wide information system beyond textual resources and is researching how best to provide broad-based access to and support for multimedia for a large community of users in a cost-effective manner. This paper discusses current plans to incorporate image, audio and video media into a networked information system, outlines the technical and organizational issues that have been identified and describes work in progress toward this goal. Networked Delivery of Multimedia Information Networked Multimedia Benefits For faculty and students, multimedia resources can add new dimensions to the learning experience. Most concepts are easier to present and comprehend when words are complemented with images and animations. Learners retain more when a variety of senses are engaged in delivering information. The "intensity" of the experience aids retention and recall by engaging social and emotional as well as intellectual responses. Unlike paper books, networked electronic media can allow many people simultaneous access to the same materials. These resources are available all the time, from office or home. More sources can be consulted and relevant information can be obtained immediately. More time can be spent on content, not locating information. Most topical special collections include material in a variety of forms such as drawings and photographs. This information has not been as readily available in computerized form since the technology has not yet been appropriate. Recent advances in workstations, networks and storage technologies are providing a new opportunity to create an even more sophisticated information system, one that will contribute to the widespread use of multimedia applications in the community. These are the most important benefits we hope to realize in adding multimedia to the Dartmouth College Information System (DCIS). Selecting a Multimedia Delivery Technology Multimedia information can currently be delivered via broadcasting, analog cable TV, laser disks, digital networks or CD-ROM disks. For some data sources or uses, particularly long video programs, existing digital networks are not yet an adequate delivery method. However, technical, economic and political developments are all pointing in the direction of digital networks which will eventually integrate the capabilities of alternative delivery mechanisms. Technically there is much interest and activity around the merging of television and computing devices for several reasons. On the end-user's desktop it is certainly preferable from the standpoints of cost, maintenance and complexity to integrate information delivery on a single screen and network connection versus having both analog and digital screens and networks provided in each delivery location. Once these capabilities are combined by common data formats and displays, many interesting applications become possible. The need to equip each workstation with the proper players for CD- ROMs or laser disks as well as handling the disks when a large number of people are involved, makes these approaches relatively unattractive at present and into the near future. Analog television's current absence of uplink capability and the need for analog to digital conversion at each workstation makes this approach similarly unattractive relative to a digital network. Cable television and telephone companies are plunging into the digital network business with huge deals, mergers and projects to capture part of the fast growth of networked information sources and delivery to homes and businesses. The attraction of an exponentially growing market is irresistible compared to their relatively mature and slowly growing primary businesses. This activity will certainly push forward equipment capabilities and hopefully reduce equipment costs. There is concern about what it will do to the cost of network access. These factors and trends combined with the existing universal digital network on the Dartmouth College campus are focusing our development efforts around digital networking for the delivery of multimedia. Organizational and Infrastructure Issues Many organizational and infrastructure issues need to be addressed when deploying a campus wide information system. Adding the technical goal of delivering multi-media information over the campus network adds some new issues and additionally impacts some existing ones. The areas listed here seem to be the most important factors influencing what DCIS can reasonably do with multimedia in the near term. Of course, there is also a financial component to all of these problems as well. * Acquisition and Preparation of Material * Copyrights * Access Control and Billing * Data Storage * Network Requirements * Desktop Workstation Features, Performance and Availability Acquisition and Preparation of Material A few years ago, few materials were even available in electronic form. Often a work was added to the library's collection because it was the only one of that type available in an electronic form. As computers have become more prevalent in the publishing industry, many new works have become available by virtue of having been manipulated electronically in production. Now there are more sources of materials that are unfortunately in many formats, with occasionally erratic quality. The accuracy and edition identity of an electronic work is crucial for serious scholarship. Some standardization of formats would be a big improvement. The labor involved in digitizing (by keyboarding or scanning) and organizing a collection is substantial. The DCIS project is identifying and at times developing small tools to automate tedious operations as we proceed. We try to use personal computers where possible to simplify transferring these tasks into the originating department. Copyrights The ownership of material is currently a primary concern in selecting projects for development. The pilot projects DCIS has under-taken have been selected in part because their source materials had clearly defined ownership and negotiable permissions. The requirements of complying with copyrights are at times murky, particularly in deciding what is a "fair use" of certain materials already purchased in a paper format. The simultaneous-usage aspect of electronic media has many publishers struggling with how to control and retain fair value for their property. An electronic copy lacks printing costs and has lower distribution and transportation costs. Should an electronic copy cost more, the same or less than a paper one? Some publishers have been unwilling to discuss the possibilities. Others have been more creative in working out mutually beneficial experimental arrangements. These questions need to be addressed by working with publishers to help them develop new ways to derive fair return on their investment and ultimately by additions to copyright law. A number of software refinements are being considered to help implement "fair use" policies. The possibilities include limitations on single-search data retrieval and limits on per- session data printing and electronic copying. Access Control and Billing At present the licensed data sources Dartmouth has made available have been site licensed for the entire campus. This is quite satisfactory for certain widely appealing resources. DCIS has developed a flexible and effective distributed access-control system. It is possible to limit access to either the entire campus or a subgroup. This has made it possible to adequately and efficiently control site licenses and to develop collaborative licensing arrangements with other organizations. The next iteration of the system will provide limits on the number of simultaneous users of a particular database to provide more flexibility in negotiating licenses. The next step beyond that would be to incorporate a billing system into the access-control process to allow per-use types of pricing arrangements. Data Storage The demands on digital storage capacity depend largely on the type of data and how it is indexed. The following example should help provide a frame of reference. A typical typewritten page of text consumes 5,000 bytes of storage. A significant work such as the full text of Shakespeare's plays fits in 5 million bytes. The text databases already online at the Dartmouth library range in size from a few million bytes to almost 1 billion bytes. The indexing overhead for text is typically 100 percent; for example, the 500 million byte Oxford English Dictionary requires an additional 500 million byte index. Dartmouth's current text collection fills approximately 30 billion bytes. In contrast to text, the storage requirements for images can grow several orders of magnitude more quickly. For example, an uncompressed 1,000 by 1,000 pixel monochrome image (roughly the size of a 2 page monitor) requires around 100,000 bytes of storage. To represent 256 colors in this same image, the storage requirement increases to 1 million bytes. The same size image with 16 million colors requires 3 million bytes. The raster image of a laser printer page at 300 dots per inch requires about 7 million bytes uncompressed. Fortunately there are a variety of compression techniques or higher-level descriptions of printer pages to reduce these requirements. Digital video pushes the storage requirements even further. For example, 30 frames (one second) of video digitized at 300 by 300 pixels stored with 256 colors would fill a 20 million byte hard disk (the average personal computer hard disk of a few years ago). The data storage requirements to provide networked multimedia are substantial. Cost-effective solutions will eventually need to incorporate a hierarchy of storage devices. Early system efforts will likely require some management of storage by the network media servers themselves. Eventually some of the required functionality will move into operating systems as the needs become more widely applicable. Network Bandwidth The demands on network bandwidth depend on the type of data, the network protocols used to move it, the rate at which it needs to arrive and the amount of traffic on the network. Measurements indicate that practical bandwidth can at times be reduced to one- tenth of the theoretical maximum when protocol overhead (such as addressing and error correction information) is included. The load variability is a large practical problem for data that must be synchronized (audio or video). The multimedia system software must provide methods for gracefully degrading performance on networks where bandwidth cannot be preallocated in order to cope with this problem. Text or still-image data do not have real-time delivery constraints like video. When delivering text or still-image data, getting it to the user in an acceptable time frame is the constraint. Simple analysis estimates and practical experience confirms that fast modems (at 14.4 Kbits/sec) or local area networks like LocalTalk (at 230 Kbits/sec) are capable of transferring text and medium-sized images (e.g. 50 KBytes) in tens of seconds. Ethernet speeds (10 Mbits/sec) combined with current compression techniques are needed to transfer larger color images and low frame-rate video. To deliver high-resolution video will require networks capable of at least 100 Mbits/sec. Networks need to employ compression technology at current bandwidths, and compression requires more workstation processing power. It is likely that the data demands will always stay ahead of the economically available bandwidth so compression will continue to be important. Additional network capacity is a key constraint in the widespread deployment of multimedia applications. Workstations The capabilities of a user's workstation historically have constrained what was possible to deliver in an information system. Early terminals and personal computers dealt only with text. A second generation of personal computers opened the richer worlds of graphics and typography. A new generation of machines is now making audio and video manipulation readily available. The development of DCIS has followed this progression of workstation capability: initially text, and then typography and graphics, with the addition of audio and video now being considered. There is a balance to strike with regard to workstation capabilities. Some basic requirements can't be gotten around. Each person needs a reasonably sized screen and local memory, adequate processing power and network access. However, all these resources can be stretched by implementation tradeoffs in the client/server division of labor and the amount of labor invested in software performance. The balance point is always changing as equipment capabilities increase, although workstation upgrades must compete with many other budget priorities. Development Objectives After studying possible applications Dartmouth has developed the following list of objectives to outline our plan to introduce multimedia into DCIS. * To develop, or integrate when possible, a suite of applications that enable individuals to make use of these media for research and instruction. * To implement a server architecture that allows applications to locate, retrieve, and manipulate media resources. * To develop media resources and to make them available for such a server architecture. * To develop an environment that will allow users to be both readers and creators of media sets. * To study aids to retrieving, locating, and describing images and video clips. * To develop the necessary maintenance tools and procedures. Recent DCIS System Developments The requirements of two new library services, an image catalog and electronic document delivery, have been used to focus the development of the basic software components required to support multimedia. The DCIS project's initial goal was to produce a image database that contained searchable textual descriptions of the image. In addition to the existing software, we needed image format readers, decompression modules, bitmap transforms, a suitable database manager, network protocol extensions and a client application to retrieve and display images None of the database management systems DCIS is using inherently supported binary data fields. Extensions to our servers, which front-end these database systems, added the ability to link to external files of binary data. The server can either open this external file and pass the data through as a binary field, or deliver a reference to it which can be passed to another program. The network protocol was enhanced to allow data transfer to be segmented. We modified an existing client application used to search text databases to handle image fields as well. This text client can also locate, start-up and communicate with other client applications built specifically to manipulate other media types. A universal document identifier (UDI) protocol has been implemented based on design proposals being considered for the Internet. A UDI permits references to other media types to be stored in a text database. The text client application is able to retrieve these references from the text database and pass them on to another viewer program that will retrieve and display a particular type of multimedia object. We have developed an image viewer application that can retrieve and display color and gray-scale images in TIFF, GIF, PICT, and JPEG formats. This application has an interface allowing it to open and display disk files of scanned documents distributed via electronic mail as well. Multimedia enhancements have been added to several of the existing servers. Several applications have been prototyped including a Dartmouth College Photo Records catalog, an electronically published magazine which includes illustrations, and an electronic document-delivery service. The components produced are generalized enough to be applicable to several other related areas by supplying different data. For example, the same software can display satellite photographs and weather map images stored in a Wide Area Information Server (WAIS). The document collection of the WAIS source is searched using the text client application, which in turn passes on image references for retrieval and display by the image viewer application. Experiences and Observations The DCIS development effort has followed a phased implementation approach linking other media to the text database facilities developed earlier. The following discussion summarizes our experiences and some observations. End-User Equipment The balancing act between system functions and hardware requirements is a difficult one. Currently at Dartmouth most faculty and staff have a Macintosh. One hundred percent of the students have a personal computer, most of those are Macintoshes. The capabilities of the installed base is, however, relatively modest. The incoming class of students is always the best equipped. The current freshman class has workstations capable of handling the image capabilities we have developed, earlier classes mostly do not. The faculty and staff have difficulties keeping up with the pace. The DCIS team remains concerned about frustrating the end-users by requiring more computer power than they have available. The realization that computing equipment has a relatively short life span is slowly working its way through the community. Networking is much the same problem; most buildings have LocalTalk networks. Certain locations and new buildings have at least Ethernet networks. A campus-wide network upgrade is planned, although funding has been hard to obtain. Data Preparation and Maintenance Multimedia data is mostly captured from other sources, such as scanners and video digitizers. Creating digital multimedia requires special equipment, higher performance workstations and relatively sophisticated computer skills. End users have typically required assistance and funding to get started digitizing and manipulating their source materials. Construction and maintenance of the databases in the DCIS system are complicated by their delivery from UNIX workstations. The performance of these systems are necessary, although their user interfaces discourage less-sophisticated users. We have converted a number of the maintenance tools to run on the Macintosh, which has greatly simplified setting up the databases and preparing updates. The completed files are then transferred to the server workstations for indexing. A complex set of tradeoffs surrounds image quality. In principle there is a need to store the originals in relatively high fidelity, perhaps to preserve them but also to facilitate the quality of their reproduction when employed in other work. The time and labor to handle and scan a large collection is a large cost which would be nice to avoid repeating. In contrast, it is desirable to conserve server storage and to produce fast delivery on finite bandwidth. An image scanned beyond screen resolution wastes both these resources for the majority of uses. One desirable image-format feature in view of this dilemma would be the ability to deliver a base image rapidly, to which subsequent detail can optionally be added. Another possible approach is to generate image derivatives (e.g. a scaled-down size) on the server in response to certain queries. Database Design It has proved convenient for maintenance sake to store media objects in individual disk files. These can be manipulated easily with standard tools. Using standard naming conventions helps the maintainer manipulate them in groups. The separate text catalogs can likewise be easily edited. Some of the image formats provide the capability to include textual descriptions of themselves, although this has not yet been widely exploited. This may allow catalog databases to be produced mechanically from a directory of annotated images. It is difficult to describe all the different aspects users may be interested in when browsing an image database. Developing methods of viewing samples of many images quickly is an important feature to bridge this gap. Pre-computing certain locational aids, such as miniatures, may be a cost-effective approach. A thesaurus of terms seems to be an important aid in locating images while minimizing the amount of subject indexing. A fairly specific classification combined with a hierarchy of terms can allow an image to be selected for a variety of reasons. A number of interesting computer programs attempt to analyze and describe the contents of images and to locate scene changes in video. These may develop into both cataloging and retrieval aids. Building multimedia databases has an enormous up-front digitizing cost. It will be important to make it easy to incrementally add to collections. It may be necessary to set up some collections so an image is scanned and entered when it is retrieved for use the first time. The steps to do this will have to be simple to fit into the work flow. Image Formats The variety of image formats is a difficulty. There always seems to be one more you don't have that someone wants to use. At present DCIS has accommodated the most popular formats on the Macintosh. Developing efficient "readers" for these formats is a considerable amount of work because of the variety of coding and compression techniques. For example, different formats provide color maps, provide progressive detail buildup, are revisable in place, are byte order-independent or provide high compression ratios. Some of these features are handy for certain applications and some are incompatible with high compression. Server Design As mentioned in the discussion of workstation capabilities, the division of labor between the client and server and the design of the server's features can be used to moderate performance requirements of the workstation. Some additional server capabilities DCIS intends to develop will include result caching, format transformations, scaling transformations and compression transformations. The network servers could be enhanced with the addition of several logical network services. Since the servers are distributed, static links in the databases and the resulting dependency chain are best avoided. Binding component names at server or session start up through a name-resolution service is much more flexible. Client Design Objected-oriented programming techniques have greatly simplified incremental development and code reuse. Access to source code is, however, essential and far too much effort is still expended in reimplementing similar functions and ideas. It seems that every data provider also wants to be a user interface designer. This is a significant problem, since developing a complete, tested client application is more effort than anything else we do. The extreme of a different user interface to every database would not serve end-users well. The present system does allow the result displays to be customized by delivering a description language from the server to the client. This has proved adequate for a large class of information resources. The exceptions continue to inspire new ideas. Future Directions In using the computer to present data, the designer needs to apply all of the bookmaker's and graphic designer's techniques. Screen layouts should please the eye and guide the reader to the most important information, as does a well designed page. The subtleties of fonts and colors need to be used to encourage the reader and draw attention to important information. The object is to deliver the maximum amount of information in the minimum time. It is crucial that multimedia be well crafted since there is an even larger potential to create confusion when compared to paper materials. There is a basic dilemma in having the power to manipulate all these factors and the amount of time and effort one must invest in doing it properly. The ability to add to an already well-crafted framework is a significant need in creating high- quality work more quickly. Easy access to vast amounts of electronic information causes some problems as well. Users will need help to make sense of and find their way around large collections. Large topical databases may be accompanied by "Guided Tours." A guided tour would be laid out by an expert in the field, highlighting the most important information and providing a main thread for new users to follow or return to while browsing a collection. The underlying information system provides the raw data behind the tour. Humans are good at visually spotting an anomaly or patterns in data. Reducing large data sets into visualizations can be a powerful research tool. The user should also be able to gain access to the raw data in order to verify conclusions or pursue other questions or analyses. To generate new relationships while studying a data collection with searches, computational reductions and visualization graphics seems to be a desirable additional capability. Conclusions DCIS has deployed an information system on the large scale of the entire institution. Over 60 local databases and hundreds of external sources of information are reachable by the system. Five central computers running about a dozen different servers currently provide information services for more than 700 daily users. Portions of the DCIS system have also been installed at a number of other institutions. Well into the third year of production use, we are trying to cope with substantial growth in usage and demand for more information resources. Enhancing the system further with multimedia resources is consistent with many requests received. DCIS is trying to introduce multimedia to the campus-wide information system in small steps in order to stay in touch with its audience. Up-front considerations of the existing installed base of equipment and stressing efficiency in the software to make best use of the available equipment has kept this fundamental problem in check. Fortunately the evolution in workstations and networks is moving in the right direction to make these new applications attainable. Delivering new applications that aren't too far beyond current practice can concretely demonstrate the benefits of further investment in infrastructure improvements. Although better networks are certainly very desirable, Dartmouth's existing computing environment provides a unique laboratory for exploring the frontiers of distributed, network-based computing and for applying the benefits of technology to academic and other pursuits. Networked multimedia resources have great potential for having a significant impact on how people learn and work.