New Tools for Multimedia Development: ScriptX Copyright CAUSE 1994. This paper was presented at the 1994 CAUSE Annual Conference held in Orlando, FL, November 29- December 2, and is part of the conference proceedings published by CAUSE. Permission to copy or disseminate all or part of this material is granted provided that the copies are not made or distributed for commercial advantage, that the CAUSE copyright notice and the title and authors of the publication and its date appear, and that notice is given that copying is by permission of CAUSE, the association for managing and using information resources in higher education. To copy or disseminate otherwise, or to republish in any form, requires written permission from CAUSE. For further information: CAUSE, 4840 Pearl East Circle, Suite 302E, Boulder, CO 80301; 303-449-4430; e-mail info@cause.colorado.edu NEW TOOLS FOR MULTIMEDIA DEVELOPMENT: SCRIPTX Edwin J. Pinheiro Academic Consultant IBM Academic Information Systems INTRODUCTION There are two trends in desktop computing that have become popular: Multimedia and Objects. There has also been one problem that has bedeviled higher education institutions as they create and implement software on desktop computers - the incompatibility between the two dominant standards, Windows and Macintosh. What I propose to present here today is a tool designed to facilitate the implementation of multimedia programs, taking full advantage of object oriented technology, and designed to run on most popular desktop platforms of today, not only Windows and Macintosh, but also OS/2 and UNIX. ScriptX is a product of Kaleida Labs, a company formed and jointly owned by IBM and Apple Computer. Before delving into ScriptX, let me take a few minutes to justify the use of multimedia in higher education. If multimedia is just a fad without real educational benefit, we are wasting our time and our investment. MULTIMEDIA EFFECTIVENESS We have all heard cliches such as "A picture is worth a thousand words." Intuitively we agree with such cliches. We know that no matter how well someone describes a face to us, we rarely, if ever, can "picture" the face correctly until we see the person, or a photograph or drawing of the person. We assume that multimedia can somehow enhance the teaching and learning process, but are not quite sure why. The answer lies, as you would imagine, in the way our brains function. Research done at Woods Hole Oceanographic Institute and other centers of learning has shown that learning is enhanced not only by repetition, but also when multiple senses are involved in the process in a short span of time. Thus if we see and hear something we are more apt to remember it than if we only saw it or heard it. If this something is repeated enough times, physical and chemical changes take place at the synapses in the brain, and the memory becomes permanent. It is also true that memory is diffused. A memory is not stored in a single synapse, or within a single nerve cell, but over large areas of the brain. In retrieving a memory the brain performs a pattern matching function, trying to match the current stimulus to the memories stored in the brain. The match does not have to be complete. The brain retrieves memories based on partial information. This is why we sometimes confuse a stranger with someone we know. A glimpse of the stranger matches enough of the pattern that the brain retrieves the memory. On closer inspection we find out that the patterns are not a complete match, and we apologize to the person we have just mistakenly recognized. This partial pattern matching is an essential feature in our lives. It allows us, for example, to instantly recognize a dog, although dogs come in a wide variety of breeds, sizes and colors. The richer the pattern of the memory, the easier it is to match. This is why we often form memories by association. A certain smell will invoke, perhaps, memories of childhood or of a loved one. A certain song might have the same effect. How is this related to multimedia, and specifically to multimedia programs implemented on a personal computer? In several ways: * Multimedia engages more of the senses. A picture provides a richer pattern than words. Pictures plus sound, moving pictures or animation do the same. Multimedia on the personal computer can be interactive, which engages the higher order thinking processes of the brain, thus facilitating learning. * Multimedia on a personal computer lets the learner learn at his or her own pace. * Multimedia on a personal computer lets the learner go over the information multiple times, which also enhances learning. Although the last two points are not restricted to multimedia programs alone, they are an important factor in successful multimedia courseware. Evidence continues to accumulate on the effectiveness of computers in instruction. Dr. G. Philip Cartwright writes in the 1993 EDUTECH Report: "Do Computers Help Students Learn?" "There have been literally hundreds of research studies carried out in which computer based instruction was compared with conventional instruction in a controlled research environment. These summaries reveal that CAI is equal or superior to conventional instruction on the following variables: student achievement, covering both immediate and long term retention; attitude toward both the subject matter and the instructor; and time to complete the task. The generalization is that CAI students realize higher achievement in significantly less time than the conventionally instructed students." Recently EDUCOM published a document titled "101 Success Stories" which documents many cases in which computers and multimedia have helped the instructional process. Programs such as Perseus and the Great American History Machine use multimedia not to entertain or to allow mindless pursuit of facts, but to elucidate, explain, and help students develop theories and test them. OBJECT ORIENTED TECHNOLOGY Lets leave the discussion of multimedia for now and look at another promising technology, the technology of objects. While the technology is not new, its benefits had not touched us until programs, such as HyperCard, used some object oriented technology to make developing software easier. Today there are many more recognizable examples of object technology at work: Authoring programs such as Authorware and ToolBook, drawing programs that treat elements of the drawing as objects such as Corel Draw, and even operating systems incorporating object technology such as NextStep and OS/2 Presentation Manager. Behind object oriented concepts lie many technical terms such as encapsulation, inheritance, polymorphism and instantiation. However it is not necessary to understand the technology to appreciate its benefits. As we explore ScriptX, we will highlight some of the advantages that object technology provides. INTRODUCING SCRIPTX ScriptX is an expression based, interpreted language, which means that every statement you type returns a result. Thus if I type3 + 4, ScriptX returns the result of the expression, 7. One of the benefits of ScriptX is its interactive nature. I do not have to write a lot of code before I can test it. I can actually run each statement as I type it. Everything in ScriptX is an object, including the numbers I just typed in, and the returned result. ScriptX comes with a wealth of object classes, so I can create a new window as easily as I added two integers. A window is just another kind of object. Clocks A unique feature of ScriptX is that one of the object types (classes) that are implemented is clock. Thus you can create clock objects in your program, and assure that events in your program occur at the times you specify, independent of the speed of the machine running the program. You can also synchronize activities in your program and maintain the synchronization, even on a slower machine. As a matter of fact, all of the multimedia objects in ScriptX are derived from the clock class. This makes it easy to start, stop, synchronize, and control the speed of multimedia. The Foundation Authoring Model Another interesting architectural feature of ScriptX is the division of function among what are called models, controllers, and presenters. This is termed the Foundation Authoring Model in ScriptX. A model is an object, or set of objects, that we are interested in. A presenter is an object that provides a view of our object on the screen. A controller is an object which controls the interaction between objects in spaces, as well as the interaction between objects themselves. For example, consider a Text object. It contains a string of text. To show the text on the screen one needs a TextPresenter object. The TextPresenter object handles such issues such as justification, left to right or right to left text display, etc. Consider also a 3D object (these are not currently implemented in ScriptX, but the foundation is there for them). If you wish to show it in two dimensions (for example, the view of a ball would be a circle) you can use a 2D presenter. If you wish to show it in three dimensions, with shading, hidden line removal, etc., you would use a different presenter. The object itself does not change and does not have to be concerned about how it looks. It is the presenter's job to do that. As a final example consider an object that has a temperature property. One presenter can be written to display the temperature as a number, another to represent the temperature as a thermometer-like display, while a third might represent the temperature as a color, so that red is displayed if the temperature is high, and green if it is low, and shades between the two colors can represent the intermediary temperatures. This division between an object and its properties, and the presenter (how the object is displayed), is very powerful. The objects I construct can be simpler. The presenters can be general purpose. Once I construct a presenter for temperature, I can use that same presenter to display the temperature (or any other linear property) of any object. The third element of the Foundation Authoring Model is the controller. A controller is an object that controls some aspect of another object's behavior. For example, the Projectile controller gives an object velocity and elasticity properties. The Movement, Bounce and Gravity controllers are able to control one or more projectile objects inside a space. We can, of course, create other controllers as well. Re-Use of Objects One of the benefits of object oriented systems is that you can easily re-use objects. Consider the case where there is a portion of a program that performs a function you would like to implement, but is perhaps missing a feature or two you would like, sort of like finding a car on the dealer's lot that is the right make and model, but is missing a desired option or two, or is the wrong color. In traditional programming you muck around with it the code to understand how it works and to incorporate the changes. This has the unpleasant side effect of often introducing bugs in the program, making the code more complicated, and perhaps changing the interface to the code, so that other code that depends on it might have to change as well. You can get rid of some of the problems by copying the code and creating a new piece of code by modifying the copy. Of course, the problem then becomes that your program gets too large, as there are several pieces of near identical code performing separate functions. On the other hand, in an object oriented program if an object exists that is close in function or appearance to what you desire to implement, you can create a new object that is derived from the original object. You only write code for the new behavior you wish to implement. All of the rest of the object's function is inherited from the original object. There is no danger of adding bugs to the original, working object, there is no duplication of code, and you don't even have to understand how the original object works. A ScriptX Example Lets use an example to put it all together. Say we have a number of bitmaps representing the movements of a fish. In order to animate the movement of the fish we need to display the bitmaps in succession at a constant rate. To show a bitmap on the screen I need a presenter, in this case the TwoDShape presenter. To change the bitmaps I will need a clock. Although ScriptX has many object types to support animation, we'll do it the hard way by defining an Animation class (Figure 1). This class will accept a collection of objects, and display them one at a time using the rate of a clock I define. Notice that, in true object oriented fashion, I can pass an Animate object a collection of anything that has a visual representation, not just bitmaps. [FIGURE NOT AVAILABLE IN ASCII VERSION] I can now run a program and see the fish moving because the animate object is playing the bitmaps in sequence. However, the fish does not move through the water. By creating a new object type, which incorporates the characteristics of the Animation object and the Projectile object (which incorporates the concept of velocity) we can get the fish to move across the window. While I am at it, I would also like to be able to have the fish reappear on one side of the window after it exits the other side, or the animation is going to be short lived. I can create a controller object that will do this. Again, once this object is created, it will take any object that falls off the screen on one side and place it on the other side, so this same controller object, which I will call Wrapper, can be used in many other programs. I would also like to be able to drag the fish in the window and move it to a new position. ScriptX provides a DragController object which works on Dragger type objects. So, somehow, my fish will have to have the characteristics of the Dragger object. To create my example, then, I define a new class of object called Fish, which inherits the characteristics of: * Animation (to play each cell in succession). * Projectile (so it can move in the window), and * Dragger (so that I can drag it). I will place this object in the window and activate three controller objects to work on the window: * Movement (to move the fish across the window). * Drag Controller (to cause the fish to follow the mouse cursor as a result of a drag operation), and * Wrapper (to move the fish to the edge of the window when it falls off the other edge). Figure 2 illustrates the program. A new object type (class), called Fish, inherits from the Animation, Projectile and Drag classes. I gave the Fish object the behavior I desire not by writing more code, but by having it inherit the behavior from the other classes. [FIGURE NOT AVAILABLE IN ASCII VERSION] I also apply the Movement, DragController and Wrapper controllers to the window, so that these controllers will interact with the Fish object to produce the desired behavior. In other words, the Fish object has characteristics that allow it to be moved, dragged, etc. The controllers utilize these characteristics. For example, the Fish object inherits the velocity property from the Projectile class. The Movement controller looks at this property and moves the Fish object by the proper amount at each time interval. Multimedia Multimedia is well supported in ScriptX. There is built in support for animation, digital audio, digital video, and MIDI. As previously mentioned, all these object classes are derived from the Clock object, so it is easy to set up and maintain multimedia synchronization. However, multimedia is often file based, and incompatible file formats are a problem for a cross platform product. A PICT file is not the same as a BMP file, a AIFF file is not the same as a WAV file, and a QuickTime movie file is different from an AVI file. ScriptX handles this problem by defining its own internal file formats, and providing importers that will convert the various file types into the internal ScriptX formats. A file may be imported "on the fly," or may be imported ahead of time and saved as an object. Importing a file on the fly can be a time consuming process, so the author of a ScriptX title is likely to do it ahead of time. To handle the situation where a file is not available until run time, such as is the case with a generic media player, ScriptX allows one to invoke an external multimedia player. ScriptX also allows one to manually control multimedia devices via an MCI command interface, and through a similar mechanism on the Macintosh. Other Features A vexing problem in object oriented systems is that they normally cannot permanently store the objects they have created. Storing an object is not as simple as writing the object to a file. The object might reference (point to) many other objects. These references, which are usually implemented as internal pointers, are a problem because when the object is read back at another time the objects might not be in the same place anymore. In C++ all of the objects are built when the program starts, from the code in the program. In other words, a C++ program starts with cookie cutters (class definitions) from which the program creates the necessary cookies (objects). However, objects created by the program as it runs cannot be stored. To solve this situation third party object stores are available. ScriptX, on the other hand, implements its own object store. What this means is that a ScriptX program can start out with the cookies - it does not have to make them. It can also store new kinds of cookies it makes while it runs, so they can be used by other programs. ScriptX uses the Bento architecture (also to be used in OpenDoc) as the mechanism for implementing its object store. One of the facts of life of an object oriented program is that when it runs, objects are created and others, no longer needed, are destroyed. The programmer does not manage memory as is done in C, where to create, say, a new array, the programmer must invoke the Malloc function to grab some memory, and remember to free it later. The cost of running an object oriented program is that it must include what is called a garbage collector, a part of the program that scans memory and finds objects that are no longer in use and removes them from memory. This is not a simple process, as in object oriented programs objects often reference other objects, so it takes some doing to determine whether an object is to be discarded or not. The problem is that when the garbage collector kicks in, the program ceases to run for a few seconds while the garbage collector does its job. This is of necessity, since the state of objects should not change when the garbage collector is examining them. In most programs this is at most an annoyance. In a multimedia program it is not acceptable. For this reason, the garbage collector in ScriptX (patent applied for) runs on a separate thread in the background and does not cause the program to hiccup. Although it has not been mentioned before, ScriptX is a multi-threaded program, which implements its own thread system so that it can run in single thread environments such as Windows 3.1 and Macintosh System 7.5. ScriptX supports the Unicode standard. In addition it supports the ISO 10646 standard for representing characters as 4 byte quantities. The storage requirements are minimized, as ScriptX automatically recognizes ASCII characters and stores them as one byte values. ScriptX is also architected to represent text from right to left, and from bottom to top, to be able to render text in several foreign languages and scripts. The Dynamic Nature of ScriptX Perhaps one of the most interesting features of ScriptX is its dynamic nature. That means that new objects, say, from an object store, can be added to the system while it is running and these objects will be incorporated into the system dynamically. It is not necessary to recompile the program. Of greater importance is the meaning that the author does not have to plan all of the interactions between objects in the program in advance. New objects can be added later, and they will adapt to the program in much the same way that a child adapts to a new neighborhood. The potential of this feature is that independent parties can create libraries of objects for ScriptX, in the same way that vendors now create and sell libraries of clip art. One can envision Chemistry objects, Biology objects, Psychology objects, and many others. Thus an author can build his or her own ScriptX program, but populate it with existing objects they did not have to create. ScriptX Today Where does ScriptX stand? This fall the ScriptX team at Kaleida has been busy working on footprint and performance issues. Any significant new technology such as ScriptX tends to stress the currently available hardware platforms. This was certainly the case when Windows was first released. Given the rapid advances in platform speeds and capacities, this should not be a problem for long. ScriptX is slated for release on December 16 of this year, initially on both Windows and Macintosh platforms, and soon thereafter on OS/2. Perhaps a more significant concern for faculty authors is the lack of a visual front end for ScriptX. While ScriptX will ship with tools to translate some aspects of Macromedia Director and Asymetrix ToolBook programs, these are stop gap measures at best. Kaleida hopes that third party developers will develop such tools, in the same way that companies such as Watcom developed Visual REXX for the REXX language. Once such tools are available, ScriptX should provide an exciting, powerful and cross platform enabled multimedia authoring tool. It is also a terrific tool on which to learn the ins and outs of object oriented concepts and programming. The presentation at CAUSE94 will include demonstrations of live ScriptX code and examples.