Ron Block, ACT
Henry Jaime, ACT
Jude Poole, Biology
Administrative Computing and Telecommunications (ACT), the group two of the authors work for, is the central IS shop for the campus, housing an IBM mainframe, and numerous Unix servers. All of the Campus' core financial, payroll and student systems run on ACT's mainframe. Key data from these applications needed for decision support and other operations is provided via a campus data warehouse based on Sybase and hosted on Solaris servers.
The Biology Department at UCSD, where the third author works, is the largest general campus academic department with more than 3,000 undergraduate majors. Biology makes heavy use of ACT core applications and also provides local applications to its users which handle everything from research grant financial information to student activities.
Java development, both at UCSD and in general, can be somewhat artificially divided into three classes: applets, applications and servlets. Both ACT and the Biology Department develop servlets, ACT develops applets and Biology develops applications. We will briefly describe some things we are doing with each, how we are pursuing development in each area, why we made the decisions we did, and some things we learned along the way.
In October 1997 ACT completed a new client/server application to be used in each department by designated departmental timekeepers for recording employees hours. A major design requirement was to provide a grid area which lets the user view and update a variable number of time lines for each employee. This program has now been used for 2 years by 225 timekeeper users, updating a monthly average of 21,500 lines for 8700 employees. This program has allowed our campus to eliminate the time consuming tasks of distributing paper timesheets to each department, collecting and keypunching the filled out timesheets, and producing edit reports which have to be verified by each timekeeper.
While this program has been very successful, being a client/server application requires that the application program and associated vendor software be individually installed and maintained on each timekeeper's PC. This often makes the timekeeper dependent on having technical support personnel available for software installation. Additionally, running the program on the most current operating systems would have required an upgrade to existing vendor software and significant system testing.
Decision to use Java applets
In early 1998, a decision was made to replace all campus-wide client/server applications with web-based applications. While most of the reporting applications could be written as CGI/HTML, an HTML presentation for timekeeping would not have provided a user interface that was equivalent to what the users had with the client/server product. In order to duplicate the ability to rapidly update lists of data, we decided to try building the application as a Java applet. Our decision relied heavily on our knowledge that the projected users were running primarily on fast campus networks with relatively powerful PCs/Macs. We also believed that most users would use the application over a long enough continuous period of time such that the load time for the applet would constitute a relatively insignificant portion of a typical working session.
Approach
One of our first decisions was the selection of an Integrated Development Environment (IDE) tool which would help with the GUI development of the application, as well as provide an integrated environment for coding, compiling and testing the program. ACT chose JBuilder, primarily because it allows relatively free-form changes to be made to the generated code. We also decided to develop the code using jdk1.1 which was just starting to be supported by the browsers.
Since our major goal was to develop a web replacement for a specific client/server product and complete it within a given time period, we decided to keep it as simple as possible without doing anything that would be too time consuming or too dependent on developing standards . We developed Java classes that were needed for the application, but did not attempt to design and develop reusable enterprise classes. We decided to perform all database access from within the applet using the Java Database Connectivity (JDBC) driver from Sybase. We also decided to avoid attempting to print from the applet, planning to use some combination of Perl/CGI, HTML, and PDF.
While several ACT applications required complex interactions with the users as described above, many of the applications provided by ACT for general campus clients use standard CGI/Perl to produce HTML. The use of a Perl/CGI framework was a reasonable and successful strategy for building these applications, but ACT came to believe that a more scalable solution was needed to support the rapidly increasing customer base and application development.
Decision to use Servlets
The existing Perl/CGI framework was well received by the users, but several concerns arose regarding the use of Perl/CGI. Perhaps chief among these concerns were performance and scalability. The Perl/CGI framework was written in such a way that converting it to use such techniques as fast cgi would have involved an almost complete re-write. Absent such a change, the framework was stuck in the usual CGI lifecyle involving a very large number of processes starting up and exiting immediately after serving one request. Unlike such CGI processes, Java Servlets can easily maintain state between sessions and maintain persistent resources, such as database connections. As a result, we believe that server-side performance can be improved significantly.
Another important reason for using Java Servlets to build the application framework was to take advantage of the benefits of Object Oriented Design/Object Oriented Programming (OOD/OOP) and the robustness of the Java language. An added benefit of using Java and OO was the increased potential for collaboration between campus departments interested in building common class libraries.
Unlike ACT, the Biology Department has not made a significant investment in CGI applications and so its decision to use servlets has a slightly different rationale. While Biology administrative users are accustomed to highly polished interactive applications which are difficult to match using HTML, non administrative users in the department are just beginningto directly access departmental database information and so have few pre-conceived interface notions. For these users the convenience of simply pointing a web browser at a URL is much more important than the highly customized interaction possible with applets or applications. Thus for business applications aimed at users whose primary job duties are not administrative, Biology has turned to servlets. These servlets, however, rely in most cases on the exact same object libraries (java packages) we use in our applications.
Approach
ACT began its transition to servlets by selecting and training existing staff members to use object oriented techniques and Java. The senior among those who advanced most quickly evaluated various servlet engines and settled on JRun as the deployment platform. Once staff was trained and JRun was selected, a migration plan was developed with the ACT Systems support staff. This involved a redesign of the existing CGI framework and identification of business, control and GUI classes.
The Biology Department has a much smaller development staff and followed a somewhat less formalized approach. Biology did not develop a complete framework, but rather concentrated on some specialized class libraries to handle parameters, generation HTML and PDF, and persistence of back end resources.
We have great hopes that the fruits of these two efforts can be successfully merged.
The Biology department has used a variety of tools for general database programming including the Ingres 4GL based application development environment, custom Tcl/TK programs, OMNIS 7 applications and a variety of other languages. The applications created using these tools worked quite well, but the development support costs were significant and changing platforms was quite difficult.
Decision to use Java Applications
Faced with these problems with our then current development practices, Biology reviewed both current methods and a large number of alternatives. While a number of different approaches would have been adequate, it was felt that Java, particularly with its cross platform promise and strong object oriented foundation was the best option.
Once Biology made the decision to move to Java for all future development, we first focused on porting existing applications which had problems or needed significant redesign. Mostly these were traditional data entry applications for browsing, entering and updating a SQL based database (Sybase in our case). Most of the applications also have one or more complex functions such as ledger reconciliation and groupware like shared data interaction.
Approach
We decided very early on to focus on developing supporting object libraries and build our applications on top of these. Biology does not face quite the same urgency that ACT does for its applet projects, but at the same time has far fewer programmers than ACT. It was an absolute requirement that Biology reduce the maintenance and development costs associated with application development; the development of sharable support libraries seemed the only way to do that.
While Biology chose to create applications instead of applets, we still take advantage of Java's ability to do dynamic network class loading. Applets by their very nature use such class loading, but Java makes it quite easy to extend this mode of operation to traditional applications. Our most obvious use of this technique is our NetLoader. This is a stub class that implements only a custom class loader. This class loader looks to a hardcoded URL (which can be overridden on the command line or from within the program) and retrieves a list of applications, descriptions of those applications, initial starting classes for those applications and supporting class URLs. The user can select any of the applications and NetLoader will start that application, loading any necessary classes and resources over the network.
Development Coordination/Management
Perhaps the most serious management issue associated with Java is finding and keeping experienced Java programmers. Hiring such programmers from outside one's organization is both time consuming and expensive. Most of the Java programmers at UCSD are either long term employees who moved from other languages or are students/recent graduates. We believe that it is very important to be realistic about the time needed for such programmers to gain experience; to fully transition to objected oriented Java programming may in some cases take years.
In part because of the shortage of Java programmers and the resulting staff turnover problem, documentation is critical to Java projects. Java has a built in documentation system called Javadoc. It is both a commenting convention and a tool for creating HTML based upon those comments. Use of Javadoc will generally improve developer coordination, and we recommend it highly. It is also a very efficient way for managers to review some aspects of their programmers work. By itself, however, Javadoc is insufficient to properly document a Java project. It does not provide a convenient way to present a high level overview of a given package. Such an overview is particularly important for bringing programmers new to a project up to speed.
In addition to formal documentation, certain code design techniques can greatly ease managing Java projects. Java interfaces define an API without providing an implementation and therefore capture the essence of an object oriented programming contract between developers. The use of interfaces also greatly simplifies changing implementations and so is a great benefit to managers who may wish to defer tasks such as optimization without creating future havoc for their projects.
Interfaces are an excellent way to support partitioning of Java work, but other object oriented techniques common to many languages should not be neglected. For instance, Java provides strong language support for data hiding. Rigorous use of private members, in addition to being good programming practice, will make it much easier to share responsibilities within or between Java projects and provide clear boundaries for developers.
Object Design
The background of the administrative Java programmers at UCSD is quite varied. Backgrounds range from mainframe cobol to Macintosh pascal. Regardless of what non Java background a programmer might have, experience in applying object oriented principles using Java specifically is critical. Despite superficial similarities to C++, Java is quite different, especially in the way it is used. It is a nearly pure object oriented language. Writing a Java program, whether it is an applet, an application or servlet, can reasonably be viewed as a process of designing objects and interfaces, implementing them, and then wiring them together. The key to success in this type of environment is proper object design.
The objects to be designed can be broken down into two broad categories: reusable library objects and custom, implementation specific objects. The design of library objects is in most cases significantly more difficult and has a much more significant impact on the overall success of an object oriented project. It is crucial to give programmers who will be designing such libraries the training necessary to fully master object oriented programming and the time to develop experience designing in the Java environment. The design of the API to a given library is much more important than the actual implementation since Java allows you to cleanly hide the implementation details, but every piece of code written against a library will have dependencies on the API.
A big advantage of Java is that one does not have to design as many libraries as in some other languages. The core Java classes provide rich functionality and for many uses one can use these core classes. One key efficiency in object design is knowing when it is not needed and to simply use existing objects, including the numerous free and commercial third party libraries. ACT, for example, has had significant success with the KLG widget libraries.
Design of more mundane objects, while perhaps not as difficult or critical as that of library objects, is nonetheless a nontrivial skill. Good object design principles and knowledge of design patterns will prove invaluable to programmers working in Java. Code written without adherence to object oriented design principles may well work, but will not reap all the possible benefits of Java.
JDK Version Issues
The Java language version is usually indicated by the version of the associated Java Development Kit (JDK) from Sun Microsystems. The JDK's released so far fall into three key versions, 1.0.x, 1.1.x and 1.2.x. (A beta of 1.3 has also been released). Sun added some confusion to the mix by officially changing the name of Java 1.2.x to Java 2 (though both Sun and third parties will frequently use the 1.2 designation). In addition to the version number confusion, Sun has both Reference releases and Production releases. Reference releases are released first and are meant to define functionality and API's. Production releases come later and are optimized versions of the reference releases.
JDK 1.0.x has major problems, perhaps the most serious of which is the horrible event model for the AWT. In addition, several very important Java technologies, notably JDBC, were outside the core or missing in 1.0. While code written to the 1.0 API is generally upward compatible (though import statements are a problem for API's such as JDBC) , few choose to use the deprecated API's if they can avoid it. Since we can avoid it, we make no effort to remain 1.0 compatible.
JDK 1.1 introduced some major improvements. The AWT model in JDK 1.1 is much cleaner than the 1.0 version. Inner classes have been added and careful use of them can lead to cleaner, more easily understood code. Various API deficiencies were also remedied. Key new API's were added to the core, including, perhaps most importantly in our view, JDBC and servlet support. For these and other reasons (except in those cases where you must deploy applets in older browsers without 1.1 support) it is our recommendation that 1.0 not be used and that code should be written to take advantage of the improvements in JDK 1.1.
After the release of 1.1 and before the full release of 1.2 some major new packages were released by Sun. Swing, the part of the Java Foundation Classes (JFC) which provides a cross platform UI with pluggable look and feel is probably the most well known. Most of these packages have versions which work under 1.1. Wherever possible we use these packages in our 1.1 environment.
JDK 1.2 is a release which brings many nice features, but also a fair bit of controversy. Unfortunately we do not have the space to review the important issues to consider when choosing between 1.1 and 1.2.
Runtime Environment Issues
All Java programs run on a Virtual Machine (VM). While the VM's should (theoretically) conform to the Java Language Specification there is tremendous variation in speed, bugs, garbage collection, etc. We have stayed pretty much with the mainstream VM's, since by and large they provide us with the stability and performance we require. Below we discuss some concrete information on the VM's we have used and note some other VM's we think are worth considering.
The simplest VM choice exists on the Macintosh. There is only one VM under active development: the Macintosh Runtime for Java (MRJ). Earlier versions were quite poor and almost everyone uses the most recent version. Apple's version scheme for their runtime is slightly confusing: MRJ 1.5 is a JDK 1.0 environment; MRJ 2.0.x and 2.1.x are JDK 1.1 environments. Since the latest MRJ is a 1.1.x VM, one cannot run any programs which depend on 1.2 features (though one can use Swing and other post 1.1 features which have a 1.1 implementation available). MRJ has sluggish AWT performance and less than ideal I/O speeds (both largely as a result of Macintosh architecture issues). The most recent versions are largely stable and one of the Biology programmers uses Metrowerks on the Macintosh as a primary development platform (for work not requiring 1.2 features).
Under Solaris we use the JDK VM. The Java 2 VM's are much faster and we don't have any particular reason to use 1.1 VM's for anything other than testing. We have not done extensive testing of HotSpot, but what little testing we have done has shown no serious problems (and unfortunately no particularly huge performance win).
Under Windows we also the JDK1.2 VM. Its 2D performance is slower than we'd like, but we really like the support for custom class loaders. We have also used IBM's free VM, which is blazingly fast, but only supports 1.1.8 (they recently upgraded from 1.1.6). When IBM releases a 1.2 compliant VM we expect great things.
We don't run Java under Linux at the moment, but expect that IBM's 1.1.8 VM would be the one to use, at least when it gets beyond alpha. The Blackdown group is porting 1.2 to Linux with Sun's active support, but their timetable is quite unclear at the time of this writing.
For those who need absolute top servlet performance, TowerJ is worth a look. It is a compiler for Java in the more traditional sense. It heavily optimizes at compile time and creates a machine code executable rather than class files. This executable is then run like any other executable for the target system (and, of course, cannot be run on a system with a different architecture or operating system). TowerJ doesn't yet support the AWT, so one cannot use it for general Java work, but its benchmarks for servlet work are impressive. We would recommend carefully comparing other VM's performance for your particular configuration, however, since it is a commercial product and the other top VM's are essentially free.
ACT's applets are carefully coded to work with the Java Plugin from Sun. In addition the Netscape PC VM is supported.
Neither ACT nor Biology uses the Microsoft JVM. Biology programmers suggest careful thought be taken when considering this VM; it was once the clear performance leader, but current development plans at Microsoft are unclear at best.
When reviewing our experiences on various VM's, we believe that Sun's claims of "Write Once; Run Anywhere" are overstated. Many of the problems, however, come from VM bugs. As the VM market has matured we have seen fewer of these problems. And it should be noted that we have been very happy with cross VM development. Biology develops on Solaris, NT and Macintosh (except for 1.2 development) for deployment on all three platforms. Biology was once a heavily Macintosh shop. In part because of poor support for Java, we have moved away from Macintoshes. Planning for Java projects on the Mac will be complicated significantly by Apple's continued refusal (at the time of writing) to discuss any specific plans with regard to MRJ, especially any related to Java 2.
There is one final VM issue everyone should be aware of. The one area where the Java specification gives implementers a great deal of freedom is the thread priority model. While many VM's implement a similar model, it is quite possible for a VM to be completely compliant with the spec and yet have thread prioritization characteristics which surprise programmers. Anyone writing heavily multi-threaded applications should very carefully test on every target VM (which is good advice in general, but particularly critical here).
Performance
Java's performance has been an issue since the first release. The earliest versions of Java were very slow, but the situation has improved significantly in the few years since then. For our applications, current performance is more than adequate. Since both benchmarks and our first hand experience indicate that VM speed is improving steadily and significantly we have no real worries about this issue.
Despite our confidence, we would caution new Java developers that there is nothing particularly magical about Java or the new fast VM's. Poor algorithms, slow network interfaces, old hardware and all the other traditional causes of performance problems afflict Java programs at least as much as those in any other language. In fact, older hardware is often a more serious problem for Java programmers than C or C++ programmers because of the memory requirements of the VM's.
It is prudent software engineering to design and test with performance requirements in mind. Decisions about Java tend to be somewhat more complex, however, because of the large number of runtime environments to choose from. In our experience, fortunately, we have not had to make any major design or coding decisions because of VM performance limitations.
Browser Environment and Applet Issues
A Java applet is downloaded from a server as byte code and runs in a web browser. All current browsers come bundled with a virtual machine that is specific for the platform it runs on (i.e. PC, Mac, unix). Each virtual machine supports a particular version of Java and, although it must follow certain standards, will have certain variations between vendors.
The most current versions of Netscape and Internet Explorer support different versions of the Java 1.1 standard. On a PC, the only way to override the virtual machine of the user's browser is to code the applet HTML to require the Sun Java plugin. When requested by the Java HTML, it prompts the user to accept a download of the plugin software if it does not already reside on the local machine. On a Mac, although both Netscape and Internet Explorer provide a bundled virtual machine, the only viable virtual machine is the MRJ from Apple. Internet Explorer provides a method for configuring the browser to use the MRJ through a preference option.
For our timekeeping applet, we use a Perl CGI program to build the applet HTML. The Perl script detects the OS, the browser type, and the browser version. It uses this information to determine whether the java plugin is required and whether the proper Netscape or MRJ version is being used.
It is necessary to test an applet using every browser and virtual machine it will eventually run on. Despite the goal of write once, run anywhere, there are differences in virtual machines which can effect how the applet runs.
We also found the browser back button introduced some potential for confusion and errors. Once an applet was running and the user hit the back button and then forward to return to the applet, some browsers reloaded the applet and therefore the user could lose data entered but not saved to the database. We solved this problem by having the applet launched in a separate browser window, eliminating the ability to move away from the applet within that window.
When we began testing with different browsers on PCs, we found certain differences between Netscape and Internet Explorer as have many others attempting to build cross-platform applications. Netscape version 4.07 and greater was able to run the applet, but we found some navigation problems within the grid component when using Internet Explorer. We decided to use the Sun Java plugin product for Internet Explorer users to get the navigation to be consistent with Netscape users. On Macs, we were not able to run the applet until the Apple MRJ2.1 was released in early 1999. (The MRJ is a freely distributed product from Apple which must be downloaded and installed on each Mac). We also found that running the applet on a Mac platform required a higher-end Mac with at least 64MB of memory to obtain satisfactory performance.
Testing of the applet was more difficult and time consuming than comparable testing of a client/server application due to the number of pieces involved (such as the Java JDK, vendor components, JDBC drivers, browsers, and operating systems) and the retesting that must be done when any of these pieces change. It is also critical to perform testing on as many combinations of platforms and browsers as is feasible.
Server Side Communications and Database Access
For all our database access we use JDBC. This is the standard Java package for accessing data sources and code written to the JDBC standard should be portable to many SQL based databases (provided one does not use vendor specific extensions). So far we have relied almost exclusively on jConnect, the JDBC driver from Sybase (which is now free) and JDBC 1.x.
We use some wrapper classes to provide extended database support and to insulate our applications from the details of JDBC. One of the first things most java database programmers run across is the limitation of the JDBC ResultSet. ResultSets hold the results of queries, but they are unidirectional data structures -- once the programmer has advanced beyond a row of data it is no longer accessible. This is not a huge problem if you are sequentially processing the data, such as in a report, but for interactive form type applications one usually needs to store the data and perhaps scroll through it.
Security
By default, applets have no access to system resources outside the directory from which they were launched. This prevents an applet from being downloaded to a users PC and either reading or writing files on the local drives. This can be overridden by signed applets which must be accepted by the user.
Applets are also limited to the server resources they can access. A downloaded applet is restricted to accessing databases that reside on the same server from which the applet was downloaded, or may go through a proxy server that is running from the applet download server to access database on other servers.
Printing
Java support for printing started out quite poor and has evolved to semi-poor. We have chosen to sidestep the standard Java printing architecture and output documents as either HTML or PDF.
The HTML we use is generated primarily by servlets and, since it is intended for a browser environment, fits very well with user expectations. Unfortunately HTML is less than ideal at meeting precise printing requirements. Page breaks, headers, printed pages which look substantially different than the rendered HTML (lack of colored background for instance) are just some of the problems encountered by users. Designing an HTML document which looks roughly the same for all browsers (under all possible user preferences) is pretty much impossible. DHTML, style sheets, XML and similar technologies promise to address these problems, but none are completely here yet and it will be some time before all browsers support such features. Despite all these problems, HTML output is generally functional and the ability to view and print it is nearly universal.
For more precise printing requirements we use a custom developed Java PDF library. We examined a number of available third party libraries, but each had shortcomings that led us to develop our own. We only needed a subset of the capabilities of PDF, so we were able to simplify our development task by ignoring some of the more arcane aspects of PDF. Since PDF is an object oriented file format it is a natural fit for Java -- the core coding was done in several weeks.
Summary
Over the past two years, our Java development efforts have been very successful for developing administrative applications in a university environment. Since Java technology has significantly improved over that time and the trend continues, we firmly believe that our success with Java development will continue.While Java may not live up to the extreme hype which surrounded it initially, we have found it to be an efficient and effective tool for our applications development.
Your success with Java depends on the commitment of upper management and the ability to establish the necessary Java resources. Whether you choose to use Java applets, servlets, or applications, depends on your specific application needs and environment.
Resources
Web Sites