Cloud Support for Web Development (EDUCAUSE Quarterly)

By Mike Richwalsky

Cloud services can help colleges and universities meet sudden surges or long-term increases in demand for processing, bandwidth, and storage from the campus community.
The cloud can also support low-risk, low-cost testing of large technology systems before the institution purchases them.
Hosting a large fundraising video on the cloud solved a bandwidth problem for Allegheny College, while testing content management systems on cloud servers reduced risk for John Carroll University.
Balancing the pros and cons of cloud services, including cost and project length, helps in determining whether to look to the cloud to solve IT problems.

The cloud has multiple potential impacts on the business and processes of IT in higher education. One of the areas the cloud has the possibility to affect positively is web development.

In the web world, we often run up against a number of challenges: processing, storage, bandwidth, security, and others. We're often asked to solve complex issues quickly and under budget. The cloud frequently makes that job a little easier.

These days, companies large and small offer compelling cloud-based services that are simple to integrate into our websites. Amazon has a world-class content delivery network. Wufoo does a great job at serving and managing forms online. Basecamp has become a best-of-breed project management platform, and Salesforce is a strong customer relationship management (CRM) tool in the cloud. Some companies even run their PBX systems in the cloud. As Bob Dylan sang, "The times, they are a-changin'."

Perhaps the most intriguing thing about integrating the cloud into your web options is that many cloud services only charge you for what you use. This compares favorably to situations where IT infrastructure has to be built to handle large usage spikes.

Think about the server that handles class registrations. It's only really pushed to the limit a handful of times a year, and the rest of the time it sits, only marginally used, but the hardware, power, and cooling all have to be paid for year-round. Services in the cloud, especially in the world of web development, can easily and quickly scale up to meet demand, and when demand declines, scale back down.

When it comes to the cloud, many people believe it's an all or nothing proposition. In fact, many companies and colleges test the waters to see if the cloud makes sense as a solution for a particular problem or need. At my previous institution, Allegheny College, we started integrating the cloud into our web operations by taking that approach. We didn't know much about the services available, and we had a real-world project with real-world issues to solve.

Managing Bandwidth

Allegheny staff had created a video fundraising appeal to send to thousands of alumni across the world. We shot the video in high definition, which left us with a large video file to embed on a web page.

A constant struggle on campuses of any size is providing adequate bandwidth to meet demand. There's never enough — students playing "Modern Warfare 2" or downloading the latest episodes of "Doctor Who" are eating it up. It didn't take long to realize that inviting thousands of alumni to watch this HD video would quickly overload our services. In today's era of instant-on videos courtesy of YouTube and Hulu, any buffering of a video longer than a few seconds and the user leaves to find the latest cute kitten video. Each person who gives up and leaves your site is one more person who doesn't donate, ensuring the project's failure.

I solved this particular problem by hosting the video at Amazon. Having invested considerable resources in their infrastructure, the company realized that they could resell some of that infrastructure to people at a profit. We used Amazon's cloud storage service, although they also offer a server-on-demand product, database services, and more cloud support products.

But Mike, you say, why didn't you just use YouTube to host your video? At the time of this project, YouTube wasn't serving videos in high definition or widescreen, both requirements for the project. In addition, this video wasn't intended for public consumption — we wanted to control where and how it was hosted and who had access to it.

Storing the video at Amazon meant that visitors to the campaign landing page received the web page and images from my server, but received the video from Amazon's servers, which are spread around the world. The video started quickly and played smoothly for viewers, who remained unaware that it was not coming directly from Allegheny. Thanks to this web-based support, we handled the initial spike in traffic and were well positioned to serve the long tail of visitors as well.

The total cost for file storage and bandwidth from Amazon for the first month was about $6 USD. The performance and cost quickly convinced me that Amazon's Simple Storage Service would become a key piece of my web development arsenal.

Testing

Serving video is just one use of the cloud. Let's say you want to try a new learning management system (LMS) or web framework. Often, it's faster to fire up a server at Amazon or Rackspace to test something than to provision a development server, which you might not have full root access to anyway. By starting a cloud server, you have full control. If you install a library that hoses the machine, for example, no worries — you just terminate it and start again. Cloud servers from Rackspace start at a penny an hour, making cloud testing a no-brainer.

In this video, you'll see just how easy it is to start a server in the cloud.

Ganesan Ravishanker, CIO at Pace University, was charged with quickly getting a new e-portfolio platform online for on-campus testing and review. He started up a server at Amazon and installed a product for use by his constituents. The system was tested, approved, and moved from Amazon to a production environment in two months. His cost? About $60, according to his blog.

At John Carroll University, where I now lead web development, we're currently reviewing content management systems and have, on several occasions, used Amazon and Rackspace servers to quickly and easily try out CMS platforms. Amazon's pre-built server images let me select the type of server I want to run, such as one that already has Drupal or Joomla installed. Once instantiated, I merely have to log in to the CMS and start testing. It's been a valuable tool as we go through this system selection process.

Expanding Storage

At Allegheny, I rolled out WordPress as a content management tool for users across campus. Users were encouraged to add photos, videos, audio files, PDFs, and more to their sites. Knowing that it would require quite a bit of disk space to manage, I installed a plug-in in our WordPress system that took any file a user uploaded and seamlessly moved it to Amazon's storage network for storage and delivery. This activity was completely behind the scenes, and neither our on-campus content creators nor our web visitors knew that the image they were viewing or the PDF file they were reading was coming from Amazon. From an infrastructure standpoint, I knew that we had a near-limitless storage platform and that these files were being served quickly from a global network at minimal cost to the institution.

Backing Up Data

If you're like me, you're paranoid about backups. Still, it took watching an external hard drive full of pictures of my kids, songs, and videos go up in smoke — literally — before I considered using the cloud to help me keep my data safe and secure. Many services, such as Backblaze, Carbonite, and Mozy, automatically back up your desktop computer to the cloud, usually at a fixed cost. Backblaze, for example, backs up my home desktop for $5 a month.

Desktops are easy to back up. What about servers and other web platforms, especially given the fact that your IT team already does a set of nightly and weekly backups, with tapes rotated off-site? As a web developer, I see value in having access to my own backups in case of emergency. I once needed to restore just one MySQL table. It took days to find the tape, restore it, rebuild MySQL on a development server, and get access before I finally got the small amount of data I needed. The difficulty here had nothing to do with server admins, who I think would agree this type of recovery is an arduous process.

After that experience, I began running my own backups and storing them in the cloud.

To back up blogs and sites using WordPress as a CMS, I use the Automatic WordPress Backup plug-in. Based on a schedule I determine, it packs up my WordPress database tables, entries, pages, assets, plug-ins, and themes, zips them up, and moves them to Amazon's storage network. It does so on a schedule I set, and it deletes older backups automatically. You can find similar plug-ins for Drupal and other open-source CMSs.

Don't want to use a plug-in? Check out this script by Eric Nagel that not only zips up your web directories (or any directory you instruct it to), it also dumps your MySQL databases, zips them up, and moves the whole pile of data to Amazon. This makes finding a particular file or database much faster and easier. For a few dollars a month, I have easy access to several days of backups.

Integrating the Cloud into Web Projects

If you're interested in integrating the cloud into one of your web projects, it's good to ask yourself a few questions to determine if the cloud makes sense.

First, what is the duration of your project? If you are working on a three-year project, the cloud might not be for you. Each hour you run a server in the cloud, you pay for it. Need a server for a few weeks to test your code or to transcode a few videos? The cloud would be a good fit.
Second, how much data do you need to move around? If you have terabytes and terabytes of data to crunch, the cloud could do it — but there are costs for storing and moving all that data around, costs that could add up very quickly. You can save some money by actually mailing Amazon a hard drive to upload to your storage account, but that also has a price.
Finally, think about security. People often ask me about this topic when it comes to the cloud. Your data and processes in the cloud are only as secure as you make them. If you are buying service on a cloud server and uploading code and applications, that's no different than a server that lives in your data center. You need to put the same protections on it that you put in your on-campus infrastructure. You will lock out certain ports, make sure you patch regularly and get the most up-to-date software, and so on. If your data is not secure on your own campus, it will not be any more or less secure in the cloud.

I hope you'll take a look at some of the cloud services I've touched on briefly here. I think we'll see their increasing use in higher education web development in the years to come. Getting familiar with them now will help in planning your projects in the future.

Resources

Amazon Web Services: http://aws.amazon.com

Rackspace Cloud Servers: http://www.rackspacecloud.com/

Google App Engine: http://code.google.com/appengine/

The Cloud Market: http://thecloudmarket.com

HighEdWebTech: http://highedwebtech.com

37Signals (Basecamp): http://37signals.com/

HighScalability.com: http://highscalability.com