A Case Study in Web 2.0 Application
Development

Paul R. Marganian
Software Engineer – National Radio Astronomy
Observatory, Green Bank

Authors:
Paul MarganianMark Clark
Amy Shelton
Mike McCarty
Eric Sessoms

2

Introduction•••

Discussing our recent experiences creating a Web 2.0 application: the new
Dynamic Scheduling System (DSS) for the Green Bank Telescope (GBT).
Addresses ADASS focus topic: Commonly available development
environments & tools
What is Web 2.0?
– Wikipedia: “is commonly associated with web development andweb design that facilitates interactive information sharing,interoperability, user-centered design and collaboration on theWorld Wide Web.”– Desktop quality applications for the Web Browser.

Oct 7, 2009, ADASS XIX

3

Overview
Background
Beta Test
Release4

5

Background: The GBT
•

•

•
•
•

The Robert C. Byrd Green Bank
Telescope is the world's largest fully
steerable radio telescope.
The Green Bank Telescope (GBT) is
located at the National Radio
Astronomy Observatory's site inGreen Bank, WV.
100 m diameterCan observe over a larger range of frequencies than other comparable
centimeter/millimeter single-dish telescopes.
Is located in a continental, mid-latitude region where weather is dominatedby water vapor and small scale effects.

6Background – The Problem: Dynamic
Scheduling•••

The primary goal of the DSS is to improve the efficiency of GBT
observations by matching the observing schedule to current weather
conditions, while allowing each observer to retain interactive control of
the telescope
Example: use the GBT for high-frequency science for clear, calm weather,and schedule low-frequency science when the weather is otherwise.
Exceptional Scheduling Paradigm: not scheduling scripts, but schedulingobservers.

7

Background – Problem: Dynamic
Scheduling••

How it works:– Each day the DSS will examine the weather forecast, equipment
availability, observer availability, and other factors, and set an observing
schedule for the 24-hour period beginning the next day.
– Observers will therefore get 24-48 hours notice before their projectwill observe.
– Additionally, observers will have the opportunity to pause theirobserving program, set blackout dates indicating when they areunavailable for observing and back out of current observations if theyfind the observing conditions are not suitable to their science goals.
ADASS talk on this last year, by site directory, Dr. Karen O’Neil; along withseveral posters.

8Backround – The Problem: Dynamic
Scheduling••

Since we’re scheduling observers, the observers have to have a high levelof interaction with the system.
Hundreds of geographically dispersed users.
Observers
Schedulers

Operators

DSS
9Background – The Solution••••

What is required to dynamically schedule the GBT is not just complex
scheduling algorithms (though we certainly need that)
We also require complex interactions with all the users.
Solution: A Web Application!
And not just a traditional series of web pages, but a Web 2.0 Application.

10

Background – The Green Bank Software
Development Division
•
•
•

•

We have to implement this solution.
Who’s ‘we’? ~2.5 FTEs over the project life-time.
Until quite recently, we have focused on three main areas of activity,
utilizing three languages:– Monitor & Control – C/C++– Observation Management – Python– Data Analysis – IDL
What type of skill-set and knowledge-base are required to make a web
application?

11

Background – Web Applications, Grossly
Simplified
Concepts: Database, Server, HTTP, Authentication, HTML, CSS, JavaScript, DOM,
AJAX, XML, JSON

DB

Server(Apache,Applica8on
Code)

HTTP

Browser(HTML, CSS,JavaScript)

12

Overview
Background
Beta Test
Release13

Frameworks
•
•

•
•

As discussed previously, many diverse components involved in a web
application
A framework pulls a lot of these components together for you with anumber of benefits:– Manages many crucial activities by default– Enables a web application to get up off the ground quickly– Allows you to work in the language of your choice
Many frameworks available, several in the language of our choice, Python
For the Beta Test (2008), the choice came down to two: TurboGears &Django14

TurboGears – a Python web framework••

All in python.
TurboGears follows the MVC design pattern: isolates business logic from
input and presentation.
– Model: the domain-specific representation of the data on which theapplication operates.
– View: Renders the model into a form suitable for interaction– Controller: Receives input and initiates a response by making calls onmodel objects.

15Model – Object Relational Mapper
(ORM)
•

•
•
•

Wikipedia: “in computer software is a programming technique forconverting data between incompatible type systems in relational databases
and object-oriented programming languages”
Can be a good compromise if data persists in a DB, but you’d like to workwith objects.
We used SqlAlchemy as our ORM
Worked very well for our purposes:– DSS DB is fairly complex, but mapped to objects fine
– Performance not an issue; DSS DB size is moderate

16

Controller - CherryPy
•
•
•
•
•

Wraps the HTTP Protocol
CherryPy maps URLs from the browser to python methods on the server
Each method can then query the model, then render a template (a pagerefresh to the user), or redirect to another URL.
Example: url = /addValues?a=1&b=2
Calls method addValues, with keyword-value pairs a = 1, b = 217View – Genshi Templates•••

A templating language that allows you to embed python in html.
Excellent way of separating your display style from your content.
Example: building a table dynamically.

18

TurboGears – other features
•
•
•
•
•

With all the previously mentioned tools, you can get a functioning web
application up and running quickly.
More tools for making working with forms (validation, etc.) very easy.
But apart from a few special widgets (e.g. date picker), web pages using justthese components are mostly form based (very Web 1.0).
TurboGears allows you to enhance your web pages with CSS andJavaScript, and supports AJAX.
This is the basic approach we took to the Beta Test’s Observers’ Interface

19

DSS Beta
Test
Observers
Interface
Created with:• TurboGears• CSS package:
Blueprint• a sprinkle of
JavaScript.

20

DSS Beta Test Observers Interface••

There some other technologies that bear mentioning in this interface:
iCalendar format – wikipedia: “a computer file format which allows
internet users to send meeting requests and tasks to other internet users,
via email, or sharing files with an .ics extension”

21

DSS Beta Test Observers Interface•••

Other tools used in the observers interface:– RSS – wikipedia : “Rich Site Summary” is a family of web feed formats
used to publish frequently updated works—such as blog entries, news
headlines, audio, and video—in a standardized format.”– Chat software – we incorporated third party chat software so thatobservers can get live help from GBT staff.
Although a more sophisticated interface for observers was desired, these
simpler web pages worked well.
However, for the scheduler’s interface, some of the requirements for theinterface required us to enhance what the browser was doing.

22

DSS Beta Test Scheduler Interface•••

Basically, we had to provide a graphical calendar that supported thegraphical manipulation of observing periods on the telescope, with all the
feedback necessary so that the scheduler can maximize the observing
efficiency of the GBT.
So, we had to move to JavaScript., and make use of AJAX. We took
advantage of a JavaScript library, JQuery.
JQuery provides:– DOM element selections– DOM traversal and modification– Events– CSS manipulation– Effects and animation

23

DSS Beta Test
Schedulers
Interface
Features:• Drag and drop
calendar• Feedback• Color coded
efficiencies• Easy access to project
information
Only supported Firefox
2.0

24

DSS Beta Science Algorithms••••

Without these there’d be no need for the Web Application.
Implemented in Python.
Interfacing them with the web application was trivial: simply imported andcalled from the CherryPy controller methods.
Example:– Scheduler hits ‘Schedule’ in the interace,– which gets mapped to a ‘schedule’ method on the server– Calls ‘schedule’ algorithm in the science python modules

25

Beta Test: Lessons Learned•••

Feedback from users from helpdesk and survey:– Observers interface was actually as sophisticated enough for users.
– Scheduler’s interface was *not* sophisticated enough.
Experience of developers:– TurboGears framework worked well for us.
– Working with JavaScript was pain-full.
– Science Algorithm code much to slow, and error prone.
Used lessons learned to prepare for the production release (Oct. 1, 2009)26

Overview
Background
Beta Test
Release27

Release: Django Framework
•
•
•
•
•

TurboGears served us well in the Beta Test, but in the mean time, theDjango framework really took off.
Django framework is extremely similar to TurboGears. Switching over wasvery simple.
Instead of brining together different 3rd party libraries like TurboGears,
Django uses it’s own ORM, templates, etc.
On the whole offers more features then TurboGears.
Larger user base meant more ‘freebies’. Some of these turned out to save
us a great deal of time.

28

Release:
Observers
Interface
We were able toquickly develop an
observer’s interface
using mostly
features from
Django.
Project Calendar using
the FullCalendar
plugin for JQuery.

29

Release: integrating with other NRAO
systems•••share a single databaseof user contact info
(avoiding the Beta Test
nightmare)other NRAO systems
provide URL query
services that serve up
XML encoded data.
Ex: user contact
information

30

Release: Authentication with CAS•••

NRAO is using an open-source single-sign-on service: Central
Authentication Service (CAS)This way, users can log in to one NRAO service (ex: Proposal SubmissionTool), and seamlessly move to another serve (ex: DSS) without logging in
again.
Django allows you to specify the authentication backend you want to use.

31Release: Django Admin Tool•••

Django comes with a built in Admin Tool, which is basically a sophisticatedview into your database (like phpMyAdmin).
Is extensible to handle custom actions.
Result: functionality that may not be covered in your own interface is often
covered by the Admin Tool.

32

Release: Schedulers Interface•••

Recall that the Beta Test’s Schedulers Interface was not sophisticatedenough.
But creating a more sophisticate interface in the browser would mean
more JavaScript, and we’d had enough of that in the Beta Test (evensupporting just one browser!).
Decided to be bold and try out a completely new technology: Google
Web Toolkit

33

Google Web Toolkit
•

•
•

•

Google Web Toolkit (GWT) is an open source set of tools that allows
web developers to create and maintain complex JavaScript front-end
applications in Java.
Using GWT, developers can rapidly develop and debug AJAX applicationsin the Java language using the Java development tools of their choice.
When the application is deployed, the GWT cross-compiler translates theJava application to standalone JavaScript files that are optionally obfuscatedand deeply optimized.
Creating an advanced UI for the browser feels like developing a typical Java
Desktop interface.

34

Google Web Toolkit
•
•
•
•
•
•
•
•

GWT Java-to-JavaScript Compiler – code in Java!
GWT Hosted Web Browser – run app as Java in the JVM; great for
debugging
JRE emulation library – common features of java.lang and util for JavaScript
GWT Web UI class library – a set of custom widgets
A set of custom interfaces and classes for creating widgets.
GWT handles all cross-browser issues for the developer.
JUnit integration
Lot’s of 3rd party libraries; including one for even fancier widgets: Ext GWT

35

GWT and Eclipse
We found it
easiest to
develop
GWTwith theGWT
plugin for
Eclipse:36

Ext GWT•••

Provides a library of advanced widgets for GWT
See the demo at: http://www.extjs.com/explorer
We took advantage of the Grid libraries

37Release: Scheduler’s Tools with GWT••Created using
GWT and the
Ext GWT
Grid library.
Emulates a
spreadsheet
for project
data.

38

Release: Scheduler’s Tools with GWT••Createdusing
GWT
and thegwt-calpackage
Emulatesa googlestylecalendar39

Scheduler’s Tools with GWT: conclusions••••

Team was inexperienced using Java & Eclipse. A learning curve when wereally didn’t have time for it.
Using GWT to emulate some of the more basic Desktop widgets (panels,
combo-boxes, etc.) was relatively straight-forward.
Implementing the more advanced widgets, such as the grids (Project
Explorer) with Ext GWT were more difficult.
So, is the problem Ext GWT, GWT, or us?

40

DSS Release: Science Algorithms•••

Recall that in the Beta Test, our Science Algorithms were coded in Python.
This made it trivial to integrate into the application, but was slow anderror prone.
For the release, we again decided to make the bold move of rewriting thealgorithms in a completely different language: Haskell

41

Science Algorithms with Haskell
•
•

•
•
•

Haskell is a standardized, general-purpose purely functional
programming language, with non-strict semantics and strong static typing.
Purely functional is a term in computing used to describe algorithms,
data structures or programming languages that exclude destructive
modifications (updates). According to this restriction, variables are used ina mathematical sense, with identifiers referring to immutable, persistentvalues.
A different programming paradigm then Python, C/C++.
Referential Transparency: put the same thing in, always get the same thingout. It’s State-less.
No ‘for’ loops: everything is recursion and pattern matching42

Science Algorithms in Haskell
•

•

Pros:
– Reduced code size by a factor of 2
– Increased performance by a factor of 10 !!!
– Mathematical nature of Haskell more closely resembles science.
– Code easier to understand by team scientists
Cons:– Steep learning curve: a completely different programming paradigm forus– Not as many practical third party libraries as a language like Python(though enough)– Will someone be around to support this in 10 years?
– Doesn’t interface easily with other languages (just C right now)43

Putting it all together:••

When you use what you think is the tool for each part of your problem,
how do you get them to all work together?
Proxy server – let the browser think there’s just one server

44

Conclusions
•
•
•
•
•

Inexperienced team delivered two web applications (Beta, Release) ontime and on budget, using new tools and environments.
We would strongly recommend using some kind of framework.
There’s lots of tools and resources to help build a better user experience(iCal, RSS, etc.).
GWT has a lot of promise. Would strongly recommend it if you arealready familiar with Java.
Haskell is a great programming language, but is it worth the risk?

45