Employing Task Parallelism to Facilitate Dynamic Comparison of Model Output

08/01/2012 - 6:00pm
FL-1001, Small Seminar Room
Hannah Aizenman

Abstract:

Many climate models, such as the CESM, require extensive and expensive computing and storage resources to analyze them. Since these resources are often out of the reach of many young scientists and small research groups, the goal of this project was to make a commonly used diagnostic tool available through the web so that users could make use of NCAR resources in a simple and public way. The web interface was built using the Pyramid web framework in a RESTful manner so that the tool could be easily customized, maintained, and extended as the needs of the user and application grow. The tool also provides both interface and raw url access to the job dispatch, giving the user multiple methods for using the tool and allowing developers to easily add in new interfaces without destroying function ones.

A major aspect of building the web interface was rewriting the OMWG-CESM ocean model diagnostics c-shell driver scripts as a Python library to simplify running the diagnostics with user defined settings. The library wraps the swift version of the diagnostics to take advantage of the benefits using task parallelism; functions for creating swift configurations were also built into the library to make working with swift easier. The web version takes full advantage of this by running independently configured jobs on every call and returning the results as either an archive, a styled folder, or a plain folder, depending on the url. This flexibility was a major design goal in creating this tool, because that makes it easier for users to fit it into their workflow, which is key to any form of widespread adoption of the tool.