Fall “hackathon” to predict El Niño held at NCAR

By Marijke Unger
12/09/2015 - 11:30pm

As part of this year’s climate informatics workshop held at NCAR this fall, participants also had the opportunity to engage in a data challenge designed to apply new approaches to a large data problem, in this year’s event, predicting the nature of El Niño using statistical methods applied to data from a regional (NCAR) CCSM simulation.

The event, referred to more formally as a Rapid Analysis and Model Prototyping (RAMP) had 28 participants, and took place on September 26, immediately following the workshop, which was hosted by CISL’s IMAGe.

The RAMP encourages participants to test different analytics solutions for a problem and delivering a prototype. “It’s not just about the prototype, though,” said Balázs Kégl, a research scientist at CNRS in France, and one of the event organizers. “The outcomes are multiple: we introduce a scientific problem to data scientists, we train novice data scientists in hands-on analytics, and we benchmark different solutions to a concrete problem. The focus is deliberately on collaboration and efficient exploration.”

“Cooperation is key,” said Kégl. “The goal is to get data scientists to work on real-world problems. Data challenges have constraints, and the idea for the hackathon came from trying to address those constraints to produce hands-on solutions to science problems. Prototyping solutions for problems like climate change, for example.”

The climate informatics problem for this RAMP – a six-month prediction of El Niño using sea surface temperatures –  was proposed by Claire Monteleoni (George Washington University), along with Timothy DelSole and Kathleen Pegion from George Mason University. This RAMP first took place when Monteleoni visited Kégl in Paris in June 2015. Monteleoni, who co-founded the climate informatics workshop series, then invited Kégl’s team to Boulder for the 2015 workshop.

Sample template image
Members of the Climate Informatics Conference steering committee: shown from left to right are Imme Ebert-Uphoff (Colorado State University), steering committee and program committee; Yan Liu (University of Southern California), workshop co-chair; Claire Monteleoni (George Washington University), co-founder of the workshop series, steering committee, and co-organizer of the hackathon; and Doug Nychka (NCAR), steering committee, host, and Director of IMAGe. (Photo by AJ Lauer, CISL)

“I have been personally driven by a vision that data science can shed light on climate change,” said Monteleoni. “We’re involved in starting an experiment in developing an interdisciplinary field. The goal is to use these data analysis techniques that have shown so much promise in other fields to understand some of the remaining questions in climate science.”

The participants worked in teams to formulate solutions, periodically posting their results. As the most effective approaches were identified, other teams could use and build on those approaches to make continual improvements in the methods.

The RAMP teams used the Python programming language to implement machine learning and data mining algorithms, subsequently comparing their performance on the climate informatics problem. It served to solidify the training from the preceding workshop, as well as provide networking and collaboration opportunities for participants.

 “The interesting thing about this is that the techniques for prediction really focused more on computer science methods that are characterized as machine learning or data mining, so the idea was to see how well these data methods could be used for predicting the science, without using a really complex geophysical model to do it,” said Doug Nychka, director of CISL’s IMAGe. “It was just really fun. I would really like to do this again.”

Nychka also noted the importance of support the team received from CISL’s WSST, who set up a backup server for the competition.

Sample template image
Perhaps several days working in python helped prepare the group for this encounter as well. Can you spot the snake? (Photo: Doug Nychka)

At the end of the workshop, participants enjoyed a hike along the trail to Yuri Point, near the Mesa Lab. Claire Monteleoni heroically saved the group from a lurking rattlesnake, earning Doug Nychka’s gratitude in particular, as he was first in line to unwittingly disturb the rattler.

“Claire really went above and beyond,” said Nychka. “Her commitment to solving real-world threats came in particularly handy that day.”