From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows.
Vance, Tiffany C.
Potemra, James T.
Signell, Richard P.
MetadataShow full item record
Keywordocean observation; ocean modeling and prediction; cloud; data management; archiving; technology
Advances in ocean observations and models mean increasing flows of data. Integrating observations between disciplines over spatial scales from regional to global presents challenges. Running ocean models and managing the results is computationally demanding. The rise of cloud computing presents an opportunity to rethink traditional approaches. This includes developing shared data processing workflows utilizing common, adaptable software to handle data ingest and storage, and an associated framework to manage and execute downstream modeling. Working in the cloud presents challenges: migration of legacy technologies and processes, cloud-to-cloud interoperability, and the translation of legislative and bureaucratic requirements for “on-premises” systems to the cloud. To respond to the scientific and societal needs of a fit-for-purpose ocean observing system, and to maximize the benefits of more integrated observing, research on utilizing cloud infrastructures for sharing data and models is underway. Cloud platforms and the services/APIs they provide offer new ways for scientists to observe and predict the ocean’s state. High-performance mass storage of observational data, coupled with on-demand computing to run model simulations in close proximity to the data, tools to manage workflows, and a framework to share and collaborate, enables a more flexible and adaptable observation and prediction computing architecture. Model outputs are stored in the cloud and researchers either download subsets for their interest/area or feed them into their own simulations without leaving the cloud. Expanded storage and computing capabilities make it easier to create, analyze, and distribute products derived from long-term datasets. In this paper, we provide an introduction to cloud computing, describe current uses of the cloud for management and analysis of observational data and model results, and describe workflows for running models and streaming observational data. We discuss topics that must be considered when moving to the cloud: costs, security, and organizational limitations on cloud use. Future uses of the cloud via computational sandboxes and the practicalities and considerations of using the cloud to archive data are explored. We also consider the ways in which the human elements of ocean observations are changing – the rise of a generation of researchers whose observations are likely to be made remotely rather than hands on – and how their expectations and needs drive research towards the cloud. In conclusion, visions of a future where cloud computing is ubiquitous are discussed.
© The Author(s), 2019. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Vance, T. C., Wengren, M., Burger, E., Hernandez, D., Kearns, T., Medina-Lopez, E., Merati, N., O'Brien, K., O'Neil, J., Potemrag, J. T., Signell, R. P., & Wilcox, K. From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows. Frontiers in Marine Science, 6(211), (2019), doi:10.3389/fmars.2019.00211.
Suggested CitationVance, T. C., Wengren, M., Burger, E., Hernandez, D., Kearns, T., Medina-Lopez, E., Merati, N., O'Brien, K., O'Neil, J., Potemrag, J. T., Signell, R. P., & Wilcox, K. (2019). From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows. Frontiers in Marine Science, 6, 211.
The following license files are associated with this item:
Showing items related by title, author, creator and subject.
Signell, Richard P.; Pothina, Dharhas (MDPI, 2019-04-19)The traditional flow of coastal ocean model data is from High-Performance Computing (HPC) centers to the local desktop, or to a file server where just the needed data can be extracted via services such as OPeNDAP. Analysis ...
Mechoso, C. R.; Wood, R.; Weller, Robert A.; Bretherton, Christopher S.; Clarke, A. D.; Coe, Hugh; Fairall, Christopher W.; Farrar, J. Thomas; Feingold, G.; Garreaud, R.; Grados, Carmen; McWilliams, James C.; de Szoeke, Simon P.; Yuter, Sandra; Zuidema, Paquita (American Meteorological Society, 2014-03)The present paper describes the Variability of the American Monsoon Systems (VAMOS) Ocean–Cloud–Atmosphere–Land Study (VOCALS), an international research program focused on the improved understanding and modeling of the ...
Recent advances in Arctic ocean studies employing models from the Arctic Ocean Model Intercomparison Project Proshutinsky, Andrey; Aksenov, Yevgeny; Kinney, Jaclyn Clement; Gerdes, Rudiger; Golubeva, Elena; Holland, David; Holloway, Greg; Jahn, Alexandra; Johnson, Mark; Popova, Ekaterina E.; Steele, Michael; Watanabe, Eiji (Oceanography Society, 2011-09)Observational data show that the Arctic Ocean has significantly and rapidly changed over the last few decades, which is unprecedented in the observational record. Air and water temperatures have increased, sea ice volume ...