From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows.

dc.contributor.author Vance, Tiffany C.
dc.contributor.author Wengren, Micah
dc.contributor.author Burger, Eugene
dc.contributor.author Hernandez, Debra
dc.contributor.author Kearns, Timothy
dc.contributor.author Medina-Lopez, Encarni
dc.contributor.author Merati, Nazila
dc.contributor.author O'Brien, Kevin
dc.contributor.author O’Neil, Jon
dc.contributor.author Potemra, James T.
dc.contributor.author Signell, Richard P.
dc.contributor.author Wilcox, Kyle
dc.date.accessioned 2019-07-05T18:20:49Z
dc.date.available 2019-07-05T18:20:49Z
dc.date.issued 2019-05-21
dc.description © The Author(s), 2019. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Vance, T. C., Wengren, M., Burger, E., Hernandez, D., Kearns, T., Medina-Lopez, E., Merati, N., O'Brien, K., O'Neil, J., Potemrag, J. T., Signell, R. P., & Wilcox, K. From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows. Frontiers in Marine Science, 6(211), (2019), doi:10.3389/fmars.2019.00211. en_US
dc.description.abstract Advances in ocean observations and models mean increasing flows of data. Integrating observations between disciplines over spatial scales from regional to global presents challenges. Running ocean models and managing the results is computationally demanding. The rise of cloud computing presents an opportunity to rethink traditional approaches. This includes developing shared data processing workflows utilizing common, adaptable software to handle data ingest and storage, and an associated framework to manage and execute downstream modeling. Working in the cloud presents challenges: migration of legacy technologies and processes, cloud-to-cloud interoperability, and the translation of legislative and bureaucratic requirements for “on-premises” systems to the cloud. To respond to the scientific and societal needs of a fit-for-purpose ocean observing system, and to maximize the benefits of more integrated observing, research on utilizing cloud infrastructures for sharing data and models is underway. Cloud platforms and the services/APIs they provide offer new ways for scientists to observe and predict the ocean’s state. High-performance mass storage of observational data, coupled with on-demand computing to run model simulations in close proximity to the data, tools to manage workflows, and a framework to share and collaborate, enables a more flexible and adaptable observation and prediction computing architecture. Model outputs are stored in the cloud and researchers either download subsets for their interest/area or feed them into their own simulations without leaving the cloud. Expanded storage and computing capabilities make it easier to create, analyze, and distribute products derived from long-term datasets. In this paper, we provide an introduction to cloud computing, describe current uses of the cloud for management and analysis of observational data and model results, and describe workflows for running models and streaming observational data. We discuss topics that must be considered when moving to the cloud: costs, security, and organizational limitations on cloud use. Future uses of the cloud via computational sandboxes and the practicalities and considerations of using the cloud to archive data are explored. We also consider the ways in which the human elements of ocean observations are changing – the rise of a generation of researchers whose observations are likely to be made remotely rather than hands on – and how their expectations and needs drive research towards the cloud. In conclusion, visions of a future where cloud computing is ubiquitous are discussed. en_US
dc.description.sponsorship This is PMEL contribution 4873. en_US
dc.identifier.citation Vance, T. C., Wengren, M., Burger, E., Hernandez, D., Kearns, T., Medina-Lopez, E., Merati, N., O'Brien, K., O'Neil, J., Potemrag, J. T., Signell, R. P., & Wilcox, K. (2019). From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows. Frontiers in Marine Science, 6, 211. en_US
dc.identifier.doi 10.3389/fmars.2019.00211
dc.identifier.uri https://hdl.handle.net/1912/24316
dc.publisher Frontiers Media en_US
dc.relation.uri https://doi.org/10.3389/fmars.2019.00211
dc.rights Attribution 4.0 International *
dc.rights.uri http://creativecommons.org/licenses/by/4.0/ *
dc.subject Ocean observation en_US
dc.subject Ocean modeling and prediction en_US
dc.subject Cloud en_US
dc.subject Data management en_US
dc.subject Archiving en_US
dc.subject Technology en_US
dc.title From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows. en_US
dc.type Article en_US
dspace.entity.type Publication
relation.isAuthorOfPublication 4694dc8b-cd31-4b07-a94a-f9fc9f94cadf
relation.isAuthorOfPublication d03da8a1-fda9-4c43-9f75-2996190a8483
relation.isAuthorOfPublication 465ed94a-2e06-4fc4-80c1-0794cb0198f8
relation.isAuthorOfPublication 438dd9cb-0c98-475c-a877-a527fc778900
relation.isAuthorOfPublication f551a9ba-aea4-4979-89d8-7f88c1172b89
relation.isAuthorOfPublication 10671691-0d4d-4610-b023-15d78819093d
relation.isAuthorOfPublication b24d1438-ed26-4529-a212-6593dd8a1f87
relation.isAuthorOfPublication 880285be-5dd4-462c-966b-02130ce91c32
relation.isAuthorOfPublication 479f5271-81f4-4234-89c1-273374ab1f7c
relation.isAuthorOfPublication 46c532c2-c1db-4176-9321-3b7d53bf2e78
relation.isAuthorOfPublication fc91f062-f792-4fa4-815d-4219636d5f75
relation.isAuthorOfPublication b366aaa5-acc9-4bce-a58c-e72e5ea8e096
relation.isAuthorOfPublication.latestForDiscovery 4694dc8b-cd31-4b07-a94a-f9fc9f94cadf
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
fmars-06-00211.pdf
Size:
1021.38 KB
Format:
Adobe Portable Document Format
Description:
Article
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.89 KB
Format:
Item-specific license agreed upon to submission
Description: