Dynamic reusable workflows for ocean science
MetadataShow full item record
KeywordNumerical modeling; Reproducibility; Catalog services; Data services; Web services; Metadata; Ocean forecasting; Ocean modeling; Data management; Data system; Interoperability; OPeNDAP; THREDDS; CSW; Jupyter Notebooks
Digital catalogs of ocean data have been available for decades, but advances in standardized services and software for catalog searches and data access now make it possible to create catalog-driven workflows that automate—end-to-end—data search, analysis, and visualization of data from multiple distributed sources. Further, these workflows may be shared, reused, and adapted with ease. Here we describe a workflow developed within the US Integrated Ocean Observing System (IOOS) which automates the skill assessment of water temperature forecasts from multiple ocean forecast models, allowing improved forecast products to be delivered for an open water swim event. A series of Jupyter Notebooks are used to capture and document the end-to-end workflow using a collection of Python tools that facilitate working with standardized catalog and data services. The workflow first searches a catalog of metadata using the Open Geospatial Consortium (OGC) Catalog Service for the Web (CSW), then accesses data service endpoints found in the metadata records using the OGC Sensor Observation Service (SOS) for in situ sensor data and OPeNDAP services for remotely-sensed and model data. Skill metrics are computed and time series comparisons of forecast model and observed data are displayed interactively, leveraging the capabilities of modern web browsers. The resulting workflow not only solves a challenging specific problem, but highlights the benefits of dynamic, reusable workflows in general. These workflows adapt as new data enter the data system, facilitate reproducible science, provide templates from which new scientific workflows can be developed, and encourage data providers to use standardized services. As applied to the ocean swim event, the workflow exposed problems with two of the ocean forecast products which led to improved regional forecasts once errors were corrected. While the example is specific, the approach is general, and we hope to see increased use of dynamic notebooks across geoscience domains.
© The Author(s), 2016. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Journal of Marine Science and Engineering 4 (2016): 68, doi:10.3390/jmse4040068.
Suggested CitationArticle: Signell, Richard P., Fernandes, Filipe, Wilcox, Kyle, "Dynamic reusable workflows for ocean science", Journal of Marine Science and Engineering 4 (2016): 68, DOI:10.3390/jmse4040068, https://hdl.handle.net/1912/8763
The following license files are associated with this item:
Except where otherwise noted, this item's license is described as Attribution-NonCommercial-ShareAlike 4.0 International
Showing items related by title, author, creator and subject.
Cooley, Sarah R.; Kite-Powell, Hauke L.; Doney, Scott C. (Oceanography Society, 2009-12)Ocean acidification lowers the oceanic saturation states of carbonate minerals and decreases the calcification rates of some marine organisms that provide a range of ecosystem services such as wild fishery and aquaculture ...
Cohesive and mixed sediment in the Regional Ocean Modeling System (ROMS v3.6) implemented in the Coupled Ocean–Atmosphere–Wave–Sediment Transport Modeling System (COAWST r1234) Sherwood, Christopher R.; Aretxabaleta, Alfredo L.; Harris, Courtney K.; Rinehimer, J. Paul; Verney, Romaric; Ferré, Bénédicte (Copernicus Publications on behalf of the European Geosciences Union, 2018-05-14)We describe and demonstrate algorithms for treating cohesive and mixed sediment that have been added to the Regional Ocean Modeling System (ROMS version 3.6), as implemented in the Coupled Ocean–Atmosphere–Wave–Sediment ...