Analysis and visualization of coastal ocean model data in the cloud.
MetadataShow full item record
The traditional flow of coastal ocean model data is from High-Performance Computing (HPC) centers to the local desktop, or to a file server where just the needed data can be extracted via services such as OPeNDAP. Analysis and visualization are then conducted using local hardware and software. This requires moving large amounts of data across the internet as well as acquiring and maintaining local hardware, software, and support personnel. Further, as data sets increase in size, the traditional workflow may not be scalable. Alternatively, recent advances make it possible to move data from HPC to the Cloud and perform interactive, scalable, data-proximate analysis and visualization, with simply a web browser user interface. We use the framework advanced by the NSF-funded Pangeo project, a free, open-source Python system which provides multi-user login via JupyterHub and parallel analysis via Dask, both running in Docker containers orchestrated by Kubernetes. Data are stored in the Zarr format, a Cloud-friendly n-dimensional array format that allows performant extraction of data by anyone without relying on data services like OPeNDAP. Interactive visual exploration of data on complex, large model grids is made possible by new tools in the Python PyViz ecosystem, which can render maps at screen resolution, dynamically updating on pan and zoom operations. Two examples are given: (1) Calculating the maximum water level at each grid cell from a 53-GB, 720-time-step, 9-million-node triangular mesh ADCIRC simulation of Hurricane Ike; (2) Creating a dashboard for visualizing data from a curvilinear orthogonal COAWST/ROMS forecast model.
© The Author(s), 2019. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Signell, R. P., & Pothina, D. Analysis and visualization of coastal ocean model data in the cloud. Journal of Marine Science and Engineering, 7(4), (2019);110, doi:10.3390/jmse7040110.
Suggested CitationSignell, R. P., & Pothina, D. (2019). Analysis and visualization of coastal ocean model data in the cloud. Journal of Marine Science and Engineering, 7(4), 110
The following license files are associated with this item:
Showing items related by title, author, creator and subject.
From the oceans to the cloud: Opportunities and challenges for data, models, computation and workflows. Vance, Tiffany C.; Wengren, Micah; Burger, Eugene; Hernandez, Debra; Kearns, Timothy; Medina-Lopez, Encarni; Merati, Nazila; O'Brien, Kevin; O’Neil, Jon; Potemra, James T.; Signell, Richard P.; Wilcox, Kyle (Frontiers Media, 2019-05-21)Advances in ocean observations and models mean increasing flows of data. Integrating observations between disciplines over spatial scales from regional to global presents challenges. Running ocean models and managing the ...
Advancing coastal ocean modelling, analysis, and prediction for the US Integrated Ocean Observing System Wilkin, John L.; Rosenfeld, Leslie K.; Allen, Arthur; Baltes, Rebecca; Baptista, Antonio; He, Ruoying; Hogan, Patrick; Kurapov, Alexander; Mehra, Avichal; Quintrell, Josie; Schwab, David; Signell, Richard P.; Smith, Jane (2017-04-19)This paper outlines strategies that would advance coastal ocean modeling, analysis and prediction as a complement to the observing and data management activities of the coastal components of the U.S. Integrated Ocean ...
Analysis of apparent optical properties and ocean color models using measurements of seawater constituents in New England continental shelf surface waters Green, Rebecca E.; Sosik, Heidi M. (American Geophysical Union, 2004-03-17)We used budgets of absorption (a), scattering (b), and backscattering (bb) for particles and chromophoric dissolved organic matter (CDOM) to investigate sources of seasonal variations in apparent optical properties (AOPs) ...