VisualEsse toolkit for visualization of environmental events and remote sensing data

 

 

S. Berezin¹, D. Voytsekhovsky¹, M. Zhizhin², V. Liutsarev³, E. Kihn⁴
1) Department of Computational Mathematics and Cybernetics, Moscow State University, Russia
2) Geophysical Center and Space Research Institute, Russian Academy of Sciences, Russia
3) Microsoft Research, Cambridge, UK
4) National Geophysical Data Center, NOAA, USA
esse@wdcb.ru



Contents

• 1. Introduction. 1
• 2. Supported data sources. 2
• 2.1. Metadata
• . 22.2. Gridded data. 3
• 2.3. Geolocated images. 3
• 3. Architecture. 4
• 4. Visualization clients
• . 44.1.NASA World Wind plugin. 5
• 4.1.1. Supported visualization types. 5
• 4.1.2. Data exploration. 6
• 4.1.3. Asynchronous data loading and progressive visualization. 7
• 4.1.4. Geolocated images. 7
• 4.1.5. DMSP satellite images. 8
• 4.2. OpenGIS server and thin client 8
• 4.2.1. Supported visualization types. 9
• 4.2.2. AJAX and Virtual Earth toolboxes. 9
• 4.2.3. Web Map Server request 10
• 4.2.4. WMS server implementation. 11
• 5. Conclusions. 11
• References. 12

 

Abstract:

In this paper we present a visualization system VisualEsse, that is built as a part of the Environmental Scenario Search Engine (ESSE). We need to interactively render data from various sources over the surface of the Earth of two similar classes: gridded data and geolocated images. Gridded data are a scalar or vector two-dimensional arrays of points usually (but not always) aligned with meridians and parallels with a constant step for latitude and longitude; each dimension of the gridded data array has hundreds of points. Geolocated image is a high resolution image (thousands by thousands pixels) with mapping to latitude and longitude defined for each pixel. Currently we use only one type of geolocated images: DMSP satellites visual and infrared images.

Visualization system provides two applications: MS Windows application based on NASA World Wind and web applications built upon Microsoft Virtual Earth Control. Both applications allow researcher to show data arrays and images exported by web-services and to pre-process the data before visualization.

The ESSE visualization engine is easily extensible not only for the new data sources, but also for the new types of visualization. At the moment our module supports color maps, isolines and vector fields for gridded data rendering and texturing of the Earth surface with geolocated images possibly with transparency effects.

 

1. Introduction

Environmental informatics combines the research fields of Artificial Intelligence, Geographical Information Systems (GIS) Modeling and Simulation, and User Interfaces and is a rapidly expanding area of computer and natural science [Hilty, 1995]. The increasing data volumes from today’s collection systems and the need of the scientific community to include integrated and authoritative representation of the natural environment in analysis requires a new approach to data mining, management and access [Reichardt, 1999].

 

At the same time, the environmental modelling community has begun to develop several archives of continuous environmental representations. These archives take observational data and through modelling create a regular, parameterized view of the Earth system. The models use all available observation data as initial conditions for the numerical models, so the resulting petabyte-size data sets jointly may be considered as authoritative high-resolution representation of terrestrial weather and the near-Earth space during the last 50 years [Kalney, 1996].

 

For the parallel mining of a set of conditions inside distributed, very large databases from multiple environmental domains we have developed a software toolbox called Environmental Scenario Search Engine (ESSE). The prime requirement of the ESSE system design is to allow the user to query the environmental data archives in human linguistic terms. The mapping between human language and computer systems involves fuzzy logic. Imagine for example that the end user doesn’t need all of the weather data covering the Moscow region for the last 60 years, but rather needs an example of an atmospheric front near Moscow. Further imagine that this user needs satellite images of the front and he wants to know how often such fronts occur or if they have been increasing in the last 10 years. The ESSE search engine and data mining portal will address such problems [MSTR, 2006].

 

In this paper we present a visualization system VisualEsse, that is built as a part of the ESSE project. We need to interactively render data from various sources over the surface of the Earth. Data falls in one of two similar classes: gridded data and geolocated images. Gridded data are a scalar or vector two-dimensional arrays of points usually (but not always) aligned with meridians and parallels with a constant step for latitude and longitude; each dimension of the gridded data array has hundreds of points. Geolocated image is a high resolution image (thousands by thousands pixels) with mapping to latitude and longitude defined for each pixel. Currently we use only one type of geolocated images – DMSP satellite visual and infrared images [SABR].

 

Following our approach, data for visualization is specified using simple language of Data Access Expressions (DAE). DAEs are evaluated on client side and support optimal strategies data downloading from remote sources. DAEs are applicable to array data and images. DAE consists of mandatory reference to data source and optional list of operations applied to data. Thus, it enables client-side processing of data before its visualization or analyzing.

 

Visualization system provides two applications – windows application based on NASA World Wind [WorldWind] and web applications that is built upon Microsoft Virtual Earth Control [MVEC]. Both applications allow researcher to show data arrays and images from various sources and to specify custom DAEs to pre-process data before visualization. DAE evaluation engine is designed as shared and reusable component.

 

The ESSE visualization engine is easily extensible not only for the new data sources, but also for the new types of visualization. At the moment our module supports the following visualization types: color maps, isolines and vector fields for gridded data rendering and texturing of the Earth surface with geolocated images possibly with transparency effects.

 

Interesting feature of visualization applications is support of data visualization along trajectories. Along with possibility to load trajectory into application and plot values of specified DAE along it, both application allows researcher to download DAE values into file or into spreadsheet. This data exploration ability can be very useful in many scenarios.

 

2. Supported data sources

2.1. Metadata

Metadata plays important role in visualization. We have two sources of metadata – it can be requested from an ESSE data source web service or it can be loaded from a local file if the service is unavailable or when the gridded data is loaded from a local file. Metadata is stored in XML document. A part of the document describing one parameter is shown below.

 

<Parameter display="Relative Humidity" localKey="rhum" name="RelHumid" units="%">    <scale>1</scale>    <offset>0</offset>    <Visualization>        <Palette opacity="140">RedGreenBlue</Palette>        <Range min="0" max="100" />    </Visualization> </Parameter>

 

Metadata that is related to visualization is located inside Visualization elements, including a preferable visualization type, a range of the parameter values and a palette for mapping the parameter values to colors.

 

All parameters returned by ESSE services are scalar values. We introduce  special virtual parameters with corresponding virtual metadata elements in order to form a vector field. Currently we have only one vector field – wind velocity, but in the future the  number of such parameter will grow. XML fragment describing a virtual parameter is shown below.

 

<compositeParameter display="Wind velocity" localKey="wndvel" name="WndVel"                       units="m/s">     <components>         <parameter name="WndUComp"></parameter>         <parameter name="WndVComp"></parameter>     </components>     <Visualization>         <Palette opacity="140">RedGreenBlue</Palette>         <Range min="0" max="20" />     </Visualization> </compositeParameter>

 

2.2. Gridded data

As a source of the gridded data we use the ESSE Data Resource Grid Service which consolidates data from meteorological NCEP/NCAR [Kalney, 1996][MSTR] and space weather SPIDR archives [SPIDR]. The Data Resource Service is an OGSA-DAI [OGSA-DAI] component that provides a web service returning gridded data as XML documents. Using the OGSA-DAI specifications, a client can communicate with the data server using the standard web or grid service protocols.

 

NCEP/NCAR reanalysis data service provides more than 20 meteorological parameters on several vertical levels such as air temperature, humidity, wind velocity or cloud cover. Reanalysis data are defined on a regular grid over the Earth surface with 2.5 degrees step in latitude and longitude and 6 hour in time. NOAA global weather forecast is defined on a regular grid with spatial step of 1 degree and temporal step of 3 hours. Currently available data sources return data on regular grids only but our visualization engine supports irregular grids (grids with variable spatial and temporal steps) as well. Query results can be saved as XML documents and used later for the visualization, when the Internet connection is unavailable. Type and resolution of the grid are determined automatically by the visualization engine when loading XML document.

 

So far we have two data sources for the gridded data:

• local XML document compliant with the OGSA-DAI data source output schema possibly compressed using GZip;
• web service providing data using the OGSA-DAI protocol and output schema.

 

Weather reanalysis and forecast data can also be available through THREDDS servers [THREDDS], such as NOAA NOMADS [NOMADS]. This option can be easily added to the system by creating a new data provider.

 

2.3. Geolocated images

The second data type supported by our visualization engine is geolocated images, which are high resolution images with additional information on how the pixels are mapped to the latitude/longitude coordinate system. Geolocated images are different from the gridded data because they have much higher resolution and they cannot be mapped to rectangular latitude/longitude gird. Currently the visualization engine supports only DMSP satellite images provided by NOAA in visible and near-infrared spectral bands with bindings to ephemeris of a satellite orbit. DMSP images are accessed thought THREDDS service via OPeNDAP protocol [THREDDS][OPeNDAP]. A typical size of a DMSP satellite image is about 14000 by 1400 pixels.

 

We support two data sources for the DMSP images:

 

• Local file with one DMSP orbit. File format is identical to the OPeNDAP server output data stream with a simple ASCII header describing syntax of the data followed by XDR-encoded data. It can be optionally compressed using GZip. This option is used to display locally cashed data, when the Internet connection is not available.
• THREDDS service providing data thorough OPeNDAP protocol.

 

3. Architecture

In terms of the client-server architecture, the visualization engine can be deployed either on a server or on a client side (thin or thick clients respectively), providing different capabilities to display the data and different hardware requirements for the client workstation. The system architecture is split into three modules (Fig. 1). Two of them, namely VisualEsse.DataEngine and VisualEsse.Visualization, are shared by all visualization clients. The VisualEsse.Renderer module is client-dependent.

 

 

Fig. 1. Visualization engine architecture

 

The VisualEsse.DataEngine module provides data upon request in the same way irrespective of the data source type. Thus a new data source (netCDF, WCS, etc.) [netCDF][OpenGIS] can be added simply by extending this module with a new data provider. To increase the efficiency of the data request processing, the VisualEsse.DataEngine is charged to cache the provided data.

 

The VisualEsse.Visualization module is responsible for applying of visualization algorithms to the data, at this stage being no matter how the data was retrieved. All algorithms implement the same interface describing a single method to convert certain abstract data model into a set of abstract geometric objects (GeometricObject). An implementation of the GeometricObject depends on the computer graphics API (DirectX, OpenGL or GDI+). Therefore the VisualEsse.Visualization module can be easily extended with new visualization types and algorithms as well as with support for a new computer graphics API.

 

4. Visualization clients

Below we compare both approaches. For the thick client container we use NASA open source application World Wind [WorldWind]. For the thin client we use Web Map Service approach [OpenGIS] with our own web map server implementation combined with the MS Virtual Earth toolkit [MSVEC].

4.1. NASA World Wind plugin

We have developed the VisualEsse.Plugin as an extension of the NASA World Wind application [WorldWind] which implements the three-dimensional Earth model based upon BlueMarble, Landsat 7 and Shuttle Radar Topography data. The combination of VisualEsse.Plugin with World Wind is a “thick” client performing the entire functionality related to the visualization at the user workstation. Provided with the Internet connection, it leaves to the server side only the data source services.

Advantages: great visualization abilities comparable with Google Earth, high interactivity, support for DMSP images.

Disadvantages: high hardware requirements, requires DirectX compliant graphics hardware, requires installation step before use.

4.1.1. Supported visualization types

 

The VisualEsse.Visualization module is used as a part of the World Wind plugin to visualize the data, therefore all its functionality is inherited by the plugin. Thus it is possible to build animated transparent color maps and isolines upon the 3D Earth surface. Additionaly, a user can select different palettes for such visualization. Besides, VisualEsse.Plugin allows mapping of DMSP orbital data as images to the Earth surface (Fig. 2).

 

 

World Wind plugin with colormaps	World Wind plugin isoline visualization
World Wind plugin vector field	World Wind with DMSP image

Fig. 2. Different visualization types

 

4.1.2. Data exploration

 

In addition, in the World Wind plugin we can visualize gridded data along tracks. Track is an ordered collection of points with latitude, longitude and time specified at each position, e.g. GPS log. We can display the track as a polyline. At each track point we can display values of selected parameters interpolated from the grid. Track visualization is combined with the MS Excel interop, so user can drag a table with the track data from an Excel spreadsheet for rendering or one can drag a tarck with the displayed parameters from the visualization client back to the MS Excel spreadsheet.

 

This interoperability turns VisualEsse.Tracks plugin from data visualization to data retrieval and exploration tool  that can be very useful. We plan to add similar data retrieval abilities to VisualEsse.Grids plugin (Fig. 3),

Fig. 3. Data exploration and MS Excel interoperability in the World Wind plugin

 

4.1.3. Asynchronous data loading and progressive visualization

 

To be more efficient and interactive, the World Wind plugin renders data only over a part of the Earth surface which is visible for the user. If the visible part of the Earth is changed, the new data request is dispatched in background allowing the user to continue to work with the World Wind application while the new data are loading. To increase the data processing efficiency, a data request caching is used for networked data sources.

 

In our plugin we have implemented progressive visualization, where the data are requested on a sparse grid first which to show a preview picture to the user as quickly as possible. At the same time a background request for full-resolution data is performed. Progressive visualization currently is used only for visualization of geolocated images; it is not used for gridded data sources because even for the full Earth coverage the grid size is relatively small for currently available data sources.

 

4.1.4. Geolocated images

 

The ESSE World Wind plugin provides visualization of geolocated images. The geolocated image is hierarchically subdivided with the depth of subdivision depending on the resolution of source image. When rendering geolocated image, its visible parts and resolution are chosen accordign to the World Wind viewport parameters.

Abstract Orbital Data Provider retrieves data for specified part of the geolocated image. Two main methods of the Orbital Data Provider get rectangular array of pixels for specified part of the image and get array of latitude/longitude coordinates for the pixels. When working with remote data sources, the Orbital Data Provider ancestors can implement efficient caching and filtering. By introducing a new Orbital Data Providers we can add new sources of geolocated images to the system. Up to now we have implemented Orbital Data Provider for DMSP satellite images, both local and available throught OpenDAP interface.

We use the following algorithm to retrieve and render a geolocaged image.  Entire image is divided into square parts (ABCD polygon in Fig. 4). For each vertex at the top level the latitude and longitude are calculated and points are projected on screen. If the texture with specified resolution (we use 256 x 256) is too small and  it requires magnification of the vertex, then we subdivide the top level region into four subregions. For each vertex of each subregion we compute latitude and longitude and repeat the process until the texture of specified size covers the whole viewport.

Fig. 4. Subdivision of geomapped image

 

4.1.5. DMSP satellite images

 

We have several data providers for retrieving and loading DMSP satellite images in VisualEsse.DataEngine component: DMSPDataProvider requests data from OPeNDAP servers, DMSPLocalDataProvider works with local data files (from local cache), and DMSPPreviewDataProvider requests satellite image preview (coverage) from OPeNDAP servers.

Each DMSP orbit is divided into 8 nearly square geolocated images images. These images have resolution of 1400 by 14000 pixels. We use 4 levels of subdivision for progressive visualization. Although the file size for each DMSP orbit is about 20 Mb, the progressive preview approach combined with the data request caching allowes us to get interactive response from the World Wind application when working with remote OPeNDAP data servers through a cable Internet connection.

4.2. OpenGIS server and thin client

We have implemented the “thin” client approach as a web application based on Microsoft Virtual Earth service. In this case the data retrieval and visualization tasks are performed by a special web map server, compliant with the OpenGIS specifications [OpenGIS]. The resulting images are displayed in a regular web browser. Interaction with the web application requires JavaScript interpreter on the browser side.

 

Advantages: cross platform compatibility, requires no installation.

Disadvantages: limited interactivity, 2D graphics only.

 

4.2.1. Supported visualization types

 

The thin client can visualize environmental data available from ESSE sources over road maps, satellite images or mixed images of the Earth available from the Microsoft Virtual Earth. We took the Microsoft Virtual Earth control as a base for building the thin client. The Virtual Earth control renders the Earth images in background and they are combined with the data visualization images from our WMS server. Currently we support the data visualization with color maps and isolines and time navigation controls. A test version of the ESSE web client is available at http://depot.cs.msu.su/VisualEsse

 

4.2.2. AJAX and Virtual Earth toolboxes

 

Microsoft Virtual Earth SDK [MS VE SDK] can display vector graphics containing segments and polygons over combined road maps and the Earth surface images. It provides a special event mechanism activated when a visible part of the map is changing. It works well with Microsoft Internet Explorer and Firefox browsers, but it has some problems with vector graphics in Opera browser.

 

In order to move all complex code away from the web browser’s JavaScript interpreter, we have implemented the following scheme. When the Virtual Earth or time control is used or a visible part of the map is changing, a special JavaScript handler is invoked. It retrieves a new visible part of the map using the Microsoft Virtual Earth API and requests new data visualization from our special WMS server. The WMS server in response generates an image that renders the requested environmental parameter using a user specified or default visualization type. The WMS server can return either a vector image in SVG format or a raster image with transparency in GIF or PNG format. The resulting WMS image is added to the Virtual Earth control as additional layer (Fig. 5). We use Prototype [JS] library to perform AJAX requests for asynchronous processing of isolines in SVG format.

 

Fig. 5. Web client sample screenshot

 

4.2.3. Web Map Server request

 

Our implementation of the OpenGIS WMS server (VisualEsse.WMS) [OpenGIS] provides images of specified region of the Earth with visualization of the requested environmental parameter using a user specified or default visualization type. The web client requests a set of layers and styles (one style for each layer). A name of the layer matches the environmental parameter to be plotted (for example air0). A style defines visualization type (for example Isolines, Fig. 5). The WMS server request also contains date and time, bounding coordinates of the viewport region in CRS:84 coordinate system and, optionally, a vertical height level for the requested parameter. When multiple layers are requested, they are combined in one image (first specified layer will be bottom layer in image). VisualEsse.WMS supports raster image formats GIF, PNG, JPEG and vector XML-based SVG.

 

WMS server request example:

 

http://depot.cs.msu.su/EsseWMS/perform.aspx?REQUEST=GetMap&SERVICE=WMS&

VERSION=1.3.0&LAYERS=air0&STYLES=Isolines&CRS=CRS:84&TRANSPARENT=false&

BBOX=0,0,60,60&WIDTH=640&HEIGHT=640&FORMAT=image/gif&TIME=2000-01-01

 

An image returned by the WMS server on the above request is shown in Fig. 6.

 

 

Fig. 6. ESSE WMS server reply on the isolines visualization request

 

4.2.4. WMS server implementation

 

In many cases size of a raster image with isolines appears to be smaller than SVG vector image with the same isolines (40KB versus 70KB for sample image). Unfortunately the Microsoft Virtual Earth SDK version 4.0 has known problems when displaying raster images with transparency, so we currently have to use SVG graphics for the WMS server output. The SVG image returned by the server is an XML document which is parsed by a custom JavaScript function. All the polylines and polygons are appended to the Virtual Earth control using the Virtual Earth API calls. Special approach was taken to display the text labels. SVG format provides rich support for rendering text, but VE control does not. So we choose to convert text labels into polylines using a simple vector font inside the WMS server. 

 

Our WMS server implementation is based upon the Data Engine with data requests cache and Visualization modules. The data are transformed within the Data Engine to the common model GeometricObject using specified visualization algorithm. Then the GeometricObject is rendered to a raster image of to vector SVG image inside the Visualization module using GDI+ API.

 

 

5. Conclusions

 

Major data types related to Earth sciences include vector and scalar arrays on grids aligned to parallels and meridians and high resolution images received from satellite. Efficient visualization and data exploration often require client-side processing of data when researcher can easily try different types of visualization for data from multiple sources. We present a visualization system that is built as a part of the Environment Scenario Search Engine (ESSE) project.

 

Parallel query and visualization of several web data services is implicit feature of the NASA World Wind virtual Globe. VisualEsse plugin brings a new set of data sources, namely historical weather archive and weather forecast together with the archive of polar orbiting meteorological satellite images – DMSP. Combination of this data sources with powerful 3D visualization platforms provides a new way of analysis of meteorological conditions. For example in Fig. 7 we present an interactive visualization of the 2006 hurricane Ernesto with DMSP satellite visible image and wind speed and direction selected from ESSE web services.

 

 

 

Fig. 7. interactive visualization of the 2006 hurricane Ernesto with DMSP satellite visible image and wind speed and direction selected from ESSE web services.

 

 

As a result of this study environmental scientists have a new very flexible and configurable visualization toolkit, which can be used in many research areas, possibly unknown at the development stage. For example, the World Wind visualization service can be used not only for ESSE data services, but for any OPeNDAP-compatible networked data service.

 

VisualEsse toolkit is an Open Source project under BSD license hosted by CodePlex [VisualEsse].

 

 

References

 

[Hilty, 1995] Hilty, L. M., Page, B., Radermacher, F.J., and Riekert, W.-F., Ed. Environmental Informatics as a New Discipline of Applied Computer Science. Environmental Informatics, Kluwer Academic Publishers, 1995.

 

[Reichardt, 1999] Reichhardt, T. It's sink or swim as a tidal wave of data approaches. Nature, 1999, 517-520

 

[Kalney 1996] Kalnay, E., and Coauthors. The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77, 1996, pp. 437–471.

 

[MSTR, 2006] M. Zhizhin, A. Poyda, D. Mishin, D. Medvedev, E. Kihn, V. Lutsarev.

Scenario Search on the Grid of Environmental Data Sources.

MSR Technical Report, July 2006. Online at http://research.microsoft.com/research/pubs/view.aspx?type=Technical%20Report&id=1116

 

[SABR] Satellite Archive Browse and Retrieval web interface to the DMSP OLS data archive at the NGDC NOAA. Online at http://sabr.ngdc.noaa.gov/

 

[WorldWind] NASA WorldWind viewer web site http://worldwind.arc.nasa.gov/

 

[MSVC] Microsoft Virtual Earth Control SDK web site http://dev.live.com/virtualearth/sdk/

 

[SPIDR] Space Physics Interactive Data Resource web site http://spidr.ngdc.noaa.gov

 

[OGSA-DAI]   M. Antonioletti, M.P. Atkinson, R. Baxter, A. Borley, N.P. Chue Hong, B. Collins, N. Hardman, A. Hume, A. Knox, M. Jackson, A. Krause, S. Laws, J. Magowan, N.W. Paton, D. Pearson, T. Sugden, P. Watson, and M. Westhead. The Design and Implementation of Grid Database Services in OGSA-DAI. Concurrency and Computation: Practice and Experience, Volume 17, Issue 2-4, Pages 357-376, February 2005;

K. Karasavvas, M. Antonioletti, M.P. Atkinson, N.P. Chue Hong, T. Sugden, A.C. Hume, M. Jackson, A. Krause, C. Palansuriya. Introduction to OGSA-DAI Services. Lecture Notes in Computer Science, Volume 3458, Pages 1-12, May 2005;

Web site: http://www.ogsadai.org.uk/

 

[THREDDS] THREDDS catalog of the OpenDAP data sources. Online at

http://www.unidata.ucar.edu/projects/THREDDS/

 

[NOMADS] NOAA National Operational Model Archive & Distribution System of the NOAA National Climatic Data Center. Online at http://nomads.ncdc.noaa.gov/

 

[OpenGIS] OpenGIS standards and specifications. Online at http://www.opengeospatial.org/standards

 

[OPeNDAP] OPeNDAP data access protocol. Online at http://opendap.org/

 

[netCDF] UNIDATA Network Common Data Format. Online at http://www.unidata.ucar.edu/software/netcdf/

 

[JS] Prototype Java Script framework web site http://www.prototypejs.org/

 

[VisualEsse] VisualEsse Open Source project home page http://www.codeplex.com/VisualEsse