GOES-R AWG Product Validation Tool Development Aerosol Optical Depth/Suspended Matter and Aerosol Particle Size Mi Zhou (IMSG) Pubu Ciren (DELL) Hongqing Liu (DELL) Istvan Laszlo (STAR) Shobha Kondragunta (STAR) OUTLINE Products Validation Strategies Routine Validation Tools Deep-Dive Validation Tools Ideas for the Further Enhancement and Utility of Validation Tools
Summary 2 Products Aerosol Optical Depth (AOD) / Suspended Matter (SM) Aerosol Particle Size (APS) Angstrom Exponent is reported at 2 km every 15 minutes for CONUS and FD 3 Monitoring & Validation Background Functions of tools: routine monitoring (may not need reference data)
routine validation (reference data, matchup procedure) deep-dive validation (reference data, other correlative data, matchup) Basic elements: data acquisition (ABI, ground, other sat products) spatial and temporal matching analysis (computing statistics) present results (display maps, scatter plots) Reference data: ground: Level 2.0 aerosol products from AERONET independent satellite data: standard aerosol products from MODIS
4 Routine Validation Strategy Spatial and temporal match-up with ground truth AERONET ground data are temporally averaged within a 1-hour window around the MODIS overpass time and the MODIS data are spatially averaged in a 50x50-km box centered on the ground station Quality control comparison data set AERONET uses level 2 quality assured product (no additional QC) Interpolating AERONET AOD to 550nm on logarithm scale Good ABI retrievals indicated by the overall quality flag Temporary measures until ABI clear mask is not available: Highest 50% and lowest 20% of ABI retrieved AODs in 50x50-km boxes are filtered out
Spatial variability test on ABI AOD for screening out possible cloud contamination Appropriate statistics (bias, std) are calculated and 5 Primary Reference Data AERONET aerosol product The ground-based AERONET remote sensing network provides a comprehensive dataset of aerosol properties and has been widely used for
evaluating satellite retrievals and model simulations in the aerosol community. All-Points AERONET Level 2.0 AOD data can be downloaded at the website http://aeronet.gsfc.nasa.gov/cgi -bin/combined_data_access_n ew . The ground measurements AERONET Stations 09/2007 6 More Reference Data
MODIS aerosol product AOD, ngstrm Exponent, fine-mode weight over ocean, surface reflectance over land, aerosol type over land from MODIS collection 5 aerosol products that can be downloaded from ftp://ladsweb.nascom.nasa.gov/allData/5/. Current state of the art, extensively validated product. Large variety of atmospheric and surface conditions. CALIPSO AOD and aerosol type from CALIPSO Lidar Level 2 5 km aerosol layer data at http://eosweb.larc.nasa.gov/cgi-bin/searchTool.cgi?Dataset=CAL_LID_L2_05km 09/2007 ALay-Prov-V3-01 Provide aerosol model profile
MAPSS A Multi-sensor Aerosol Products Sampling System in development at http://disc.sci.gsfc.nasa.gov/aerosols/services/mapss/mapssdoc.html#caliop Provides co-located MODIS-AERONET, CALIPSO-AERONET, and MISRAERONET data 7 Routine Tools Monitor operational Level-2 aerosol products Displays images of product and quality flags Plots histograms for specified granules Collocate aerosol products with the reference (truth) observations Compare with ground truth
Time series for collocated AERONET stations Frequency scatter-plot and linear regression Statistics for comparison with F&PS requirements IDL is used for visualization and calculation of statistics 8 Routine Tools (1) Example of Product Displays of GOES-east region for Aqua Day 2006213 from framework output (g/cm2) 9 Routine Tools (2) Granule Display AOD at 550nm
300000 Mean Std Max Min Number 250000 200000 Terra granule 2006213_1645 from framework output 0.371
0.368 4.982 -0.394 150000 100000 50000 0 -0.5 0.0 0.5 1.0 1.5
2.0 2.5 AOD at 550nm Angstrom Exponent at 470-860nm Number 400000 300000 Mean Std Max
Min 1.636 0.542 2.732 -0.194 200000 100000 0 0.0 0.5 1.0
1.5 2.0 Angstrom Exponent at 470-860nm 2.5 10 Routine Tools (3) Example of frequency scatter-plot and the calculated statistics in comparison with requirements (in red) using co-located AERONETMODIS data set for year 2006 Accuracy/Precision/# of matchups
Land <0.04 -0.06 / 0.05 /3633 [0.04,0.8] 0.04 / 0.12 /15800 0.06 / 0.13 0.04 / 0.25 >0.8 0 .17 / 0.34 /567 0.12/ 0.35 11 Routine Tools (4) GUI
Routine Tools (5) Time series of retrieval error East US West US 13 Deep-Dive Tool 14 Deep-dive Validation Strategy ABI retrievals are co-located with reference data in space and time Expanded comparison data set to include input / diagnostic /
intermediate parameters in addition to aerosol product, such as Surface reflectance and aerosol type over land from MODIS Quality flag of aerosol product from MODIS Aerosol model profile from CALIPSO Detailed analysis, for example, dependence of aerosol products on input, correlation with diagnostic / intermediate parameters Analysis of deep-dive validation results in re-processing To check proposed solution fixes the problem 15 Deep-Dive Tools Assumes access to all ancillary and intermediate data used to generate the product; and performs detailed analysis of
the retrieval error dependence on various parameters: Geometry: solar zenith angle, satellite zenith angle, scattering angle and relative azimuth; Ancillary input: water vapor, ozone, ocean surface wind speed / direction Intermediate data: surface reflectance, aerosol type Quality flags such as cloud masks, land surface type, snow/ice mask Season AOD Display input, intermediate and/or diagnostic data (not shown) IDL is used for visualization and calculation of statistics 16 What Triggers DeepDive? When consistent over- or underestimation over a prescribed
threshold is observed at a station deep-dive assessment is triggered Analysis of the retrieval error dependence on various parameters (geometry, ancillary input, intermediate data, quality flags, season, and AOD) is conducted to identify problems When problem is identified, say due to incorrect input of satellite zenith angle, the offline-algorithm is re-run and the time series is re-plotted. 17 Deep-Dive Tools Dependence on Angle, etc. Example use of Deep-Dive Tools
ABI retrieval MODIS retrieval ABI image vs. MODIS image has much less valid retrievals; Plotting of inputs suggests it is due to bad snow mask Correct snow mask and reprocess this granule ABI snow mask Correct ABI snow mask Reprocessed ABI retrieval Deep-dive Tools - GUI
Ideas for the Further Enhancement and Utility of Validation Tools Calculate and display additional statistics temporal averages on different scales (daily, weekly, monthly) Identify signatures by which even non-experts can identify potential problems needed for routine operational monitoring Implement automatic detection of possible systematic drift or continuous abnormal retrieval in routine validation. establish reference (expected) statistics from good data compare time series of actual statistics with reference stats trigger action (e.g., sending warning email) when actual stats exceed reference stats + x std. Uniform (common) web interface for all ABI products.
21 Summary Current tools perform three functions: routine monitoring of product routine validation with reference data deep-dive validation with reference and intermediate data Validation truth data have been identified and processed Planned enhancements include: more stats automatic detection of problems 22
