Federal Work Group on Uncertainty Analysis and Parameter Estimation Tom Nicholson, U.S. Nuclear Regulatory Commission Mary Hill, U.S. Geological Survey 2012 Geological Society of America Annual Meeting November 7, 2012 Charlotte, NC 1 Outline Work Group 2 (WG2) Objective and Goals Members and Participants Activities and Technical Projects Seminars at the WG2 Meetings Methodologies, Tools and Applications Forward Strategy Recommendations for FY2013 2
Work Group Objective Coordinate ongoing and new research conducted by U.S. Federal agencies on: parameter estimation uncertainty assessment in support of environmental modeling and applications Focus on strategies and techniques Includes sensitivity analysis What is needed to achieve this objective? Coordination of research staff and their management thru efficient and targeted use of our limited resources. 3 Work Group Goals Basics: Develop a creative, collaborative environment to advance parameter estimation in the context of model
development . sources of uncertainty in the context of model predictions. Develop a common terminology. Identify innovative applications. Existing Tools: Identify, evaluate, and compare available analysis strategies, tools and software. New Tools: Develop, test, and apply new theories and methodologies. Batch scale (0.01m) Column scale (0.1 m) Intermediate scale (2m) Tracer test scale (1-3m)
Electrical Conductivity Geophysics Butler et a (2-200m) Exchange: Facilitate exchange of techniques and Plume scale ideas thru teleconferences, technical workshops, 4 (2000m) professional meetings, interaction with other WGs Members and Participants from U.S. Federal agencies, universities, and industry
Tom Nicholson, NRC, co-Chair Mary Hill, USGS, co-Chair Todd Anderson, DOE Tommy Bohrmann, EPA Gary Curtis, USGS Bruce Hamilton, NSF
Yakov Pachepsky, USDA-ARS Tom Purucker, EPA-Athens Yoram Rubin, UC Berkeley Brian Skahill, USACOE Matt Tonkin, SSPA Gene Whelan, EPA-Athens Steve Yabusaki, PNNL Ming Ye, Florida State U Ming Zhu, DOE Larry Deschaine, HydroGeologic, Inc. Boris Faybishenko, LBNL Pierre Glynn, USGS
Philip Meyer, PNNL Candida West, EPA Debra Reinhart, NSF You? 5 Activities: Conference Sessions 2011 Fall AGU: Mary Hill, WG2 Co-Chair organized session Uncertainty Assessment, Optimization, and Sensitivity Analysis in Integrated Hydrologic Modeling as Application of Hydroinformatics. 2011 NSF Statistical and Applied Mathematical Sciences Institute (SAMSI): WG2 Co-Chair M. Hill co-organized Workshop on Uncertainty in the Geosciences, Research Triangle Park, NC 2012 Society of Toxicology/EPA Contemporary Concepts in Toxicology Workshop WG2 Co-Chair T. Nicholson presented invited paper, volunteered poster, and participated in technical sessions. Focused on exposure, dose-response, ecosystem impacts, life cycle/cost-benefit, and information technology. 6
Activities: Conference Sessions 2012 Fall AGU: Co-chair Mary Hill organized session Complexity, Falsifiability, Transparency, and Uncertainty in Environmental Modeling 2012 Geological Society of America Annual Meeting: CoChair Tom Nicholson presented invited paper co-authored with Mary Hill, WG2 Co-Chair on Federal Work Group on Uncertainty Analysis and Parameter Estimation in technical session T103. Ground-Water Model Calibration and Uncertainty Analysis organized by Ming Ye, Florida State University and WG2 member. 7 Activities: Teleconferences We conduct teleconferences to: review and discuss ongoing research studies and software development formulate proposals for field applications Tracer application area Observation well
Unsaturated soil Groundwater What measurements would help discriminate between two models? from 2/22/2012 seminar by Yakov Pachepsky, USDA, on model abstraction 8
Activities: Teleconferences We conduct teleconferences to: review and discuss ongoing research studies and software development formulate proposals for field applications How does moisture travel in the atmosphere and lead to big storms? from 9/21/2012 seminar by Mike Dettinger, USGS, on atmospheric rivers 9 Seminars at WG2 Teleconferences in FY2012 Multi-Scale Assessment of Prediction Uncertainty in Coupled Reactive Transport Models by Gary P. Curtis, USGS; Ming Ye, Florida State University; Philip D. Meyer and Steve B. Yabusaki, PNNL to discuss the use of a Bayesian model averaging method to assess parametric and model uncertainty for improvement of predictive performance.
Briefing on the Chernobyl Cooling Pond Decommissioning and Remediation Proposal (ISCMEM Case Study for Improving Scientific Basis for Multimedia Environmental Modeling and Risk Assessment) by Boris Faybishenko, Lawrence Berkeley National Laboratory, to provide comments and questions for the international meeting in Kiev, Ukraine on remediation decision-making. 10 Chernobyl Cooling Pond Decommissioning and Remediation Proposal (ISCMEM Case Study for Improving Scientific Basis for Multimedia Environmental Modeling and Risk Assessment) by Boris Faybishenko, Lawrence Berkeley National Laboratory on October 2011. 11 Seminars at WG2 Teleconferences (continued) Training Range Environmental Evaluation and Characterization System (TREECS) by B. Johnson, M.
Dortch and B. Faybishenko to discuss an advanced spatially integrated, multi-scale, multi-pathway simulation capability for evaluation of distributed sources of contaminants from both on-site as well as off-site sources with applications to the Borschi Watershed, and military training range. The How of Environmental Modeling: Toward Enhanced Transparency and Refutability by Mary Hill, USGS, to discuss advantages of establishing a base set of model sensitivity analysis and uncertainty evaluation measures, to be used along with any other performance measures of interest. 12 Model Adequacy How to include many data types with variable quality? Error-based weighting and SOO, MOO* Is model misfit/ overfit a problem? Are prior knowledge and data subsets inconsistent? Variance of weight-standardized residuals, residual graphs and space/time plots, MOO* How nonlinear is the problem? Modified Beales measure , Explore objective function *, TSDE* Sensitivity and Uncertainty
Observations Parameters What parameters can be estimated with the observations? b/SDb, CSS&PCC, SV, OAT*, DoE*, FAST*, MCF(RSA)*, Sobol,* MCMC*, IR* Which observations are important and unimportant to parameters? Leverage, Cooks D, CV*, MOO* Are any parameters dominated by one observation and, thus, its error? Leverage, DFBETAS, CV* How certain are the parameter values? b/SDb, Parameter uncertainty intervals# Observations Parameters Predictions
Which parameters are important and unimportant to predictions? PSS, FAST* How certain are the predictions? z/SDz, Prediction uncer tainty intervals #, MMA* Which parameters contribute most and least to prediction uncertainty? PPR, FAST*, Sobol,* MCMC* Predictions Which existing and potential observations are important to the predictions? OPR, CV* Which models in MMA are likely to produce the best predictions? For individual model evaluations: AIC, AICc, BIC, KIC, CV* Computationally frugal methods (often 10s to 1,000s of model runs) Computationally demanding methods (often 10,000s to 1,000,000s of model runs)* 13 Methodologies, Tools and Applications Proceedings of the International Workshop on Uncertainty,
Sensitivity and Parameter Estimation for Multimedia Environmental Modeling (NUREG/CP-0187) Joint Universal Parameter IdenTifications and Evaluation of Reliability Application Programming Interface (JUPITER API) for programming computer programs designed to analyze process models, joint USGS and EPA project (Banta and others, 2006) Hydrologic Conceptual Model, Parameter and Scenario Uncertainty Methodology, cooperative project by the University of Arizona, PNNL and NRC staff (NUREG/CR6940) 14 Methodologies, Tools and Applications (continued) Model Abstraction Techniques for determining and identifying conceptual model structure and parameter estimation strategies, joint USDA/Agricultural Research Service and NRC staff (NUREG/CR-7026) Approaches in Highly Parameterized Inversion: PEST++, a Parameter ESTimation code optimized for large
environmental models by D. Welter, J. Doherty, R. Hunt, C. Muffels, M. Tonkin, and W. Schreuder. An objectoriented parameter estimation code that incorporates benefits of object-oriented programming techniques for solving large parameter estimation modeling problems. (USGS Techniques and Methods: 7-C5) 15 Forward Strategy Energize the science and technology thru closer linkage to decision making: better understand the methods being used in parameter estimation and uncertainty analyses establish a base set of model sensitivity analysis and uncertainty evaluation measures, in addition to the other performance measures use and compare different methods in practical situations 16 Recommendations for FY2013
Assist development and creation of other working groups Take advantage of the relevance of uncertainty and parameter estimation to all environmental modeling and monitoring fields. Develop and conduct joint ISCMEM teleconferences WG1 (Software System Design; design of uncertainty and parameter estimation software and data fusion) WG3 (Reactive Transport Models and Monitoring; support decision making) Act as an incubator to build support for new ideas Proposed WG on monitoring based on the importance of monitoring to uncertainty and parameter estimation, and visa versa Sponsor technical workshops on endorsed studies U.S. studies: Naturita, CO; Hanford-300 Area; OPE3 Beltsville, MD International study on monitoring and remediating Chernobyl Cooling Pond ISCMEM Website Use EPAs iemHUB to enhance Information Transfer of Technical Reports and Data Sources
The nozzle and hose fell to one side and he fell to the other on his elbow, hip and knee. The operator felt some pain but finished the fuelling procedure and drove back to the depot. During the night, the...
The number of molecular orbitals will always be the same as the number of atomic orbitals used to construct them. Bond Order Bond order is defined as bond order = ½ [(# of bonding e-) - (# of anti-bonding e-)]...
Many adverbs are formed by adding - ly. to adjectives. However not all words that end in - ly. are adverbs. The words . friendly, lively, and lonely . are usually adjectives. Therefore not all adverbs end in -ly. Adverbs...
Community Health Model. Defines recovery as "the process by which people living with mental illness and/or addictions learn how to minimize the disabilities associated with those conditions while pursuing a safe, dignified, meaningful, and constructive life in the community." (Yale...
Hip Jack Rafters. Short rafters that run between wall plate and hip rafter. Same as common rafters except for length. Consistent change in length is common difference. Valley Jacks. Start layout at building line and move toward ridge. Use common...
Hyperstudio has a kid-friendly interface and good drawing tools, but looks "less professional" than PowerPoint and doesn't have as many capabilities. Sample Kid Pix slides from a Comparatives/ Superlatives lesson and a unit on animals and their habitats.
Ready to download the document? Go ahead and hit continue!