Chapter Four · failure evidence
What Sensitivity & Uncertainty Analysis got wrong, from 68 dissertations
Sensitivity and uncertainty analysis methods frequently encounter severe bottlenecks including numerical instability, prohibitive computational cost, and distorted uncertainty estimates across complex engineering models. Practitioners often find that ignoring parameter uncertainty leads to constraint violations, while attempting comprehensive global or robust modeling can result in intractable optimization problems. These records come from PhD theses at 26 institutions, 2021 to 2026. Each links to its thesis. They were extracted by language models reading the full text, so treat each as a lead to read, not a verdict.
Input data deficits and structural model errors undermine uncertainty quantification
Large measurement errors, missing empirical probability distributions, and structural model inadequacies frequently prevent reliable uncertainty propagation and candidate model discrimination. In these settings, increasing physical model fidelity or adding secondary measurements fails to reduce output predictive uncertainty because dominant input uncertainties remain unaddressed.
Tried and failed
Reaction rate uncertainty quantification in fluid models applied to plasma discharge simulation. Reason: Kinetic uncertainties alone cannot bridge experimental deviations, pointing to transport or structural model inadequacies.
Modeling of intermediate-pressure argon capacitively coupled plasmas with uncertainty quantification · UT Austin
Considered and rejected
Considered and rejected: Rejected Monte Carlo simulation for uncertainty analysis because empirical probability distributions were not definable for novel single-year trial and pilot-scale datasets
Combined Mitigation Strategies in Irish Pasture-Based Dairy: A Six-Year Life Cycle Assessment · Research Repository UCD
Considered and rejected
Considered and rejected: Rejected modeling elevational shifts based on thermal limits due to high model uncertainty
Strengthening the conservation of biodiversity at local and regional scales · DukeSpace
Considered and rejected
Considered and rejected: Rejected building a detailed empirical sensitivity analysis on the oil market model, because input uncertainties in mock data would prevent robust conclusions.
Considered and rejected
Considered and rejected: Rejected inclusion of indirect land use change (ILUC) modeling due to excessive model uncertainty.
Life Cycle Assessment of Drop-in Bio-jet Fuel and Acetic Acid from the Bioconversion of Poplar Biomass · ResearchWorks
Considered and rejected
Considered and rejected: Rejected using ungrounded or arbitrary input probability distributions in uncertainty analysis, as resulting simulation outcome distributions are ungrounded and misleading.
Approaches to occupant-centric building performance computing · DSpace-CRIS at TU Wien
Considered and rejected
Considered and rejected: Rejected using bootstrapping or cross-validation within this thesis to estimate sampling errors in model marginal likelihoods, focusing instead on prior model probability uncertainty.
Considered and rejected
Considered and rejected: Explicitly accounting for direct groundwater discharge in the hydrometric mass-balance due to large cross-basin uncertainty and lack of spatially resolved rates
A Systems Analysis of Trace Element Cycling in the Great Lakes · Queens University Institutional Repository
Tried and failed
parameter estimation using internal transient temperature profiles applied to packed bed thermal desorption models. Reason: energy balance model inadequacy caused large parametric uncertainty and inflated sensitivity metrics
Computational design of multi-sorbent adsorption processes for post-combustion carbon capture · Imperial
Tried and failed
calibrating linear material stiffness and boundary adjustments applied to finite element model validation. Outcome: data insufficient. Reason: field measurements and loading sequence notes had too much inherent uncertainty
Impact of lean-on bracing layouts on system stiffness behavior · UT Austin
Tried and failed
error-bounded measurement falsification for model discrimination applied to structural dynamic response models. Outcome: no signal. Reason: measurement uncertainty exceeded the magnitude of the measured metric, preventing candidate model discrimination
Tried and failed
increasing model fidelity to reduce uncertainty applied to multidisciplinary engineering design. Reason: large input parameter uncertainties dominate, preventing higher fidelity physics models from reducing output predictive uncertainty
A Methodology for Identifying Experiments For Uncertainty Mitigation in Complex Multi-Disciplinary Design · Georgia Tech
Tried and failed
stress tensor eigenvalue perturbation uncertainty quantification applied to transitional boundary layer flow. Outcome: did not generalise. Reason: eigenvalue perturbations alone were insufficient to bound skin friction discrepancy reference data
Quantification of Reynolds-averaged-Navier-Stokes Model Structural Hypothesis Uncertainty in Transitional Boundary Layer and Airfoil Flows · Queens University Institutional Repository
Tried and failed
adding modal frequency data to model falsification applied to structural model uncertainty quantification. Outcome: no signal. Reason: natural frequency measurements provided negligible reduction in parameter and response prediction uncertainty
Considered and rejected
Considered and rejected: Rejected modeling f*pi directly against a uniform measure when unnormalized, because prior elicitation for f*pi is difficult and causes unreliable uncertainty quantification
Probabilistic machine learning for aggregated and multivariate data · Imperial
Considered and rejected
Considered and rejected: Decided against plain statistical modeling techniques because they fail to handle the vagueness and inherent uncertainty of human quality judgments.
Data Science Techniques for Modelling Execution Tracing Quality · De Montfort Open Research Archive (DORA)
Surrogate and propagation methods generate distorted, physically inconsistent, or overly conservative uncertainty bounds
Techniques such as Monte Carlo dropout, analytical approximations, and weak boundary condition imposition often distort uncertainty envelopes or break physical constraints during evaluation. Consequently, these methods produce paradoxical confidence under input noise, severely underestimated predictive errors, or excessively conservative bounds that degrade control performance.
Tried and failed
Monte Carlo dropout for uncertainty estimation applied to classification on noisy sensor inputs. Reason: models exhibited overconfidence under high input noise, causing predictive uncertainty to paradoxically drop as accuracy degraded
Reliable Sensor Intelligence in Resource Constrained and Unreliable Environment · Georgia Tech
Tried and failed
Monte Carlo dropout on physics-constrained neural networks applied to temperature and density depth profile modeling. Reason: Dropout perturbations broke structural constraints, causing physically inconsistent predictions during uncertainty estimation
Physics-informed Machine Learning with Uncertainty Quantification · Virginia Tech
Tried and failed
Monte Carlo sampled independent deterministic inversions applied to correlated geophysical dispersion data. Reason: sampling strategy distorted output uncertainty bounds across frequencies
Lost to a baseline
SVM with RBF achieved higher R² (0.9857) and lower MAE (3.57) than MCDNN (R² = 0.9726, MAE = 5.07), but was rejected due to severely underestimated uncertainty.
Leveraging Machine Learning Surrogates and Stochastic Optimization to Enhance Disaster-Responsive Pharmaceutical Supply Chains · TXST Digital Repository
Lost to a baseline
Under balanced, homogeneous bias where true worst-case equals average bias (Γ'_truth = Γ_truth), extended sensitivity analysis for attributable effects is more conservative than the conventional worst-case calibration.
Considered and rejected
Considered and rejected: Rejected ensemble parameter variance (Var[θ]) as an epistemic uncertainty metric due to over-parameterization yielding variance even at optima.
Structured, Constrained and Creative Learning · Publikationssystem UB Tuebingen
Tried and failed
analytical uncertainty propagation through nonlinear reconstruction equations applied to plasma distribution function reconstruction. Reason: produced unrealistically large, unusable error estimates
The structure and dynamics of the near-sun solar wind · Imperial
Tried and failed
disk uncertainty robust control design applied to MIMO vibration isolation platform. Outcome: worse than baseline. Reason: overly conservative uncertainty bounds lead to sub-optimal performance and infeasibility with deep notch filters
Data-driven IQC-Based Robust Control Design for Hybrid Micro-disturbance Isolation Platform · EPFL
Lost to a baseline
Propagating decoder on 2D convection-diffusion achieved prediction error comparable to, but slightly worse than, an exact exponential integrator baseline due to parameter extraction uncertainty.
Interpretable Physics-informed Machine Learning Methods for Scientific Modeling and Data Analysis · MIT
Lost to a baseline
Weak imposition of stochastic boundary conditions failed to propagate boundary uncertainty into the interior near the inlet, yielding misaligned wake-only uncertainty and incorrect vortex shedding strengths compared to strong imposition with forced/unforced modes.
Lost to a baseline
Using Bayesian total uncertainty (predictive entropy) did not consistently beat deterministic or Bayesian average Maximum Softmax Probability (MSP).
OPEN-WORLD VIDEO STREAM FINGERPRINTING · Calhoun
Robust optimization and complex uncertainty set formulations create computational intractability
Formulating uncertainties via stochastic variables, ellipsoidal sets, fuzzy methods, or high-dimensional hierarchical structures introduces prohibitive combinatorial complexity into system optimization. These formulations frequently become computationally intractable or fail to provide statistical advantages over simpler uniform uncertainty models.
Tried and failed
spatially varying uncertainty modeling in robust optimization applied to multiscale structural design. Outcome: infeasible cost. Reason: offered no statistical benefit over uniform uncertainty models while increasing computation by 22x
Methods for improving the robustness of optimised multiscale structures · Imperial
Tried and failed
treating recovery parameters as stochastic variables applied to uncertainty set computation. Outcome: infeasible cost. Reason: introduced combinatorial complexity and nonlinearities into computation without meaningful accuracy gains
Quantifying the Unknown: Data-Driven Approaches and Applications in Energy Systems · EPFL
Tried and failed
multiple domain matrix concept generation applied to engineering systems design under uncertainty. Outcome: data insufficient. Reason: abandoned due to excessive complexity and data requirements
Tried and failed
Advanced Mean Value method for uncertainty analysis applied to MILP network optimization under uncertainty. Outcome: unstable. Reason: Non-monotonic CDFs and mismatch with Monte Carlo due to solver noise and run-time cutoffs
A Methodology for the Inclusion of Uncertainty in Space Logistics Campaign Planning and Optimization · Georgia Tech
Considered and rejected
Considered and rejected: Rejected using fuzzy-based MCDM to handle uncertainty due to mathematical complexity, lack of standard solution technique, and difficulty incorporating quantitative factors.
Methodologies for Frequency Stability Assessment in Low Inertia Power Systems · IRIS - POLITO - prod
Considered and rejected
Considered and rejected: Rejected frequentist linear mixed models (e.g., lme4) and variational inference due to intractable or unreliable uncertainty estimation and p-value derivation in high-dimensional hierarchies.
Studying the tissue-specificity of cancer driver genes through KRAS and genetic dependency screens. · Harvard
Considered and rejected
Considered and rejected: Rejected exact optimal common precoder optimization in Massive MIMO because beamforming gain uncertainty in the denominator makes the problem intractable, approximating uncertainty as negligible.
Considered and rejected
Considered and rejected: Ellipsoidal uncertainty sets were rejected for semi-variance portfolios because the resulting optimization problem is not computationally tractable.
Essays on portfolio optimization and estimation risk · Research Repository UCD
Considered and rejected
Considered and rejected: Rejected enforcing diagonal sub-blocks in the structured uncertainty matrix due to lack of existing analysis and computational tools.
Feedback interconnection based input-output analysis of spatio-temporal response in wall-bounded shear flows · JScholarship
Considered and rejected
Considered and rejected: Rejected standard Multi-Objective Robust Decision Making (MORDM) due to lack of adaptive feedback and poor tractability under deep parameter uncertainty.
Ship and Naval Technology Trades-Offs for Science And Technology Investment Purposes · Georgia Tech
Variance-based global sensitivity analysis incurs prohibitive computational expense and numerical instability
Standard Sobol and global variance-based approaches often fail to achieve parameter convergence within restricted simulation budgets, yielding erroneous rankings or negative sensitivity indices under strong parameter coupling. Because of these computational burdens and distributional restrictions, practitioners routinely reject standard Sobol methods in favor of simpler screening techniques like the Morris method.
Tried and failed
first-order Sobol sensitivity analysis applied to techno-economic life cycle cost models. Outcome: no signal. Reason: Strong multi-variable parameter coupling made individual first-order parameter contributions negligible.
Tried and failed
second-order Sobol global sensitivity analysis applied to multi-input simulation models. Reason: Interacting input variables counteract each other's variance contributions, producing negative sensitivity indices.
A Methodology for Identifying Experiments For Uncertainty Mitigation in Complex Multi-Disciplinary Design · Georgia Tech
Lost to a baseline
The Sobol global sensitivity analysis method was outperformed by both the Morris method and VARS in parameter ranking reliability and convergence at lower computational budgets (1k-20k runs).
Barrier Island Morphodynamic Insights from Applied Global Sensitivity Analysis and Decadal Exploratory Modeling · Virginia Tech
Considered and rejected
Considered and rejected: Restricting sensitivity analysis exclusively to the standard Sobol method was rejected because its high computational burden causes extreme numerical instability and erroneous parameter rankings under limited simulation budgets (<10,000 runs).
Barrier Island Morphodynamic Insights from Applied Global Sensitivity Analysis and Decadal Exploratory Modeling · Virginia Tech
Considered and rejected
Considered and rejected: Rejected Sobol variance-based sensitivity analysis methods because they cannot explain localized observations across different nodes and time points in a WDS.
AI Methods for Anomaly Detection in Cyber-Physical Systems: With Application to Water and Agriculture · Virginia Tech
Considered and rejected
Considered and rejected: Decided against Global Sensitivity Analysis (GSA) during initial tool development in favor of OAT sensitivity analysis due to computational simplicity.
Considered and rejected
Considered and rejected: Rejected global sensitivity analysis methods (Sobol's method and Extended FAST method) due to failure to achieve parameter convergence within a feasible timeframe; substituted with weighted averaging of local sensitivities.
Systems Biology of Blood Coagulation and Platelet Activation · Penn
Considered and rejected
Considered and rejected: Sobol sampling sensitivity analysis rejected because discretization of visual parameters violated the required uniform distribution assumption, prompting a switch to the Morris method.
COGNITIVE AGENTS FOR WAYFINDING UNDER UNCERTAINTY IN UNFAMILIAR INDOOR ENVIRONMENTS · Cornell
Considered and rejected
Considered and rejected: Rejected using global Monte Carlo sensitivity analysis due to the computational overhead and file setup complexity in WUFI Pro, choosing the Morris OAT method instead.
Hygrothermal Analysis of Vacuum Insulation Panels in New and Retrofit Walls for Canada · Carleton University Institutional Repository
Local and linearized sensitivity methods fail to capture nonlinear parameter interactions
One-at-a-time, finite difference, and linearized sensitivity methods cannot represent complex multi-variable trade-offs across non-linear response spaces. These local approximations distort parameter importance rankings and fail to predict actual variance reductions in dynamic coupled simulations.
Considered and rejected
Considered and rejected: Rejected local sensitivity analysis because it fails to capture higher-order parameter interactions compared to Sobol
Urban building energy modeling : from sensitivity to big data · UT Austin
Tried and failed
model linearization for global sensitivity analysis applied to coupled multidisciplinary engineering simulation. Reason: linearization failed to capture output variance and distorted variance-based sensitivity index rankings
Enabling End-to-End Sensitivity Analysis of Integrated Models · MIT
Considered and rejected
Considered and rejected: Rejected conventional post-optimization sensitivity analysis because it is purely local and unsuited for global policy analysis.
Measuring technical efficiency in a fuzzy environment · Virginia Tech
Considered and rejected
Considered and rejected: Rejected finite differencing for global sensitivity analysis due to slow runtime and poor linear approximations over non-linear performance spaces.
Considered and rejected
Considered and rejected: Rejected sensitivity analysis in favor of multi-objective scenario-based planning because sensitivity analysis isolates individual variables without accounting for complex trade-offs.
Cost-Benefit Analysis of Anaerobic Digestion in Southern Ontario: A Case Study · Scholarship at UWindsor Institutional Repository
Considered and rejected
Considered and rejected: Rejected local sensitivity analysis methods (one-at-a-time, derivative-based) due to inaccuracy under non-linearities and parameter uncertainty.
Reliability and up-scaling of offshore wind turbines · Texas Tech
Tried and failed
ensemble sensitivity analysis for observation targeting applied to convective numerical weather prediction. Outcome: no signal. Reason: linear sensitivity metrics failed to predict actual non-linear variance reduction from data assimilation
An Examination of the Fundamental and Applied Aspects of Observation Targeting Strategies for Severe Convection · Texas Tech
Omitting parameter uncertainty and relying on deterministic assumptions causes operational failures
Employing deterministic scheduling, point estimates, or zero initial variance assumptions leads directly to frequent process constraint violations and severe undercoverage of credible intervals. Neglecting parameter uncertainty during estimation also results in unrealistic error metrics and biologically implausible parameter values.
Tried and failed
nominal model-based design of experiments applied to systems with parametric uncertainty. Reason: ignoring parameter uncertainty caused designed experiment trajectories to violate process constraints and become infeasible
Tried and failed
plugging point estimates into downstream Bayesian equations applied to low-rank matrix completion uncertainty quantification. Reason: ignoring parameter uncertainty in point estimates caused severe undercoverage of nominal credible intervals
New Guassian Process Modeling for Low-Rank and Simulated Data · Georgia Tech
Tried and failed
zero initial covariance in sensitivity analysis applied to discrete Kalman filter error evaluation. Reason: yielded unrealistic error estimates compared to Monte Carlo simulations by ignoring baseline state uncertainty
Integration of global positioning and inertial reference system data inside a flight management computer · Cranfield
Tried and failed
deterministic optimal scheduling under uncertainty applied to coupled multi-energy network dispatch. Outcome: did not generalise. Reason: ignoring stochastic uncertainty led to frequent voltage and temperature constraint violations in realization scenarios
Co-optimisation of electrical and district heating networks · Imperial
Tried and failed
least squares estimation without uncertainty modeling applied to kinetic rate parameter estimation. Outcome: unstable. Reason: lack of uncertainty modeling led to non-reproducible and biologically implausible fast rate estimates
The Kinetics of RNA Flow Across Subcellular Compartments · Harvard
Adjoint and derivative-based sensitivity analysis suffers from gradient instability and slow convergence
Adjoint sensitivity applied to chaotic systems or inverse estimation problems suffers from gradient explosion, poor convergence to global minima, and excessive runtime compared to forward methods. Furthermore, omitting indirect sensitivity terms causes rapid divergence, while computing arbitrary-order derivatives creates impractical memory scaling.
Tried and failed
adjoint sensitivity analysis applied to chaotic differential equations. Outcome: unstable. Reason: exponential sensitivity to initial conditions causes gradient explosion and unmanageable computational cost
Error behavior and optimal discretization of chaotic differential equations · MIT
Considered and rejected
Considered and rejected: Rejected computing arbitrary-order sensitivity analysis derivatives for Doppler broadened uncertainty quantification due to impractical computational and memory scaling.
Tried and failed
partial sensitivity modeling omitting indirect terms applied to temperature-dependent parameter uncertainty propagation. Outcome: worse than baseline. Reason: omitting indirect sensitivity terms broke error cancellation, causing faster divergence than uncorrected baseline
Tried and failed
adjoint sensitivity analysis with gradient descent applied to PDE-constrained inverse parameter estimation. Outcome: worse than baseline. Reason: Adjoint method had lower success converging to global minima and took significantly longer than forward sensitivity.
Data-driven Modeling of Lithium Intercalation Materials · MIT
Left open by the authors
Problems the authors named and did not get to.
Left open
Extend the Sobol' sensitivity analysis chaining framework to time-dependent dynamic systems governed by differential equations with multi-scale trajectory outputs. Blocker: Lacks specific mathematical formulation or chaining methodology for dynamic differential equation systems.
Enabling End-to-End Sensitivity Analysis of Integrated Models · MIT
Left open
Test whether the unstable component in space-split sensitivity analysis can be neglected across high-dimensional statistically homogeneous physical flow simulations. Blocker: Lacks specific test flow cases and concrete evaluation metrics for universal neglectability
Left open
Develop a computationally efficient method to compute conservative sensitivity analysis bounds for continuous exposures and continuous outcomes without signomial programming. Blocker: Lacks a specific mathematical formulation or concrete algorithmic approach to bypass signomial programming.
SENSITIVITY ANALYSIS METHODS FOR OBSERVATIONAL STUDIES WITH A CONTINUOUS EXPOSURE · Penn
Left open
Model implementation uncertainties in strategic water supply plans beyond stochastically varied antecedent climate conditions. Blocker: No specific formulation or implementation framework provided for modeling real-world implementation uncertainty
Optimizing Water System Resilience in Midsized Cities in Arid and Semi-Arid Regions · Texas Tech
Left open
Quantify uncertainty in history-matched fault permeability models using ensemble and probabilistic methods without ground truth data. Blocker: None
Numerical Modeling of Geologic Carbon Dioxide Storage in Faulted Siliciclastic Settings · MIT
Left open
Incorporate multi-layer compound uncertainty methods into sustainable watershed development models for large-scale real-world basins. Blocker: No specific basin, dataset, or mathematical model formulation is defined.
Left open
Develop dynamic non-linear sediment rating curve models and hierarchical Bayesian models to quantify uncertainty in the variance discount factor. Blocker: None
Left open
Perform uncertainty and parameter propagation analysis on the MCDM and KDE drought vulnerability framework to compute uncertainty bounds. Blocker: None
Left open
Evaluate observational uncertainty propagation in tracer-aided sensitivity analysis and multi-objective hydrologic calibration frameworks. Blocker: None
Utility of stable isotope tracer data in large scale hydrologic modeling · MSpace - University of Manitoba
Left open
Extend surrogate uncertainty quantification to include input, process, and material uncertainties, scaling to multi-physics process-structure-property-performance modeling in additive manufacturing. Blocker: Lacks specific implementation details, mathematical formulation, and proprietary training data/multi-physics simulation models from the thesis
Smart Quality Assurance System for Additive Manufacturing using Data-driven based Parameter-Signature-Quality Framework · Virginia Tech
Checking a claim in this area?
We can run the same search on any method or claim. If nothing turns up, we will say so, and that proves nothing on its own.