About Spondylus Expert

The Problems with Traditional Expert Elicitation

Organizations frequently must make decisions in areas where empirical data is sparse, unreliable, or non-existent. Traditionally, they rely on methods like "BOGSAT" (a "Bunch of Guys/Girls Sitting Around a Table") or the Delphi method. However, these approaches suffer from significant flaws:

Troublesome Group Dynamics: The Delphi method relies on experts viewing each other’s assessments and revising them to reach a consensus. This often results in experts changing their answers to align with perceived "leading experts" or dominant personalities, rather than following the strongest arguments.
The "Confidence Trap": Experts are often highly overconfident, expressing high certainty in predictions that are statistically highly unlikely to be accurate.
Poor Proxies for Expertise: An expert's reputation, publication record, or self-rated confidence does not reliably correlate with their actual ability to make accurate probabilistic predictions.
Lack of Uncertainty Quantification: Older methods often ask for single "point estimates" (best guesses) which fail to capture the uncertainty of a situation, making them inadequate for risk modeling.

The Solution: The Delft Method (Classical Model)

The Delft method solves these issues by treating expert judgment as scientific data that must be subjected to strict empirical control, bypassing forced group consensus in favor of mathematical aggregation.

Seed Questions: Before assessing the unknown variables of interest, experts are asked "seed" or "calibration" questions. These are questions from the experts' field where the true answers are known to the analysts but not to the experts.
Performance Scoring: Experts are objectively scored on two metrics based on their answers to the seed questions: Statistical Accuracy (calibration, or how often the true values fall within their stated confidence intervals) and Informativeness (how concentrated and precise their distributions are).
The "Decision Maker": The method multiplies these scores to create a performance-based weight for each expert. These weights are then used to mathematically combine the experts' individual judgments into a single, optimized distribution known as the "Decision Maker".

Advantages

Empirical Validation: It replaces blind trust in an expert's credentials with objective, trackable performance validation against empirical data.
Neutrality and Reduced Bias: The method uses a strictly proper scoring rule. This means experts maximize their weight only by stating their true beliefs, effectively removing incentives to manipulate their answers or follow the crowd.
Superior Accuracy: Validated data shows that performance-weighted combinations (the Delft method) consistently outperform equally-weighted groups of experts and harmonic weighting schemes, maintaining high statistical accuracy while drastically increasing the informativeness of the predictions.

Success Stories

The Delft method has been successfully deployed in over 100 panels across multiple high-stakes disciplines:

WHO Global Foodborne Diseases: Aggregated judgments from 72 internationally dispersed experts to provide rigorous, science-based estimates of the global burden of foodborne diseases.
Great Lakes Invasive Species: Quantified the economic damages of ship-borne invasive species (like Asian Carp), estimating a $138.3 million annual loss and demonstrating that the ecological losses exceeded the transportation savings of keeping the St. Lawrence Seaway completely open.
Nuclear Safety: Used collaboratively by the U.S. Nuclear Regulatory Commission and the European Commission to quantify uncertainties in probabilistic accident consequence codes (MACCS and COSYMA), modeling atmospheric dispersion, deposition, and health effects from potential nuclear accidents.
Ice Sheet Sea-Level Rise: A major study projecting future sea-level contributions from ice sheets revealed much larger uncertainties than previous models, making international headlines (BBC, CNN) and informing global climate change discussions.
Public Health: Forecasted the future trajectory of antibiotic resistance across European countries and estimated the impact of breastfeeding on cognitive development in low- to middle-income countries to assist the Bill & Melinda Gates Foundation.

Benefits for Organizations

Achieving Rational Consensus: When data is lacking and experts disagree, organizations are often accused of partiality if they choose one expert's view over another. By committing to the Delft method's objective mathematical rules in advance, organizations can achieve a defensible, rational consensus.
Accountability and Transparency: The method is fully auditable. All data, processing tools, and expert identities are documented and open to peer review, ensuring the decision-making process is transparent and fair.
Informed Policy and Risk Management: By accurately quantifying the uncertainty of rare or extreme events (e.g., river floods, industrial accidents), organizations can perform robust cost-benefit analyses, prioritize research, and optimize safety and security measures.