Department of Biostatistics Seminar/Workshop Series

Ruminations on Statistical Evidence

Jeffrey Blume, PhD

Associate Professor, Department of Biostatistics
Vanderbilt University School of Medicine

Wednesday, December 10, 1:45-2:55pm, MRBIII Conference Room 1220

Intended Audience: Persons interested in applied statistics, statistical theory, epidemiology, health services research, clinical trials methodology, statistical computing, statistical graphics, R users or potential users

Science looks to Statistics for an objective measure of the strength of evidence in a given body of observations. But Statistics offers only a patchwork of borrowed principles and tools from decision and belief theory to guide the assessment of evidence in data. A ‘theory of evidence’ has yet to be established and, as a result, statistical assessments of evidence are beset with irresolvable controversies, such as those surrounding the proper use and interpretation of p-values or those concerning adjustments for multiple comparisons and multiple looks at data.

In this talk, I’ll examine this patchwork and argue that it does not substitute for a general ‘theory of evidence’, which is sorely needed. We’ll see that a ‘theory of evidence’ must distinguish between three key evidential quantities and that this mere distinction resolves the aforementioned controversies. These concepts are (1) a measure of the strength of evidence, (2) the probability that a particular study design will generate misleading or weak evidence, and (3) the probability that an already observed measure of evidence is misleading. Because such a distinction does not exist in today’s Frequentist or Bayesian methodologies, I’ll appeal to the Likelihood paradigm to illustrate these concepts in some simple examples and two well known clinical trials.