You are here: Vanderbilt Biostatistics Wiki>Main Web>Education>IntroBiostatCourse2006>IntroBiostatCourseBlog (12 Apr 2007, ChunLi)EditAttach

- Analysis = software? (01/06/2007)
- An example of over-simplifying and mis-interpreting analysis results (05/03/2006)
- Missing data in likelihood ratio tests (04/24/2006)
- The coefficients of main and interaction effects are not comparable (04/21/2006)
- Questions in plain English and questions in statistical terms (04/21/2006)
- Wilcoxon’s signed rank and rank sum tests (04/19/2006)
- Think before you jump at an analysis (04/18/2006)
- Statistics is not just calculations (04/18/2006)
- Rounding values when reporting results (04/17/2006)
- Interpretation of the interaction effects in a regression analysis (04/17/2006)
- Interpretation of “main effect” when interaction effects are considered (04/14/2006)
- The full may be “greater” than the sum of its parts (04/11/2006)
- Power and sample size calculation (04/11/2006)
- Percent changes are not comparable with each other (04/10/2006)
- Cross-validation is not a cure to over-fitting (04/10/2006)
- Two eyes from an animal are correlated (04/10/2006)
- Odds ratios for continuous variables (04/09/2006)
- Intercept (04/09/2006)
- Results should always be interpreted with context (04/09/2006)
- Unit in interpretation (04/09/2006)
- “If I hadn’t carried out the other tests, I would have had the same result on this test.” (04/09/2006)
- Over-fitting (04/06/2006)
- Misperceptions of regression models (04/06/2006)
- Significance of a category in a categorical variable (04/06/2006)
- Maps of incidence rates (03/20/2006)
- “How can we find a significant result?” (03/19/2006)
- Extrapolation and interpolation: Interpretation of parameters depends on the range of the variables (03/19/2006)
- “Adjusted for …” (03/19/2006)
- Quadratic term and linear term (03/19/2006)
- Step-wise variable selection (03/19/2006)
- Permutation tests cannot do away with multiple comparisons (03/19/2006)
- Interaction is dependent on context (03/17/2006)
- Regression (03/17/2006)
- What is Statistics? Statistics is not a collection of tools/methods/tricks (03/05/2006)
- About the midterm (03/04/2006)
- We can be fooled by our labeling an analysis (03/04/2006)
- We can be fooled by contexts (03/02/2006)
- Pilot studies (03/01/2006)
- Degrees of freedom (02/26/2006)
- Permutation tests and permutation CIs (02/24/2006)
- Exactness is mostly an illusion and Fisher’s exact test (02/24/2006)
- The probability world is so different from the logic world (02/21/2006)
- P-value is not the probability of the null being true (02/21/2006)
- Pr(German | accident) ≠ Pr(accident | German) (02/21/2006)
- Statistics and probability (02/20/2006)
- Interaction is not product (02/13/2006)
- Interaction effect and main effect (02/13/2006)
- Ecological fallacy (02/10/2006)
- Watch out on p-values (02/10/2006)
- Study design is more than sample size calculation (02/09/2006)
- Measurement error and its effect (02/08/2006)
- Summary is not equal to analysis (02/07/2006)
- Interpretation of correlation coefficient in simple linear regressions (02/06/2006)
- Interpretation of t-test results in all regressions (02/06/2006)
- An example of using transformation (02/03/2006)
- One-sided vs. two-sided tests (02/01/2006)
- Hypothesis testing (02/01/2006)
- p-value (01/31/2006)
- "There is no statistical difference": A psychologically misleading phrase (01/27/2006)
- An example of simplistic understanding of p-value (01/26/2006)
- Abuse of statistics (01/25/2006)
- A guaranteed path to having a paper (01/25/2006)
- Variable selection and prediction: An example of artifact (01/25/2006)
- Ask questions (01/24/2006)
- Testing for difference and prediction are totally different animals (01/24/2006)
- Rate, frequency, probability, terminologies in general, and my confession (01/23/2006)
- Variation and interpretation of confidence intervals (01/22/2006)
- Value of exploratory analyses (01/19/2006)
- Outliers (01/19/2006)
- Categorization of a continuous variable (01/18/2006)
- Wrong ways of judging a statistician (01/18/2006)
- Wrong ways of judging a statistical method (01/18/2006)
- Wrong way of starting a project (01/18/2006)
- Miscellaneous (01/17/2006)
- Simpson's paradox (01/17/2006)

when x1 = 0, x2 = 0, right hand side = β

when x1 = 1, x2 = 0, right hand side = β

when x1 = 0, x2 = 1, right hand side = β

when x1 = 1, x2 = 1, right hand side = β

Here, the γ's are the effects estimated based on the data from the corresponding combinations. It is easy to show that β

clear set obs 20 gen x1 = invnormal(uniform()) gen x2 = invnormal(uniform()) gen z = x1 + x2 + 0.1*invnormal(uniform()) regress z x1 x2 regress z x1 regress z x2

For category h1, right hand side = β

For category h2, right hand side = β

For category h3, right hand side = β

For category h4, right hand side = β

For category h1, right hand side = β

For category h2, right hand side = β

For category h3, right hand side = β

For category h4, right hand side = β

when x

when x

Thus, effectively, you assumed the two groups (x

when x

when x

Here, you effectively assumed x

- All data analysis methods, including the simple ones, have assumptions. Only when the assumptions are met or approximately met can results be interpreted correctly. Unfortunately, people often are ignorant of this fact. It is even more unfortunate that many people who analyze data don't even know this fact. Analysis results are influenced by both assumptions and data, and you want to make sure your results are more of a function of data than a function of assumptions.
- Another type of abuse comes, surprisingly, from over use of some statistical concepts and procedures. Examples include 0.05 as a golden threshold, p-value as the only basis for judgment, hypothesis testing as a golden framework for all kinds of research, parametric analysis as default analysis, etc.
- Yet another common abuse is to apply many different methods on the same dataset. Many people have a laundry list of data analysis methods (or a boilerplate for grant applications), and they apply them all to any single dataset. They beat the data to the death, literally. This certainly increases the chance of seeing an "effect" or "pattern". Unfortunately, it is highly likely that these effects and patterns are not real at all.
- Some data analysis procedures can lead to artifacts. There are many ways of generating artifacts: multiple comparisons, over-fitting, step-wise variable selection, ecological fallacy, fishing/data-dredging, change of significance as other variables are added or removed (like Simpson's paradox), etc.
- Software availability has helped ease computation, but it also makes abuse of statistics easier. Many people appear to be able to do analysis just because they can easily generate professional-looking analysis output.

- you can give me a significant p-value while he can’t
- you propose to use more sophisticated/fancier methods
- you assure me to find the signal/gene/effect, while he told me uncertainty may exist and we may have chance to find nothing
- you tell me 150 subjects are enough while he told me 200 is minimum
- when I decide to do a study with a design well thought through (at least to myself), you always can provide statistical justifications even though statistics never played a role in my designing the study, while he told me I am too ambitious.

- We propose to collect 100 subjects for our analysis. This sample size has been sufficient to find the susceptibility gene for disease X (cite a Science paper). Fact: It is often the case that signals identified in past discoveries are stronger than that you dream to identify in your current study.
- The method is safe to use because it doesn’t generate false positives as I recall. Fact: There is no method that doesn't have false positives. The only exception is the one that never claims any positive results. Moreover, the probability for a bogus method to claim false positives is high enough for us not to trust the method, while at the same time low enough not to yield many false positives in a limited number of instances we have used the method.
- This method is good because it gives a p-value of 0.04 while that method gave 0.08, and I know the effect to be tested is real. Fact: This is using anecdotal evidence. A single instance often doesn't offer any evidence for or against a method. In fact, a method that tends to overfit the data will inflate everything, making a real signal appear to be stronger than any other methods can claim while at the same time increasing the chance of claiming false positives.

Edit | Attach | Print version | History: r113 < r112 < r111 < r110 | Backlinks | View wiki text | Edit wiki text | More topic actions

Topic revision: r113 - 12 Apr 2007, ChunLi

Copyright © 2013-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Ideas, requests, problems regarding Vanderbilt Biostatistics Wiki? Send feedback

Ideas, requests, problems regarding Vanderbilt Biostatistics Wiki? Send feedback