Basic concepts in statistical analysis

Mark of Good Math, Bad Math has two great basic "numbers" posts -

- Basics: Mean, Median, and Mode -- great succinct comparison of why you would want to see a median value to summarizing some data, while other data is more suited to a mean value (the traditional "average").
- Basics: Normal Distributions -- good refresher on what a normal distribution looks like, with tie-in to the mean, median, and mode examples used in the above post, as well as a tax-cut example that seems appropriate for this time of year too.
(found via Chad at Uncertain Principles)

evolgen reminds us of the "evil" side of statistics, linking to a couple of related examples of torturing the data until a statistically significant result is found, and the dangers of such abuse of statistical techniques.

For those interested in understanding more about statistical analysis, here's a great article that discusses some of the limitations of the traditional null hypothesis test -
Gigerenzer G. Mindless statistics (J Socio-Econ 2004; 33:587–606) - Gigerenzer examines significance testing "traditions" in the social sciences, asking "Why is statistics carried out like compulsive handwashing?" and advocating that these fields consider that there is "a statistical toolbox rather than one hammer."

The author also examines a strong rationale for reporting the actual p value obtained in research, rather than sticking to a "mechanical" threshold such as "p<0.05" or "p<0.01" as well as the role of "good descriptive statistics" such as those in the Good Math, Bad Math posts linked above, as well as other measures including confidence intervals, statistical power, and effect sizes.

There's also a brief quiz on p. 594-595 of this article that illustrates some of the more common misperceptions about the meaning of significance testing results, which tend to err on the side of inferring more from the results than is indicated by the constraints of such tests.

Publication bias and statistical significance

Chris of Mixing Memory has blogged a great discussion of statistical significance, scientific publishing, and publication bias - the lower likelihood of getting accepted for publication if your study has null results, i.e. fails to show a statistically significant difference.

For more details on publication bias, consider the Cochrane Collaboration description and the results from a quick PubMed search on the topic ("publication bias" has been a Medical Subject Heading (MeSH) term since 1994) .

David Wallace of Fayetteville State University has developed a good description of Type I and Type II statistical errors (PDF file).

Biomed Central articles: international collaboration, research quality

A few interesting articles from the Biomed Central journals:

• Gonzalez Block MA. The state of international collaboration for health systems research: what do publications tell? Health Res Pol Sys 2006, 4:7 - analyzed author information from to evaluate how much international collaboration is going on in research focusing on health systems and policies in developing countries. Concludes "While there is modest health systems research capacity in many developing countries for single country studies, capacity is severely limited for multi-country studies."


• Nieminen P, Carpenter JR, Rucker G, Schumacher M. The relationship between quality of research and citation frequency. BMC Medical Research Methodology 2006, 6:42 - looked at the quality of statistical analysis and results reporting in 4 psychiatric journals. Authors note: "Unclear or inadequate reporting of the research question and primary outcome were not statistically significantly associated with the citation counts. After adjusting for journal, extended description of statistical procedures had a positive effect on the number of citations received. Inappropriate statistical analysis did not affect the number of citations received. Adequate reporting of the primary research question, statistical methods and primary findings were all associated with the journal visibility and prestige."
• Poolman RW, Struijs PAA, Krips R, Sierevelt IN, Lutz KH,, Bhandari M. Does a "Level I Evidence" rating imply high quality of reporting in orthopaedic randomised controlled trials? BMC Medical Research Methodology 2006, 6:44 - examined reporting of evidence levels in the Journal of Bone and Joint Surgery, American Volume, found that the reported level of evidence did not correlate with quality of the corresponding study