January JMLA case posted

The newest installment in the new case study series in the Journal of the Medical Library Association is now available in PubMed Central.

This month's case looks at using case reports to answer a question about diarrhea in an ICU patient possibly caused by Clostridium difficile (C-diff) infection involving the small bowel.

Also see the JMLA case study blog for more information and discussion of the case.

Comparing diagnostic tests: verification bias

An article in this week's Archives of Internal Medicine discusses a limitation of study design and execution that can happen in comparisons of diagnostic testing options, an issue known as verification bias.

The scenario:
You have a new diagnostic test, Exciting Test A, that may be an option for seeing if patients have Awful Disease X.

You also have Old-Standby Test B, the existing "gold standard" diagnostic test for diagnosing Awful Disease X ("gold standard" means that Old-Standby Test B is the best thing you had going up until now to figure out if someone has Awful Disease X).

You want to set up a study to see if Exciting Test A is an accurate test for diagnosing this disease, in comparison to the Old-Standby.

There are lots of pitfalls in designing this kind of study (the Bandolier site has a really good discussion of the most common potential limitations of diagnostic studies).

An example of one of these pitfalls - verification bias:
The study by Lauer et al. in this week's Archives estimates the impact of verification bias - this kind of bias happens when everyone in the study gets Exciting Test A, but not everyone gets Old-Standby Test B - i.e. the "truth" of the Test A results are not verified in the whole set of patients by Test B , which should be the definition of true disease status.

The reference: Lauer MS, Murthy SC, Blackstone EH, Okereke IC, Rice TW. [18F]Fluorodeoxyglucose Uptake by Positron Emission Tomography for Diagnosis of Suspected Lung Cancer: Impact of Verification Bias. Arch Intern Med. 2007;167:161-165 (abstract).

What this study looked at:
- The patient population: 534 patients with suspected lung cancer (Awful Disease X)

- Exciting Test A: PET scan

- Old-Standby Test B: tissue diagnosis (including mediastinoscopy, transbronchial biopsy, thoracotomy, percutaneous fine needl aspiration, or thoracentesis)

- 419 patients (78%) underwent both PET scan and tissue diagnosis. In this group, sensitivity (people with the disease who test positive) of PET scanning was 95% and specificity (people without the disease who test negative) was 31% (both figures related to the test's ability to detect cancer at any site).

- Authors used two methods to adjust for verification bias (since 115 patients only underwent PET scanning): the Diamond method (relatively simple) and the Begg Greenes method (more complex formula).

- Using the Diamond method, the adjusted sensitivity was 87% and the adjusted specificity was 55%. The Begg Greenes method yielded a sensitivity of 85% and 51% specificity. So, with each method of adjustment, sensitivity went down (a lower percentage of people with lung cancer actually tested positive) and specificity went up (a higher percentage of people without lung cancer actually tested negative).

- "Real world" meaning of these estimates -- a higher proportion of diagnoses of lung cancer were probably missed by PET scanning when it was not accompanied by tissue diagnosis -- so a greater number of lung cancer cases were missed by the PET-scan-only approach than the results would indicate if you didn't account for verification bias (i.e. if you ignore the potential impact of verification bias, PET scanning looks better than it actually is for diagnosing lung cancer).

- The authors conclude that verification bias in this case has a substantial impact on the measures of diagnostic accuracy for PET in assessing cases of suspected lung cancer, and suggest that clinicians should "lower their threshold for proceeding to definitive tissue diagnosis in the setting of negative PET scan findings."


Another prominent evaluation of verification bias:
Punglia RS, D’Amico AV, Catalona WJ, Roehl KA, Kuntz KM. Effect of verification bias on screening for prostate cancer by measurement of prostate-specific antigen. N Engl J Med. 2003;349:335-342. (full-text)

Basic concepts in statistical analysis

Mark of Good Math, Bad Math has two great basic "numbers" posts -

- Basics: Mean, Median, and Mode -- great succinct comparison of why you would want to see a median value to summarizing some data, while other data is more suited to a mean value (the traditional "average").
- Basics: Normal Distributions -- good refresher on what a normal distribution looks like, with tie-in to the mean, median, and mode examples used in the above post, as well as a tax-cut example that seems appropriate for this time of year too.
(found via Chad at Uncertain Principles)

evolgen reminds us of the "evil" side of statistics, linking to a couple of related examples of torturing the data until a statistically significant result is found, and the dangers of such abuse of statistical techniques.

For those interested in understanding more about statistical analysis, here's a great article that discusses some of the limitations of the traditional null hypothesis test -
Gigerenzer G. Mindless statistics (J Socio-Econ 2004; 33:587–606) - Gigerenzer examines significance testing "traditions" in the social sciences, asking "Why is statistics carried out like compulsive handwashing?" and advocating that these fields consider that there is "a statistical toolbox rather than one hammer."

The author also examines a strong rationale for reporting the actual p value obtained in research, rather than sticking to a "mechanical" threshold such as "p<0.05" or "p<0.01" as well as the role of "good descriptive statistics" such as those in the Good Math, Bad Math posts linked above, as well as other measures including confidence intervals, statistical power, and effect sizes.

There's also a brief quiz on p. 594-595 of this article that illustrates some of the more common misperceptions about the meaning of significance testing results, which tend to err on the side of inferring more from the results than is indicated by the constraints of such tests.

Nature's new blog: Spoonful of Medicine

Nature's editors have started a new blog, Spoonful of Medicine: musings on science, medicine and politics.

My two favorite posts so far:
- Nature Medicine 2.0 - ponders Web 2.0's potential effects on scientific publishing, with some great reader comments (related editorial in Nature itself, "Nature Medicine 2.0" with the headline "In the future, Nature Medicine will be famous to 15 people.")

- The figure police - considers the journal editor's responsibility to scrutinize images in submitted or accepted manuscripts, discussing at what point the law of diminishing returns kicks in and how to reasonably address the issue of image quality and "truth" (related posts here - "How pretty are jellyfish?" and "Image editing in the medical literature")

Another variety post: ScienceBlogs round-up

Committed to cleaning out my aggregator today - here are some interesting things from the ScienceBlogs family:

- Pharyngula explains "What is a gene?" (complete with great illustrations)

- A Blog Around the Clock gives a great introductory overview of how genotype affects phenotype

- Mike the Mad Biologist discusses a new state initiative in Massachusetts to monitor and evaluate compliance with processes to reduce hospital-acquired infections

- Martin of Aardvarchaeology brings thesis work by Isto Huvila melding information science and archaeology to our attention - the thesis is titled "The Ecology of Information Work."

-Cognitive Daily talks about reasons why doctors report that they don't enroll their patients in clinical trials

- Terra Sigillata notes that legislation that will provide for herbal adverse event reporting has passed through Congress and signed into law, to take effect in one year and to be administered by the FDA.

And, finally, because it's Saturday and this article was a very fun read, A Blog Around the Clock points out a CMAJ article that discusses the medicine of Harry Potter.

BMC variety post: EBM, medical errors, research design and synthesis

A few interesting BioMed Central articles that I've read recently:

- Khan KS, Coomarasamy A. A hierarchy of effective teaching and learning to acquire competence in evidence-based medicine. BMC Medical Education 2006, 6:59 - proposes a hierarchical structure of strategies for educating yourself and others about EBM principles and implementation strategies, ranging from classroom didactic at level 3 to interactive, clinically integrated activities at Level 1

- Jansen YJ, Bal R, Bruijnzeels M, Foets M, Frenken R, de Bont A. Coping with methodological dilemmas; about establishing the effectiveness of interventions in routine medical practice. BMC Health Services Research 2006, 6:160 - discusses research design and conduct in one example of pragmatic research, a clinical trial constructed to explore the effectiveness of strategies used in routine clinical practice

- Berendsen AJ, Benneker WH, Meyboom-de Jong B, Klazinga NS, Schuling J. Motives and preferences of general practitioners for new collaboration models with medical specialists: a qualitative study. BMC Health Services Research 2007, 7:4 - a qualitative study exploring Dutch general practitioners views and preferences for developing new collaborations with specialty consultants; the discussion of motivational factors (e.g. developing personal relationships; gaining mutual respect) may be useful for researching models for clinician consultation with librarians

- Elder NC, Pallerla H, Regan S. What do family physicians consider an error? A comparison of definitions and physician perception. BMC Family Practice 2006, 7:73 - systematic review of the literature for working definitions of "medical error" and pilot survey of clinicians' perceptions of those definitions and examined how clinicians responded to 5 error scenarios, illuminating lack of consensus in the literature and in family physicians' perceptions about what constitutes a true medical error. The authors note: "...we found that three areas may affect how physicians make decisions about error: the process that occurred vs. the outcome that occurred, rare vs. common occurrences and system vs. individual responsibility."

- Bornhöft G, Maxion-Bergemann S, Wolf U, Kienle GS, Michalsen A, Vollmar HC, Gilbertson S, Matthiessen PF. Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity. BMC Medical Research Methodology 2006, 6:56 - proposes a checklist for use in examining internal validity and model validity, to help in uncovering potential sources of bias during the process of research design and evaluating existing research reports

- Lucas PJ, Arai L, Baird J, Law C, Roberts HM. Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Medical Research Methodology 2007, 7:4 - compares and contrasts two different strategies (textual narrative and thematic synthesis) for synthesizing qualitative aspects of clinical research for inclusion in systematic reviews. The authors conclude, "Thematic synthesis holds most potential for hypothesis generation, but may obscure heterogeneity and quality appraisal. Textual narrative synthesis is better able to describe the scope of existing research and account for the strength of evidence, but is less good at identifying commonality."

News on e-prescribing

Time this week published a small piece by Jeremy Caplan discussing the role of electronic prescribing tools in eliminating some medication errors - "Cause of death: sloppy doctors." It focuses on the National ePrescribing Patient Safety Initiative, which launched this week and will provide a free web-based eprescribing tool to doctors.

The article also notes that Google is working with the NEPSI project to develop a search engine to aid physicians in finding health data.

Related:
- the National e-prescribing Patient Safety Initiative, discussed on CNN Money and GovernmentHealth IT (also includes a list of collaborating companies and healthcare organizations)
- related federal regulations
- a PubMed search, retrieving a number of articles discussing challenges and benefits of e-prescribing

What do you think is the biggest medical advance since 1840?

The British Medical Journal has published a supplement on their picks for the top 15 advances in medicine since their launch in 1840. It includes computers, the discovery of DNA, antibiotics, anesthesia, figuring out how germs cause disease and how our body fights off infection, the importance of cleanliness and good hygiene, vaccination, and the risks of smoking, among others. Hey, Rachel - The pill made the list too!

When you read the list, you can't help but think about how lucky we are to benefit from all these advances, and how many lives have been saved or improved because of the changes.

If you're interested in this sort of thing, you have until the end of the day on Sunday to place your vote to help decide which of the 15 is crowned #1.

Related:The Hamilton Spectator did a nice piece on McMaster University's leading role in the development of the evidence-based medicine concept, the idea that clinical practice should be based on clinical expertise, the best relevant scientific research, and the values of the patient, which also made BMJ's list.

Update: And the winner is....
Sanitation, with almost 16% of the vote, followed by antibiotics in 2nd place and anaesthesia a close 3rd.

Clinical trial registration

In an editorial by Drazen and Zarin in this week's NEJM, "Salvation by Registration," there's a brief tale about an author who submitted a paper to the journal, and the reported trial was incorrectly entered into clinicaltrials.gov - the paper was initially refused due to this issue, and it turned out that the trial's sponser was responsible for the incomplete record in the database. One of the co-authors inadvertently worked around this issue by creating a new, complete and accurate record, and the paper is included in this week's issue.

The editorial points out the importance of a complete trial protocol submission to a public registry in compliance with the ICMJE initiative; they also point to the ineffectiveness of duplicate efforts on the part of the study sponsor and the study investigators, like that noted above.

Details on registering trials in clinicaltrials.gov in compliance with these recommendations is available here -- looking at the protocol questions/issues that need to be addressed during clinical trial registration is also a really good refresher course on the key methodological details of such studies.

Blogging and the AMA

The newsletter of the American Medical Association, AMNews, included an article last week, "Bloggers' Grand Rounds: The evolution of medicine's netroots," that focused on medical blogging - includes brief interviews with , Nicholas Genes, the founder of the Grand Rounds collection of "the best posts in medical blogging," and several other prominent physician bloggers. The article also includes a brief section of common-sense tips for physician readers interested in blogging, including:

- Never disclose information or details that identify patients.
- Tell readers you're masking identities and consider including a disclaimer to that effect.
- Ask your hospital, practice or other employer about its policy on blogging.
- Advise readers that you are not offering medical advice. If readers ask for a diagnosis, tell them to consult their physicians.
- Don't insult another doctor or patient. Don't type anything you wouldn't say in person.

The newsletter often includes interesting brief stories about the business of medicine in the US - you can subscribe for weekly email updates or add one or more of their RSS feeds (listed here).

Friday chuckle: Venn diagram humor

Today BoingBoing linked to a collection of Venn and other relational diagrams drawn on index cards -- they're posted on a blog called Indexed by Jessica Hagy.

Two of my favorites, probably because they strike close to home... "Those silly 1980s" which plots the growth of the hole in the ozone layer against the desired height of bangs, and "As the tenth bullet states" which plots the number of PowerPoint slides against the number of sleeping audience members.

Happy New Year - food post

Seems like good timing for a couple of interesting food links.

Boing Boing links to a wiseGEEK post that includes photos of 200 calories worth of different kinds of food. Interesting way to put consumption into perspective...

David Ng of the World's Fair takes a look at another site, Peter Menzel Photography that posts photos of families from different countries and the amount of food they consume on a weekly basis. Peter Menzel and co-author Faith D'Aluisio have published a whole collection of these photos and the stories behind them in a book, Hungry Planet: What the World Eats. Striking.

And finally, a funny anecdote about why everything ends up tasting like dirt if you store a certain bacterial culture plate in your home refrigerator.