The epistemology of OpenNeedle
Why we built OpenNeedle
You are told to trust your doctor, the journals, and the regulators. All three deserve some trust. None of them deserve all of it, and the numbers below show why. Every claim on this page is sourced.
The short version
Three findings, each documented in mainstream medical research itself:
- The published record is tilted. Positive results get published and negative ones go missing, and industry-funded studies favor the sponsor at measurable rates.
- Nobody can read it all. Keeping current with the medical literature is impossible for any working doctor, so everyone relies on summaries written by others.
- Checking is possible, just rare. The studies that do get checked sometimes overturn decades of standard practice. OpenNeedle is a tool for making checking routine.
The published record is tilted
In 2008, researchers obtained the FDA's internal reviews of 74 antidepressant trials. The FDA had judged 51% of them positive. In the journals, 94% looked positive. The gap is trials that were never published, or published with the conclusion reversed. The published versions also overstated the drugs' effect by about a third.6
Antidepressant trials, judged positive on their primary outcome
74 FDA-registered trials for 12 drugs, 1987–2004 (Turner et al., NEJM 2008)
A BMJ systematic review of 30 studies found that drug-company-funded trials were about four times as likely to report outcomes favoring the sponsor. The methods were no worse than independent research. What differed was the comparator chosen, the framing, and the conclusions.7
Independent replication tells the same story. Amgen's scientists tried to reproduce 53 landmark cancer papers before building drug programs on them. They could confirm 6. Bayer ran the same kind of internal check and confirmed about a quarter.5
Who funds the evidence
Guidelines are what doctors actually follow. In the 17 most recent American cardiology guidelines studied in 2011, 56% of the people who wrote them disclosed industry conflicts. In some individual guidelines it was 87%.8
The FDA runs partly on industry money. Under a law passed in 1992, the companies whose drugs the agency reviews help pay for the review program. Those fees were about $1.3 billion in fiscal 2024, roughly three quarters of the program's cost. In 1993 they covered 7%.910
None of this means every conflicted decision is wrong. It means "the FDA approved it" and "the guidelines recommend it" are weaker evidence than they sound, because the same funders are present at every step.
Nobody can read it all
In 2004, researchers counted the articles relevant to primary care published in one month: about 7,300. Evaluating them properly would take 627 hours a month, about 29 hours per weekday. No doctor can read all of that. Even the best ones rely on guidelines, review articles, and summaries, written by people who read other summaries. Everyone in that chain is competent, and the chain still ends in the tilted record described above.2
Keeping up with your own field, in hours
Estimated weekly reading load for a primary care physician (Alper et al., 2004)
What gets missed
Two cases where the standard advice was wrong, and someone checking the evidence found it.
In medicine, "no evidence of harm" usually means "no one ran the study." Those are different findings, and a lot of policy treats them as the same one.
What OpenNeedle does
When you ask a question, OpenNeedle searches a structured evidence layer, BGPT, that turns full papers into structured records. Up to 32 records per answer. Every claim links to its source, so you can check us instead of trusting us.15
Inside one evidence record
- The claim the paper makes
- The experiment behind it
- The exact results
- Who was studied, and how many
- What the study does not show
- What the authors listed as limitations
- How the paper could be falsified
- Who funded it, and where it appeared
The catch is almost always in the limitations section, which abstracts leave out.
We have biases too. We read the same literature everyone else does, with the flaws described above, and our judgment of that literature is our own. What we offer instead of neutrality is inspectability: we show the sources, we state how confident we are, and we say plainly when the evidence is thin or missing. Where a claim rests on manufacturer-funded studies, we say that too.
We are not anti-vaccine or anti-doctor. Some vaccines clearly work, and we will say so with numbers. Some doctors provide the best care available anywhere, and we will say that too. What we argue against is ending the conversation at the credential, whether the credential is a white coat, a journal, or an approval letter.
If you find an error on this page, the sources are all linked. Telling us is more useful to everyone than agreeing with us.
Sources
- Prymula R, et al. Effect of prophylactic paracetamol administration at time of vaccination on febrile reactions and antibody responses in children. Lancet. 2009. Link
- Alper BS, et al. How much effort is needed to keep up with the literature relevant for primary care? J Med Libr Assoc. 2004. Link
- Begley CG, Ellis LM. Raise standards for preclinical cancer research. Nature. 2012. Link
- Turner EH, et al. Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy. N Engl J Med. 2008. Link
- Lexchin J, et al. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003. Link
- Neuman J, et al. Conflicts of Interest in Cardiovascular Clinical Practice Guidelines. Arch Intern Med. 2011. Link
- FDA. Prescription Drug User Fee Rates for Fiscal Year 2024. Federal Register. Link
- Congressional Research Service. FDA's User Fee Programs (R44750). Link
- Testimony of David J. Graham, MD, MPH. US Senate Committee on Finance. Nov 18, 2004. Link
- BGPT, our evidence layer. Link
- NBC News. Painkiller makers still face headaches (FDA advisory committee votes on COX-2 drugs). Feb 2005. Link