A negative result! And this coming from acupuncturists – not something that happens very often. And what’s more, it’s published in a very prestigious journal, the ‘Journal of the American Medical Association’ (JAMA), impact factor = 44. So, all I can say is, wow, did not see this one coming. Because usually they will spin the result into a positive using various techniques and various media platforms, and yet, here we have a very clear negative, even their (social) media platforms proudly proclaims; “Fertility study finds acupuncture ineffective for IVF birth rates”. Sure, they still tried their best to give it some sort of positive spin by stating:
“We examined the effects of a short course of acupuncture administered during an IVF cycle….. However, in clinical practice, acupuncture may include more sessions prior to an IVF cycle starting.”
“Stress is thought to play a role in infertility…..In our earlier research, acupuncture was shown to reduce the emotional stress and burden experienced by women during IVF treatment.”
Conflicts of interests
Now I have some history with this project. Back in 2012 when the NHMRC announced that they have approved $630 000 dollars for this study, it was promptly called “universities in a wacky waste of cash” in the media. Why? Because even back then acupuncture was known to be ineffective for IVF (and pretty much everything else) so why spend so much money which could have been spent on doing useful research, on something that is known not to work? Well, if you can mislead people into using acupuncture and all sorts of other ineffective remedies, then surely, you’ll be able to fool funding bodies as well. That is after all their job – to fool people, that is what promotional researchers do!
But I did notice a couple of years ago that the ‘academics’ (Prof Caroline Smith and Alan Bensoussan) from the National Institute of Complementary Medicine (NICM) almost never declare their conflicts of interests. In other words, they received some sort of financial incentive (cash or in-kind) from acupuncture clinics which went undeclared, in clear violation of research ethics. They also failed to declare their conflicts of interests when they published their original trial design back in 2012 for this current acupuncture IVF study. After highlighting this issue with the journal, Trials, they eventually published a correction (erratum) in 2017 which simply state that the authors did not receive any financial compensation. Sure, she did not get any payments into her personal bank account but the NICM did receive substantial donations (evidence was send to the journal, but yes, what can I say, scientific journals nowadays, pfff). You can read more about it here, here and here.
Moving the goalposts!
But overall, publishing a negative result is so unlike the NICM, the winners of the Bent Spoon award for quackery in 2017. Or is there more to this than meets the eye? Indeed, there is something fishy and it is strange that the reviewers of such a prestigious journal did not pick up on this. To explain the issue let’s have a look at the abstract (my underlining).
Importance: Acupuncture is widely used by women undergoing in vitro fertilization (IVF), although the evidence for efficacy is conflicting.
Objective: To determine the efficacy of acupuncture compared with a sham acupuncture control performed during IVF on live births.
Design, Setting, and Participants: A single-blind, parallel-group randomized clinical trial including 848 women undergoing a fresh IVF cycle was conducted at 16 IVF centers in Australia and New Zealand between June 29, 2011, and October 23, 2015, with 10 months of pregnancy follow-up until August 2016.
Interventions: Women received either acupuncture (n = 424) or a sham acupuncture control (n = 424). The first treatment was administered between days 6 to 8 of follicle stimulation, and 2 treatments were administered prior to and following embryo transfer. The sham control used a noninvasive needle placed away from the true acupuncture points.
Main Outcomes and Measures: The primary outcome was live birth, defined as the delivery of 1 or more living infants at greater than 20 weeks’ gestation or birth weight of at least 400 g.
Results: Among 848 randomized women, 24 withdrew consent, 824 were included in the study (mean [SD] age, 35.4 [4.3] years); 371 [45.0%] had undergone more than 2 previous IVF cycles), 607 proceeded to an embryo transfer, and 809 (98.2%) had data available on live birth outcomes. Live births occurred among 74 of 405 women (18.3%) receiving acupuncture compared with 72 of 404 women (17.8%) receiving sham control (risk difference, 0.5% [95% CI, −4.9% to 5.8%]; relative risk, 1.02 [95% CI, 0.76 to 1.38]).
Conclusions and Relevance: Among women undergoing IVF, administration of acupuncture vs sham acupuncture at the time of ovarian stimulation and embryo transfer resulted in no significant difference in live birth rates. These findings do not support the use of acupuncture to improve the rate of live births among women undergoing IVF.
So, clearly this study was conducted in order to see if acupuncture is effective and they found it to be ineffective – or at least that is what they want us to think. This negative result was also widely covered in newspapers around the world and yet almost all of them got it wrong. Here are a couple of examples (my underlining):
“Having acupuncture to increase IVF chances might be waste of time, study suggests” ABC news (Aus)
“Acupuncture no better than placebo for improving IVF success, trial finds” The Independent (UK)
“Study finds no evidence acupuncture boosts fertility treatment” Chicago Tribune (US)
So, what did they all get wrong? There are a number of issues with these results, not with the results per se, but with the results that they did not publish. So, I decided to write an email to the authors (and the journal editor) asking them a number of questions (this email was also undersigned by Prof Edzard Ernst). This email should explain the issue at hand. Here it is:
Dear Prof Smith et al.,
Congratulations to you and your team on the publication ‘Effect of acupuncture vs sham acupuncture on live births among women undergoing IVF’ in JAMA recently.
You are probably aware that the outcome of this project has been widely reported and is currently being discussed on numerous blog sites (here and here are two examples). During these discussions a number of questions were raised and we were hoping that you, or any of your co-authors, can provide answers or some sort of explanation for these questions.
1.Why did it take so long after the completion of this study to publish the results.
2.There is a consensus that a trial of this nature would be far more expensive than the NHMRC’s funding of $630 000 – was there a lot of in-kind support or other sources of funding?
3.The live birth rate of around 18% reported in this study seems to be low when compared to the overall success rate of IVF. According to IVF Australia women between the age of 30.0-34.9 can expect a success rate of just above 35% while women in the age category 35.0-39.9 have a success rate of just above 25%. (On Repromed’s website, who co-authored this publication, similar success rates are reported). In your study the median age was 35.4 and 35.5 in the 2 groups, and yet, a success rate of around 18% was reported. If true, does this mean that acupuncture reduce your chances on IVF success?
4.Both the ANZCTR registry and your publication in Trials where the trial design was published, included a study group 3. This group was meant to receive only IVF and it was supposed to serve as a baseline comparison. This is of course a very important aspect and yet the results were not reported nor was it mentioned or discussed – could you clarify what the reason for this omission is?
5.In various newspaper reports it is mentioned that a further two publications will flow from these results. A cost effectiveness study and a paper on the psycho-social benefits of acupuncture. But when something is shown to be ineffective (as in this study) it cannot possibly be cost effective and when no 3rd group, receiving only IVF, was included in this study, how can the psycho-social benefits be determined?
We would appreciate any answers or comments. Thank you in advance.
Needles to say (pun intended), no response has been received – yet. Now if we carefully look at the design of this study you will notice that the original study had a different title and design. On the trial registry the title is;
“Acupuncture compared to sham acupuncture and standard care to improve live birth rates for women undergoing IVF: a randomised controlled trial”
In the journal Trials the title is;
”Acupuncture to improve live birth rates for women undergoing in vitro fertilization: a protocol for a randomized controlled trial”
So clearly this study had three groups including the important baseline comparison of women receiving only IVF – because this is the only way that you can determine if acupuncture actually improve pregnancy or live birth rates. This is what they wanted to determine, and they were in effect supposed to investigate two questions;
- Does acupuncture and/or sham acupuncture have a negative, neutral (no effect), or positive effect on IVF compared to IVF alone? – this is the important efficacy question.
- Does acupuncture work better/equivalent/worse than sham acupuncture? – this is a secondary question focusing on the existence of the non-existent chi (energy) that flows through non-existent meridians.
But now they have intentionally dropped the baseline comparison (group 3) and only compared acupuncture with sham acupuncture. Therefore the only conclusion that can be drawn from this study is that there is no difference between acupuncture and sham acupuncture (no big surprise because chi and meridians do not exist). Nothing can be said about the efficacy of acupuncture because they left this important information out and did not report on it, even though they misleadingly claim acupuncture to be ‘neutral’ (no effect).
And the newspapers?
Most journalists get their information from the university’s press release and this is what the NICM published “Fertility study finds acupuncture ineffective for IVF birth rates”. It is completely wrong, it should have been something like; “Chi and Meridians again shown not to exist and acupuncture might even reduce your chances on IVF success” So, the journalists did not get it wrong per se, they just reported what they were told, which is strange, because any good journalist would surely check their facts before publishing. So, the question now is why did they do it? Why did they move the goal posts? Is it possible that this publication is simply a smokescreen to hide the fact that acupuncture might have a negative impact (nocebo effect) on IVF success rates?
They have omitted it intentionally!
This is now where it gets interesting. Back in 2006 a similar study with 228 participants was published by the same lead author from the NICM where a discussion was included about the reasons why acupuncture (28%) and sham acupuncture (18%) resulted in lower pregnancy rates as compared with the clinic’s baseline pregnancy rate of 30% (primary outcome was pregnancy and not live births). Their current study with 848 participants published in 2018 had even lower pregnancy rates (live birth rates of around 18%) whereas the clinic’s baseline pregnancy rate/live birth rate has in all likelihood improved over the last 12 years (between 25-35%). So, just imagine a newspaper article stating that acupuncture might actually reduce your chances on IVF success. That would be a disaster for these people and the probable reason why they decided to keep quiet about it.
My opinion? The fact that infertility is a highly sensitive issue is simply ignored in order to protect acupuncture, and yes, they will spin this result into some sort of positive sometime in future. They have already started. Now, infertility can lead to broken relations, depression, and in extreme cases even suicide – so it is a very sensitive issue. If there is any suspicion that acupuncture might actually have a negative or even only a neutral influence on IVF then scientists should apply the ‘precautionary principle’ and advise people to stop using it. Promotional scientists on the other hand are well known to throw caution to the wind, and continue to try and convince vulnerable people to use their services or products. This is completely unethical. These people could not care less about the well-being of the public and hence they just dropped this important information from their publication and did not even bother to discuss it, let alone, warn people about it.
Because of the size of this project they were probably forced to publish something and it took this long because they needed to come up with a way that will cause the least amount of damage to acupuncture. That it was published in JAMA is of concern and one can only question how this got pass the peer review process. Maybe the reviewers were so overwhelmed by the fact that these folks are publishing a negative result that they forgot to review the manuscript. (I will follow up on these issues with the editors)
It would however be interesting to see if the acupuncture clinics who donated money to the NICM, such as “The Acupuncture Pregnancy Clinic” will now put these ‘negative’, albeit misleading, results on their website. But how will they spin it? Acupuncture is their main, if not only, source of income with some of it flowing back to the NICM. (just read their rubbish declaration of interest in the JAMA article to see how they are getting away with it).
Will keep you posted on any further developments, I’m sure there is a lot more to come.