Too many poorly conducted studies
For a number of years now, the scientific community has agreed that vaping is less harmful than smoking. However, although vapor from e-cigarettes contains far fewer harmful substances than tobacco cigarette smoke, it still contains some toxic compounds, in particular carbonyls, nitrosamines and metal compounds. Since some studies have already highlighted the presence of metal compounds in e-liquids, albeit at trace levels, it is hardly surprising that the vapor made by vaping also contains these. Given that some e-cigarette components are themselves made of metal, this increases the likelihood of detecting it in vapor.
A study to check the methodology used in other scientific work.
Scientists have been studying the subject of metals in vapor from personal vaping devices since e-cigarettes first came onto the market. However, as often happens when lots of studies tackle a subject, the results vary from one study to another.
Last month, researchers Sébastien Soulet (Ingésciences, France) and Roberto A. Sussman (Institute of Nuclear Sciences, Mexico) published their review of the scientific literature (1) on this subject. These experts examined the results of 12 laboratory studies published since 2017. Their aim was to focus on the “consistency between experimental design, real-life e-cigarette use and an appropriate assessment of the risks of exposure“.
Of the studies they examined, 9 had been carried out by universities and government institutions in the USA, 1 in China and 1 in France. In fact, just one had been financed by the vaping industry.
After careful analysis of the studies they reviewed, and the methodology used in those studies, the researchers observed that all of the studies had relied on highly specific puff protocols known as CORESTA (the French Cooperation Center for Scientific Research Relative to Tobacco), or standard CORESTA. Soulet and Sussman point out that, because this method is “incompatible with the considerable airflow and the high power required to operate sub-ohm devices, it is unsurprising that increased levels of certain metals (nickel, lead, copper, manganese) were found, particularly at a device’s maximum power, thus exceeding the strict toxicological references applicable to the general population“.
So, these results are unrealistic because they are produced during experiments that use protocols which are incompatible with the actual use of the devices being tested.
However, results obtained using the CORESTA method are realistic because this method is used with pod type e-cigarettes and starter kits (which are designed to operate at low power, see editor’s note). In these studies, the authors observed that the levels of metal in the aerosols were “far lower than the strict toxicological markers in all internally consistent laboratory tests“.
Some tests are irrelevant for assessing the risks to users’ health.
Finally, the 2 studies on emissions from low-powered e-cigarettes, whose results found levels of metal above normal, were “marked by methodological irregularities“. For example, the devices had been purchased 2 years prior to the study and no details were given about storage conditions. It is therefore possible that the devices in question had corroded. Furthermore, the researchers stated that they had tested the equipment with unflavoured, nicotine-free e-liquids. However, the devices tested included a Juul pod and a Vapor4Life pod, which do not come in unflavoured and/or nicotine-free versions.
Not to mention the fact that among other irregularities, some calculations were stated in the abstract of the study (see editor’s note) but did not appear anywhere else in the rest of the study.
Sébastien Soulet and Roberto A. Sussman concluded that:
“Although laboratory tests under extreme conditions and with no relation to actual usage may be intrinsically, theoretically or practically interesting, they are irrelevant when it comes to assessing risks to users’ health“.
The reaction of Sébastien Soulet, co-author of the study and Ingésciences researcher
Reading this analysis raises a lot of questions. How do you explain such disparity between the methods used to undertake these studies? How do you explain the fact that multiple studies on the same subject reach entirely different conclusions? We contacted Dr Sébastien Soulet, co-author of the review, to ask his opinion.
Every researcher studying e-cigarettes can define “inhalation” however they want.
For him, there are numerous issues. If the vape industryis suffering” from scientific interest, he explains, it is due to the lack of organization in the industry itself. In reality, there is no single scientific body to unify all of the stakeholders, which would have the power to “contradict poor studies“.
The industry is also a victim of the lack ofstandards“. The current absence of any industry standards means that any researcher studying e-cigarettes can “define ‘inhalation’ (referred to as the vaping regime in the lab” however they want,” according to Soulet. He gave the following example:
Every study says it has analyzed a high-powered sub ohm device, but some will have done this with greater consistency of use than others.
“A study of degradation products in the aerosol from a sub-ohm device at 200W (even though the device is recommended for use from 40 to 80W) may estimate that the user is inhaling 1000 µg of a molecule in a volume of 55ml (approximate volume in MTL). A similar study, carried out by a different researcher, may perform the same study but relate this quantity to a volume of 500ml per puff (approximate volume in DL).
Add to that the fact that they are not studying just one chemical, and you get a repeated error that leads to an overestimation of the actual risks.
In the first example, this will give a concentration of exposure to this degradation chemical that is 10 times higher than the concentration in the second example, because the puff volume being studied is 10 times smaller. And yet, every study says it has analyzed a high-powered sub ohm device, but some will have done this with greater consistency of use than others.
Add to that the fact that they are not studying just one chemical, and you get a repeated error that leads to an overestimation of the actual risks. Finally, remember that the power levels used during testing far exceed the recommended values. That seriously calls into question the relevance of any conclusions about toxicological analysis obtained in these conditions. This might seem exaggerated, but I’ve taken this example from one of the studies we examined“.
This raises another question: when can we expect standards to be brought in? According to Sébastien Soulet, who is also an expert on vaping products for AFNOR, CEN and ISO, some standards may be brought in next year, specifically “a standard required for analyzing DL devices“. Even if there is no way of enforcing the application of such standards, at least they will exist and will serve asguidance”, making it easier to contradict any research that fails to adhere to them.
There is also research that is just unfinished. As the doctor says, a study is supposed to “provide all information required for it to be reproduced” by peers. However, some do not.
It is crucial that studies whose toxicological analysis is based on conditions inconsistent with actual use are not cited and not relied on.
Finally, when it we ask why non-specialist media ignore positive studies on vaping but fall over themselves to put alarmist results on the front page, Soulet is certain it’s down to negative news selling better. Again, he feels the industry needsto learn to recognize how such studies are biased, to get organized and conduct coherent studies and, above all, to spread the news of good studies“.
Roberto A Sussman adds, “some of these studies are widely cited and used by public authorities“. He feels thatit is crucial that studies whose toxicological analysis is based on conditions inconsistent with actual use are not cited and not relied on“.
(1) Soulet, S.; Sussman, RA A Critical Review of Recent Literature on Metal Contents in E-Cigarette Aerosol. Toxics 2022, 10, 510. https://doi.org/10.3390/toxics10090510