Cite as:

Nassim Nicholas Taleb, Rupert Read, Raphael Douady, Joseph Norman, and Yaneer Bar-Yam, Response to review by Trevor Charles re: Precautionary Principle, New England Complex Systems Institute (November 28, 2014).

Download PDF

Abstract

A few days ago, Trevor Charles posted a review of our paper entitled "The Precautionary Principle (with Application to the Genetic Modification of Organisms)." Here we provide a response.

Thank you for the review of our paper. We will provide a point by point response below to your comments. Since you have focused on biological questions, it is important for us to emphasize that we did not perform a "statistical analysis" (which is inherently evidentiary and data based and anchored in biological experiments). Instead, we are engaged in a rigorous analysis of risk as it is derived from mathematical probability theory. Many of the citations you are asking for fall within the "carpenter fallacy" that we present in the text, i.e. that discussions about carpentry are not relevant to and distract from identifying the risks associated with gambling, even though the construction of a roulette wheel involves carpentry. Mathematical, probability-related arguments do not require biological citations. At the same time we have striven to explain how the biological context maps onto the risk analysis so that the connection between the two is more apparent to those who are focused on biology. For this reason we are providing the responses below. As a general comment, it would be very helpful for biologists who are contemplating or engaging in engineering strategies to read about the failures of systems engineering discussed in the text (Section VIII). This should lead to a better understanding about why the issue is not biological per se, but about the nature of engineering complex systems in cases that carry high potential harm, as has been found in the modernization of the Air Traffic Control system, for example. Reading that discussion should establish a better context for a conversation about the risks in biological engineering.

T. Charles: 1. In Section IV D. Distinguishing Global and Local Risks, it would be very helpful to provide an example or citation to illustrate how scientific analysis is able to distinguish global or local risks, and how it is determined whether global harm is probable. Are their [sic] examples of mechanisms that could be involved?

The utility of analyzing the structural connectivity of a system in order to identify the potential for non-local harm and cascading failure has been demonstrated in a variety of systems, including economic systems (e.g. [1] ), power/energy systems (e.g. [2]), and disease spreading in social systems (e.g. [3]). Biological systems that are organized in ecological networks have many interacting and mutually-supporting entities and behaviors, implying the potential for cascading failure. Human activity links distant parts of the globe, and therefore previously distant ecologies. We now are directly introducing, within short timeframes, genetically-modified organisms worldwide, creating a globally interconnected system whose lack of boundaries make globally unbounded cascades possible.

The mechanisms in each system and situation may be distinct: for example, in an economic system the behavioral state of panic may spread or bankruptcies may cause unrecoverable debts causing other bankruptcies; in energy grids, surges of power may overload relay-stations, causing an increase in surges and the cascade failure of more stations; in disease spreading, infectious agents transmit through populations of interacting individuals. In each case the specific mechanisms are different, but the pattern is the same: harm spreads through connected components. In order to limit the spread, and therefore prevent global failure, boundaries must be present that limit the spread to a "local" set of system components.

References:

[1] D. Harmon, B. Stacey, Yavni Bar-Yam, and Yaneer Bar-Yam, Networks of Economic Market Interdependence and Systemic Risk. arXiv:1011.3707v2, November 16, 2010.

[2] I. Dobson, et al. "Complex systems analysis of series of blackouts: Cascading failure, critical points, and self-organization." Chaos: An Interdisciplinary Journal of Nonlinear Science 17.2 (2007): 026103.

[3] R. Pastor-Satorras, and A. Vespignani. "Epidemic dynamics and endemic states in complex networks." Physical Review E 63.6 (2001): 066117.

T Charles: 2. In Section X B., GMOs are introduced as being the subject of debate. A definition of GMO should be provided here. Does the definition include transgenic, cisgenic, genome engineering, protoplast fusion, induced polyploidy, interspecific hybridization and/or mutation breeding? Is their justification for what is included and what is excluded under the moniker of "GMO"?

Our approach is to categorize different mechanisms based on the probabilistic structure and the multi-period dynamics of the process, not specific physical descriptions of chemical or biological mechanisms. The key distinction is the difference between non-recursive engineering (top down) approaches, versus incremental evolutionary-recursive (bottom up) approaches. For this purpose, we consider GMOs to be organisms whose genetic structure has been modified in a targeted way in order to evoke some intended effect in the organism (i.e. products of "genetic engineering"). This differs from breeding, as breeding affects genetic structure indirectly through selection of traits, while genetic engineering targets specific genetic structures.

There is no doubt that all techniques of genetic modification carry some potential for inducing unintended effects, however not all techniques carry the same degree of uncertainty. Our focus is on the relationship between uncertainty and risk. A report from the National Academy of Sciences on the potential health risks associated with GM foods (Safety of Genetically Engineered Foods: Approaches to Assessing Unintended Health Effects (2004)) ranked genetic modification techniques (both conventional and engineered approaches) according to their likelihood of producing unintended consequences. In order from least to most likely, their consensus opinion is as follows: Selection from a homogenous population, Selection from a heterogeneous population, Crossing of existing approved plant varieties, Agobacterium transfer of rDNA from closely related species, Conventional pollen-based crossing of closely related species, Conventional pollen-based crossing of distantly related species, Somatic hybridization, Somaclonal variation, Biolistic transfer of rDNA from closely related species, Agrobacterium transfer of rDNA from distantly related species, Biolistic transfer of rDNA from distantly related species, and Mutation breeding (chemical mutagenesis, ionizing radiation).

The NAS list was stated to be generated with unintended genetic effects in mind, and it is not clear that the estimates and rank-order would be the same with respect to unintended organismal or ecological effects. Thus, for example, inserting functional genes from distantly related species quite generally has an immediate effect on gene expression products and regulatory networks, while point mutations (by mutagenesis and radiation) have an immediate genetic effect but not necessarily an immediate expression or regulatory effect. Since our concern is more focused on higher levels of organization, including organismal and ecological ones, we may not agree with their estimates and precise order of risks. We note, however, that three out of the four highest levels of risk that are identified by the NAS study are consistent with generally accepted definitions of genetic engineering. The only other one that comes close to the same level of concern in the NAS report is that of mutagenesis. It is clear that at some level of genetic change by mutagenesis, risks are sufficient to warrant concern. However, there are different regimes that are possible. The general consequence of mutagenesis is a set of point mutations and thus the overall (Hamming) genetic distance of the organism from its original form is not as high as, and more likely to have been created in natural mutation processes than, typical engineered varieties. Higher levels of mutagenesis may result in poor organismal function before it results in viable organisms with harmful consequences. Engineering can prevent the plant from becoming non-viable or harmful in obvious ways while inducing changes that cause unanticipated risks. Our focus is on the risks of engineering, precisely because they are not apparent despite reductionist analytic information.

While we might continue to discuss the relative risk of mutagenesis and other molecular biological mechanisms, what is clear is that there are multiple means of modifying genetic structures that do not all carry the same risk of unintended consequences, and that conventional breeding techniques carry much less uncertainty than do engineering approaches.

T Charles: 3. A citation should be provided to support the statement that "GMOs have the propensity to spread uncontrollably".

The key aspects of the system connectivity discussed in response to point 1 (above) are the main concern. Still, evidence for spreading has been documented. Please see this report:

Bauer-Panskus et al., Transgene Escape: Global atlas of uncontrolled spread of genetically engineered plants. (2013).

and references therein.

T. Charles: 4. Adverse human health effects linked to the process of genetic modification have never been reported. Human feeding studies lack a hypothesis to justify.

Unfortunately, the FDA does not evaluate the safety of GMOs nor does it require empirical tests of either health or environmental impacts. It only provides general guidance (not requirements) about questions that might be addressed by corporations developing such organisms, and these questions are claimed to be addressable without empirical studies of either toxicity or environmental impact:

FDA Statement of Policy: Foods Derived From New Plant Varieties

That biologists accept this approach is quite surprising given the strong focus on empiricism in biological science. The agriculture industry has advocated for and achieved a situation in which scientific evaluations of GMOs independent of the industry are not part of the FDA process. It is quite apparent that the defense that testing is not needed is a weak argument, rejected by the public. Independent scientific analysis is needed.

We note that the same argument would suggest that drugs do not need to be tested because chemical modifications occur in nature. So by this logic all chemicals can be trusted along with the corporations that make them to infer that they are OK for us to consume. Similarly, mechanics is well understood, so automobiles do not need to be tested because car manufacturers know how to analyze the safety of cars.

Counter to the statement that there is no hypothesis to justify studies, there are multiple such hypotheses including:

a) Evidence of toxic substances in maternal milk and blood due to GMOs [1]. The first expected mechanism for this is that GMOs bring to agriculture much higher levels of directly applied pesticide. Note that some of this issue has nothing to do with the toxicity of the plant itself, but rather the changes in agricultural practice that these plants are designed for. The original "roundup ready" GMO corn and soy are designed to be robust to exposure to Roundup (Glyphosate), enabling higher levels of exposure of the plant. This higher exposure may be expected to lead to higher concentrations incorporated into the food itself. Glyphosate is assigned an EPA Toxicity Class of III (Caution, possibly followed by: "Harmful if swallowed", "May be harmful if absorbed through the skin", "May be harmful if inhaled", or "May irritate eyes, nose, throat, and skin"). That agricultural practices are affected by GMOs, and therefore the ultimate food that is eaten, is part of the large set of impacts they have. Note that here the debate is not about the specific toxicity or the level of incorporation or the specific agricultural practices, but that a hypothesis exists for testing.

b) The incorporation of pesticides by genetic modification into the plant themselves, e.g. BT Toxin. We note that BT Toxin has been incorporated in corn produced by Monsanto, including sweet corn that is directly consumed by people. In this case, the genetic modification does not create resistance to pesticide, but rather incorporates a pesticide directly into the plant cells and into food. BT Toxin has been subsequently identified in human blood (See [1]). Note that the question of whether pesticides are safe for human consumption is a clearly defined biological issue, as the hypothesis that organismal similarity leads to similarity of impact is well established, and thus the question of whether a pesticide is harmful for human beings is also a well established hypothesis requiring testing. Moreover, there is scientific evidence for the toxicity of BT Toxin for human beings, and the incorporated version of BT Toxin is not the same as that of the natural BT Toxin and the relative toxicity due to the difference is not tested. Thus, there is both a general hypothesis that justifies the need for testing of pesticide transgenic insertion, and a more specific hypothesis relevant to the specific incorporation of BT Toxin, as well as its variants.

c) More generally, the possibility that specific proteins being introduced into plants by genetic modification may give rise to adverse health effects should surely not be dismissed.

d) Beyond the specific protein properties, genetic insertions may have impacts on the regulatory network of cell biology. Cancer is one of the well known outcomes of regulatory network changes but so are a wide range of cell physiological and metabolic changes.

Moreover, there do exist empirical studies that provide evidence of harm from GMOs. That such studies are regularly dismissed by advocates is to be expected on economic grounds and is not a basis for scientific evaluation of those studies.

We will add additional notes to clarify this matter in the paper per your suggestion.

These points, however, miss the fundamental statement that evidence of harm is not the only, nor even the most important, thing to look for when assessing GMO risk. What is more important than evidence of harm? The underlying characteristics of actions that contain a risk of ruin for which evidence of harm does not yet exist. For these actions, the burden of proof should be on the absence of harm. In other words: The point of the PP is that we do not need evidence of harm. If it is assumed that in the absence of evidence of harm one has evidence of absence of harm, then one misses the reason why the PP is relevant.

Reference:

[1] A. Aris, S. Leblanc, Maternal and fetal exposure to pesticides associated to genetically modified foods in Eastern Townships of Quebec, Canada, Reproductive Toxicology, 31, 4, 528-533 (2011).

T. Charles 5. The statement "The widespread impacts of GMOs on ecologies and human health imply they are the domain of the PP." requires supporting citations.

In isolation this statement may be interpreted to point to prior evidence of impacts, but in context it is a summary of our probabilistic statement about the context for impact. It summarizes the section that describes pervasive GMO consumption in foods and their pervasive use in agriculture and its spreading across ecologies. The former is cited in the text (Ref. 11), and the latter is cited above in point 3. The summary statement reflects the position of our paper that policy makers should be aware of pervasive impacts from a probabilistic perspective. Here is the more complete quote:

"Genetically Modified Organisms (GMOs) and their risk are currently the subject of debate [9]. Here we argue that they fall squarely under the PP because their risk is systemic. There are two aspects of systemic risk, the widespread impact on the ecosystem and the widespread impact on health.

Ecologically, in addition to intentional cultivation, GMOs have the propensity to spread uncontrollably, and thus their risks cannot be localized. The cross-breeding of wild-type plants with genetically modified ones prevents their disentangling, leading to irreversible system-wide effects with unknown downsides. The ecological implications of releasing modified organisms into the wild are not tested empirically before release.

Health wise, the modification of crops impacts everyone. Corn, one of the primary GMO crops, is not only eaten fresh or as cereals, but is also a major component of processed foods in the form of high-fructose corn syrup, corn oil, corn starch and corn meal. In 2014 in the US almost 90% of corn and 94% of soybeans are GMO [11]. Foods derived from GMOs are not tested in humans before they are marketed.

The widespread impacts of GMOs on ecologies and human health imply they are in the domain of the PP. …"

T. Charles: 6. In Section X B., and illustrated in Figure 8, monoculture is conflated with GMO, when they are in fact independent of one another. Evidence needs to be provided to support any links between use of GMO methods and monoculture. It is well known that monoculture can be, and is, practiced without GMO.

We did not conflate them, we stated explicitly that these are two distinct aspects that contribute to the systemic risk associated with GMOs. To make this point clear in the text we listed them as follows:

“The systemic global impacts of GMOs arise from a combination of (1) engineered genetic modifications, (2) monoculture—the use of single crops over large areas."

That GMO crops are used in monoculture is an aspect of their use that dramatically increases the risk associated with the use of a particular GMO crop. As we argue in the paper, the localization of variations rather than global incorporation of specific genetic modifications is a key aspect of the distinction between evolutionary dynamics and engineering approaches.

While monoculture can be practiced independently of GMO and indeed caries risks that are of systemic concern in breeding, GMO solutions have been a key aspect of the promotion of modern monoculture. When molecular geneticists consider the process of genetic modification itself, they may not consider the agricultural mechanisms associated with them. We considered both in our paper.

Our analysis suggests that beyond the case against GMOs, movements to replace monoculture with higher diversity alternatives should be pursued. This is often advocated in other policy discussions.

T. Charles: 7. Invasive species, by definition, enter ecosystems that have evolved in their absence. It is therefore puzzling that the authors suggest that "long term evolutionary testing of harmful impacts of organisms on local ecological systems mitigates if not eliminates the largest possible risks." This statement needs clarification.

It may be expected that species do not devastate ecosystems in which they evolved, as well as ecosystems that have been exposed to them during their evolution even if they are not widely present there, i.e. ecosystems within dispersal range of the ecosystems they are part of. Almost by definition, this makes them not a global risk of devastation. They will also not devastate ecosystems that are sufficiently similar to those ecosystems in which they play sufficiently similar roles.

The point that is cited comes in the context of the analysis of the contribution of global monoculture due to breeding. We accept that global monoculture is a risk, as is widely recognized. Our statement is pointing out the relatively higher level of risk associated with GMOs in the context of global monoculture. At least there exists ecosystems in which traditionally bred organisms have been tested. What makes invasive species potentially dangerous is that they are transported out of their evolved context and into another. For GMOs, all contexts are foreign in this sense as their construction process bypassed the normal coevolutionary context that organisms naturally arise in. While invasive species may cause ecological damage by entering systems that evolved in their absence, all ecosystems have evolved in the absence of GMOs. It is precisely this feature, evolving within an ecological context, that GMOs lack.

In this regard it is good to recognize that US regulations limit importing of foreign food and agricultural products. California laws are even more strict due to the economic importance of their agricultural system. Both sets of laws arise from a recognition of the vulnerability of the agricultural system and its ecology and natural ecological context. However, the GMO regulations are remarkably different in allowing for real world experimentation on a large scale without prior empirical testing.

T. Charles: 8. Breeding involving GMOs is compared with "selective farming our ancestors have been doing for generations", when it should more properly be compared to conventional breeding programs that do not implement GMO methods.

One of the central points that is often made by those who advocate on behalf of GMOs is that such genetic modification is not essentially different from other types of genetic modification that occur in nature. According to this view anything that happens in nature is fine. Counter to this, many plants in nature are highly toxic. Unlike the experience of buying food in a supermarket, eating random plants in the wild will generally result in illness or death. Even plants that are closely related to plants that are eaten may be lethal. Even parts of the same plant we eat or parts that are harvested at the wrong time are toxic. Thus, for example, tomatoes and potatoes are part of the often-toxic nightshade family. Many wild potatoes are toxic to humans, and even commercial varieties can be toxic under some conditions. One of the main purposes of genetic modification is because agricultural varieties are vulnerable to pests. A key reason that plants we eat are vulnerable to pests is that we have bred them to eliminate toxins that would otherwise protect them from the pests, because those toxins are also harmful to human beings.

We use the term "selective farming" to describe the role of human selection in determining crops used in agriculture for food, part of horticulture but distinct from other non-food aspects. It is part of conventional breeding and not different from it. Thus our comparison of GMOs with selective farming is not about the distinction of farming from breeding. It is different from breeding in general in that it is focused on food, unlike floriculture or the breeding of dogs or cats. As such it is concerned with the safety, nutrition and taste of foods. The breeding process of selective farming is precisely that of choosing plants that are good as food. The key point that we are making here is that many plants that might be bred in general can also be toxic, as the large number of toxic plants that are closely related to foods show. To argue that breeding does not give rise to toxic varieties is not correct, unless they are subject to further evaluation of their viability as foods, i.e. selective farming related to the use of crops as food.

T. Charles: 9. An example is provided, of putting a fish gene in a tomato, and suggesting that such a process is outside of natural selection. The implication is that non-GMO crop breeding methods are subject to natural selection, when this has never been true throughout the history of crop breeding, as, by definition, the selection has been carried out by humans. Crops are not bred, nor cultivated, within natural systems.

The issue of natural versus artificial is often the subject of the GMO debate. Under what conditions does GMO modification constitute a natural modification? Under what conditions is conventional crop modification natural or unnatural? GMO advocates claim GMOs are natural in that there are transgenic transfers in nature, despite the manifest statement that GMOs are artificial because people are making the changes. Opposers say GMOs are not natural and that conventional breeding is natural. Some would like to claim that if there is opposition to unnatural GMOs we should go back to crops that are actually found in nature and reject breeding.

Our objection is different. We are not opposed to human involvement based upon its being artificial. We are opposed to the traditional engineering strategy applied to biological organisms versus the gradual selection that is characteristic of either natural selection or breeding. It is the mechanism of bottom up, small variation processes with extensive real world testing of incremental changes that distinguishes evolutionary dynamics from traditional engineering approaches. We made every effort to be clear about the distinction between engineering and evolutionary approaches to genetic modification, not because they occur in nature or by human action, but because of the difference in their mechanisms. We also included in the paper the example of systems engineering (e.g. of computer software systems) to illustrate the same distinction in the engineering context, where new approaches incorporate evolutionary dynamics to develop software systems so that they don’t fail at complex engineering tasks.

The distinct processes of top down and bottom up effects cannot be distinguished by the components of the system. Either can give rise to point mutations or transgenic transfers. What is different about them is whether the resulting systems have undergone the process of selection and testing. The evolutionary dynamics (whether natural or artificial) result in a process in which it is more difficult to cause changes that are larger in function and have them be widely adopted. This is precisely what current genetic engineering is trying to get around, but this is precisely what ensures safety.

When a bottom up approach is used by human beings, small changes occur that are consistent with the range of genetic variations that are present in nature. The process of their selection is not part of nature, but the varieties selected begin from those that have been generated by evolution with natural selection prior to their selection by human beings. We are not focused on the specific variations that occur, even if they deviate from those that arise in nature; the gradual nature of the changes is the process that is desired. This is important for their consistency with natural ecologies and human health; the latter has to be determined by testing on human beings because natural selection does not distinguish safety for human health as it does for ecological health.

No amount of human selection will put a fish gene in a tomato!!

T. Charles: 10. It is stated that "pesticide resistant crops are subject to increased use of pesticides, which are subsequently present in the plant in larger concentrations and cannot be washed away." This assertion requires a citation. Available data indicates the opposite of what is stated (Klümper W, Qaim M. 2014. A Meta-Analysis of the Impacts of Genetically Modified Crops. PLoS ONE 9:e111629.)

Roundup Ready GMO crops are designed for increased direct pesticide use on the agricultural crops (that is their purpose). Other GMOs (BT Toxin) have pesticides within them and this reduces the need for pesticides to be applied. Reduced external application of pesticides in this case is traded off against increasing the pesticides in the food itself. Having more pesticides out of the plant in one case, and more pesticides inside the plant in the other, both correspond to more pesticides in the final food product. We will clarify this point in the paper.

The article you cite considers an entirely different question which is the level of pesticide use in general, i.e. not just that directly applied to a crop. There are a wide variety of effects of GMO crops on the use of pesticides. Some of them change the total amount but increase the amount applied directly. The trend has become toward increased pesticide use even in this case because of the natural evolution of pesticide resistance in the weeds. Here is a relevant reference:

S. B. Powles, Evolved glyphosate‐resistant weeds around the world: lessons to be learnt, Pest management science 64.4 360-365 (2008).

T. Charles: 11. If PP is applied to GMO plants, owing to a risk of ruin, then should not PP also apply in a similar manner to non-plant GMOs? If not, why not?

Yes, e.g. GM fish [1-2].

References:

[1] Darek T. R. Moreau, Corinne Conway, Ian A. Fleming. (2011). Reproductive performance of alternative male phenotypes of growth hormone transgenic Atlantic salmon (Salmo salar). Evolutionary Applications, Blackwell Publishing, Ltd.

[2] William Muir et al., Possible ecological risks of transgenic organism release when transgenes affect mating success: Sexual selection and the Trojan gene hypothesis, 96 PNAS 13853-13856, at 13853 (Nov. 23, 1999).