Another call from another science to replicate experiments.

The gist for this post came from:
“Research, Report, Repeat“– an article from Discover Magazine (Yong 1/14).

The story interviews psychologist Brian Nosek, at the University of Virginia, who is at the forefront of the fight to make psychology more transparent. While you’re reading this, delete in your mind the word, “fraudulent” and put in its place “incomplete.” In place of “charlatan” put “traditionalist” in its place.

From the article:
“In 2011, Dutch social psychologist Diederik Stapel was revealed as a charlatan who had published dozens of fraudulent scientific papers. The shocking thing was that no one in his field had noticed until courageous students in his lab reported their suspicions. This spate of misconduct happened in tandem with many failed attempts to replicate some of the field’s classic results, prompting acolytes to question whether psychology was being polluted by quirky and attention-grabbing findings that might not actually be true. This issue applies to every field of science, from physics to medicine.“

The key point to the article is reproducibility.

Hypotheses, phylogenetic trees, reconstructions that fit footprints—all of these things in paleontology must be reproducible and tenable. Unfortunately, as I’ve reported here, replication often fails, typically due to an incomplete inclusion set. Sometimes, as in deep chord wing membrane papers, to wishful thinking.

We’ve seen this in phylogeny following attempts at nesting pterosaurs with archosaurs, at nesting mesosaurs with pareiasaurs, at nesting caseasaurs with pelycosaurs. All that was fine, but now that the repair is available (see there is less excuse for continuing phylogenetic errors based on incomplete inclusion sets.

The pterosaur heresies blog is here to reproduce and verify or falsify the work of other paleontologists. Just as other paleontologists are doing all the time.

Testing is key.
Did they include the proper inclusion set? Too often they did not. Did they find sisters that provide a tenable evolutionary path that provides a gradual accumulation of traits? Too often they did not. Did they invent excuses for soft tissue not behaving in situ according to their traditional hypotheses? Too often this is so. Did they really try to pull details out of the specimen or did they draw a crude circle around the crushed skull and call it a day?

all these were done with the best intentions following established procedure with no intention to defraud. Rarely, as in Bennett’s 2008 imaginary protopterosaur, hypotheses can appear that are completely imaginary with no basis in reality, no thought to untenable evolutionary scenarios, and no testing or review of prior published studies and specimens.

Back to the Discover article:
In 2011 Brian Nosek and colleagues launched the Reproducibility Project, in which a team of about 200 scientists are carrying out experiments to replicate findings published in psychology journals. They want to examine the problem of false-positive results.

Nosek reports, “There is an ethic that science is self-correcting, but replication is often a pain the in butt, and since scietists’ career success doesn’t depend on exactly replicating a study that’s already been published, they usually don’t do it. As a result, mistakes end up hanging around longer than they need to.”

The same can be said of paleontology.
Most of paleontology is on an even track. But there are still problems out there, like “modular evolution” and the exclusion of tiny (sparrow-sized) pterosaurs and fenestrasaurs from analyses.

Nosek built the “Open Science Framework” a web application where collaborating researchers can put all their data and research materials so anyone can easily see them. The same thing is happening for paleontology at and

Nosek concludes, “Replications are so rare that people perceive them to indicate a lack of trust. I want it to be very ordinary for people to say, “I’m not sure about this, so I’ll replicate it,” and for that to be a compliment, not a threat.”

I heartily agree.

19 thoughts on “Another call from another science to replicate experiments.

  1. This is why publishing your instructions in a cite-able (preferably peer reviewed) format is a vitally important. So people can say, “I did exactly these steps here (Peters, 2014) with the modification of…”. A blog is not able to be cited in virtually every journal being published today. You want other folks to use your techniques but you aren’t giving them the tool they need. It is all well and good that you put instructions on your blog but in 100 years neither you nor this blog will exist. How is anyone supposed to refer back to techniques used today unless there is a permanent archived record of those techniques?

  2. I’m not commenting on this post per-se, but on something you’ve blogged about a lot and mention in here as one of your problems with palaeontology: Modular Evolution

    You’ve quoted in the past in your blog as being something made up to explain a transitional form in pterosaurs, and “evolution does not work this way”, “There is no such thing as modular evolution.” I also seem to remember you mentioning on a blog post that modular evolution has never been suggested for anything other than Pterosaurs.

    To the contrary, modular evolution is not something that has been applied only to pterosaurs, it is not something that was made up to explain difficult fossils. It is a well established feature of evolution. It is supported not only by morphological data (for example,in the most recent SVP alone, modularity is discussed in four of the abstracts and is a major feature of two of them), but by molecular data (Moore et al 2008, Arrangements in the modular evolution of proteins; Mathiesen & Hägerhäll, The ‘antiporter module’of respiratory chain complex I includes the MrpC/NuoK subunit–a revision of the modular evolution scheme; Diaz et al. 1990, Chimeric phage-bacterial enzymes: a clue to the modular evolution of genes…I could go on), developmental data (Minelli 2003, The development of animal form: ontogeny, morphology, and evolution; Friedman & Williams 2003, Modularity of the angiosperm female gametophyte and its bearing on the early evolution of endosperm in flowering plants; Müller 2007, Evo–devo: extending the evolutionary synthesis…Again, I could go on) and in evolutionary simulation (Bongard 2002, Evolving modular genetic regulatory networks).

    Your objection to modular evolution is ridiculous. On the page you linked to you say “but “modular evolution” would play havoc with this, creating chimaeras with the head of one taxon on the torso of another”. Perfectly true. That is why identifying shoddy specimens is so difficult. There are pelenty of examples where different body parts have evolved at different rates and producing organisms rataining primitive characters in part of the body and derived characters in another. Take, for example, human evolution; it is well established now that the skull retained the ape-like characters for longer than the legs and pelvis: the legs formed a separate module due to the constraints of upright locomotion.

    (Were I in one of my snarkier moods, I might point out that simply typing “modular evolution” into google scholar, or even wikipedia, would have given you all the information on modular evolution you could ever need).

  3. Actually I did Google “modular evolution” and it came back only to the Darwinopterus work. Taking your example of human evolution: not quite true. Changes were happening in the skull and teeth at the same time as in the pelvis and feet so that anthropologists can identify australopithecines by their teeth different from other anthropoid apes. Right? Those other examples you cite appear to be gamete-based or DNA based, not morphological.

    • Of course changes were happening in the skull. The point of modularity is not that changes only occur in one module, its that they occur much slower in other modules. the point still stands: skull retained ape-like characters to a much greater extent than the pelvis and limbs, and retained them longer. The

      And I cited four morphological abstracts from one abstract volume alone. If you really want I can go and find more; there are plenty. The DNA based ones are identifying the mechanism behind the morphological change. The same goes for the developmental ones. Saying that its not morphological is a desperate kop out.

      The simple fact remains: Modular evolution is a well established principle in evolution, with multiple lines of evidence to back it up. To try and argue against it would require you to substantially overturn A LOT of studies one by one.

  4. Sounds like modular evolution is a judgment call in your definition… big changes here, unimportant changes there. But I think the changes in the teeth and occiput were just as important and necessary as the changes in the pelvis. It all goes together.

    On the other hand, the way Lü et al. explained it. all the changes in Darwinopterus were from the shoulders up. And then later all the changes hit the rest of the body to produce pterodactyloids. If I follow you, the shrinking of the tail in anurognathids is another example of modular evolution. But I would counter that such reduction was accompanied by other changes throughout the body, so that if you knew you had the front half of the anurognathid you could predict something about the tail. The same for Darwinopterus. Phylogenetic analysis demonstrates that the Darwinopterus lineage was sterile. Pterodactyloid-grade pterosaurs arose from other ancestors. So that means Darwinopterus, like anurognathids, was trying on pterodactyloid traits, but never achieved the full suite. That also means, if you know wukongopterids well, you can make predictions about missing parts, contra the modular evolution hypothesis.

    • I never said the changes in the skull were less important than the pelvis. I merely said they occurred at a slower rate. So stop trying to twist my words to support a viewpoint you made without doing the proper research. This isn’t about important and unimportant changes, its about rates; one area evolving faster or slower than others. Or possibly modes; one portion evolving by Brownian motion, so variance increases, one by Orstein Ulenbeck, leading to constant variance.

      No one, least of all me, has ever suggested that modular evolution is one part of the body stopping evolving while the other part continues evolving. evolution is constant. But different parts of the body are under different constraints and pressures, and so will evolve slower or faster.

      This is not a judgement call. I have provided almost a dozen references describing the evidence for and the mechanisms of modularity. If needs be I can provide more. It is supported by Palaeontological and neontological evidence, by morphological and molecular.

  5. Are there any other examples of modular evolution you can cite in vertebrates? Please provide the transitional taxon and sister taxa, both more primitive and more derived. I understand and appreciate allometric growth. Does modular evolution, to your mind, go beyond allometry?

    • “Please provide the transitional taxon and sister taxa, both more primitive and more derived.”
      I have heard essentially the same thing utter by creationists. Ever increasing demands for evidence when mountains already exist does no good. Modular evolution causes problems with fine-scale phylogenetic bracketing (such as you do with DGS) so I understand why you would not like the idea. In the end though what we like has nothing to do with what is actually right.

  6. Lu et al were able to provide this sequence. And it was for the first and only time. Your theory about fast vs slow evolution disposes with random mutation in that the slow evolution portion “catches up” to the faster evolution. All this is counterproductive in any case because analysis demonstrates that Darwinopterus was a terminal, not a transitional, taxon. So it had a long skull, small naris and long neck by convergence with pterodactyloid grade pterosaurs.

    • It astonishes me that you would say something like “Your theory about fast vs slow evolution disposes with random mutation”, as if it was questionable that there were changes in rate of evolution. To the contrary, it is something even you would find difficulty arguing with; GG simpson famously stated that one of the most important issues in evolutionary biology was changes in rates. Am I going to have to give you a plethora of literature about rates?

      And how on earth does changing rates of evolution dispose with random mutation? Even if the mutation rate was constant (it isn’t) evolution is all about differential survival rate in a population, not mutation rates in an individual; survival rate will change as selective pressures change. A stronger selective pressure on a particular part of the body e.g. during a new locomotor style, will increase the differentiation between what characters are needed to survive and what will lead to death, meaning that that portion of the body will evolve faster across the population.

      Anyway you asked for an example. A beautiful example of modular evolution in mammals (although they use the alternative name “mosaic evolution”) is Qiang et al. 1999: A Chinese triconodont mammal and mosaic evolution of the mammalian skeleton (Nature 398), wherein the mammal Jeholodens shows a scapula very similar to mammals as derived as marsupials, while the pelvis is more similar to cynodonts. As they say, the alternative to a modular model of evolution is several reversals of the pelvis to a plesiomorphic condition.

  7. Your mammal example, Jeholodens, nests as more derived than Megazostrodon and living monotremes, and the pelvis shows no sign of a short ilium largely dorsal to the ventral pelvis, as in non-mammalian cynodonts like Thrinaxodon. In counterpoint to your example, Megazostrodon had sprawling forelimbs and tucked in hind limbs, like the living Echinda. So, following the description of Jeholodens here: in which it is written: “In contrast to the advanced shoulder and forelimb, Jeholodens has a very primitive pelvis, a sprawling hind limb, and a splayed hind foot.” it seems you have an odd reversion at best. This is not an example of modular evolution, a transitional specimen in which the front part of the body was evolving to the derived condition faster than the rear of the body, as proposed in Darwinopterus.

    • This response is flawed in too many ways.

      Firstly the cynodont you’ve used for a comparison is a cynognathian, not even on the mammal line

      Secondly It is reasonably clear that the tucking of the hindlimbs in echidnas is a independent origin, like so many other monotreme feature absent in basal therians but present in more derived ones; they are a result of the return to the land from a platypus-like ancestor (Philips et al. 2009).

      The basalmost therian was almost certainly a sprawler in the hindlimbs, esp when you look at the Triconodonta (Kielan-Jaworowska et al. 2004, Kemp 2005); As such for this to be a reversal you have to posit multiple reversals; a single change of scapula first, pelvis second is much more parsimonious.

      But ok, lets have more examples. Barton & Harvey 2000 document brain evolution in mammals in which the neocortex substantially increases in size before locomotor-linked aspects (Mosaic evolution of brain structure in mammals, Nature 405)

      Australopithecus sebida demonstrates the evolution of a hand features for gripping tools, whilst retaining wrist features keeping arboreal locomotion rather than tool manipulation in mind (Kivell et al. 2011 Australopithecus sediba Hand Demonstrates Mosaic Evolution of Locomotor and Manipulative Abilities, Science 333)

      Finally, I will point out, given the plethora of references I have provided supporting modular evolution, giving these objections to single examples is reeking of desperation; You have yet to provide a single reference debunking modular evolution.

  8. Yes, animals evolve certain traits and then other traits. I’ve already published on the evolution of pterosaurs in which all the parts were there (sternal complex, pteroid, prepubis) before the wing finger appeared. Whales were aquatic before their hind limbs disappeared. What you need to show is the front half of the animal evolving traits, followed by the rear half of the animal evolving traits that complete the process, which is what Lü et al. proposed for Darwinopterus as a basal pterodactyloid. And, as mentioned earlier, Darwinopterus produced no know descendants. It was a false start. So, it’s not even a single example of “modular evolution.” Debunking modular evolution has already been done in the large pterosaur tree in which Darwinopterus is shown to NOT be the ancestor of pterodactyloids. With regard to Jeholodens and the angle of the femoral head and pelvis shape, please send images to argue your point. The data I’ve found does not demonstrate that this simple angle cannot be considered a “module” in the sense that Lü et al proposed. And if the pelvis shape is throwback, please provide an image of the pelvis or taxon that it more closely resembles, if not its current sisters.

    • The large pterosaur tree does not debunk modular evolution. Even if Darwinopterus is shown to have a more parsimonious position phylogenetic trees are hypotheses, not facts. Furthermore parsimony-based trees cannot show direct ancestor relationships. Finally, as an unpublished and unreviewed hypothesis, it is not accurate to think of a single blog post overturning literally hundreds if not thousands of publications pointing to the same explanation (modular/mosaic evolution).

    • The large pterosaur tree does NOT debunk modular evolution. Even if it is accurate, disproving a single example does not disprove an entire theory. Really, what a ridiculous notion, what a pathetic way of arguing. That’s like creationists using a single example of scientists changing their minds over a transitional sequence to argue there are no transitional fossils. Do you think Darwinopterus was the entire basis for the theory of modular evolution? There is literature going back far further than that on it. The position of Darwinopterus in the tree affects modular evolution not one jot

      You’ve found one, maybe two, examples of modular evolution which you consider flawed, and are saying from that that modular evolution doesn’t exist, in spite of the dozens of references I have provided that you have ignored, providing evidence from multiple facets of biology. You have just now tried to change the definition of modular evolution to suit your argument, yet another tactic commonly used by creationists.
      “What you need to show is the front half of the animal evolving traits, followed by the rear half of the animal evolving traits that complete the process”. NO. That is not what modular evolution is. That is just one example of what modular evolution can be. A Module is not an entire half of an animal, it is a region of the body that is functionally linked, and therefore constrained by evolution to evolve at a different rate than another region. This may be an entire half of the animal, but it will more likely be a smaller region of the body like the pelvis or the skull (again see the human evolution example), or a functionally linked collection of bones, like the wrist (again see ardipithecus) or soft tissues (mammal brain example). The functionally linked part is the key bit.

      Really I think these posts on modular evolution have done little more than show what poor science this blog represents. You decided because of one example you didn’t like, that modular evolution was rubbish, despite the fact that there are hundreds of pieces of literature on it spreading back before this example and continuing on after. After I called you out on your poor research a gave you dozens of references which demonstrate the mechanisms and theory behind it, rather than do further research to try and correct your misconception, you continue to argue single examples and ignoring the total evidence.

      Like I said, to overturn such a well established idea you need to provide evidence against the fundamental proofs and the mechanisms. Nit picking single examples is the tactic of a poor scientist.

  9. Sorry, I was under the assumption that modular evolution was invented by Lu et al. to explain their specimen. They described it as a front half and a back half that ultimately caught up to the front half in later generations, which the tree debunked. When I Googled modular evolution Lu et al is what came up. In an earlier reply I provided two examples of modular evolution, according to your new broader definition. Were those indeed good examples of modular evolution. If so, then all evolution is modular evolution. Since DNA and biochemistry is included in modular evolution there is no lower limit to modular evolution, evidently. I stand corrected. Thank you for providing me an education. I don’t think that is what Lu et al. had in mind, though.

    • Well, I did say a few times that modular evolution was not invented to deal with that pterosaur, and gave references from before the lu paper..

      “then all evolution is modular evolution”

      Indeed. Modular evolution was not intended as one of many ways evolution can occur, it was intended as a model for how evolution actually works, an alternative to what Tom Kemp calls the “Beanbag model”. Our current parsimony analyses work on the basis that each character is completely independant (like the beans in a beanbag, apparently; its an odd analogy) and has an equal chance of changing no matter what changes occur in other characters. Modular evolution recognises that certain characters are functionally linked, and that they are more likely to evolve as a unit, and that each unit can evolve at different rates depending on its constraints.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.