|Posted on December 23, 2010 at 4:05 AM|
An article recently published in the prestigious scientific journal, Biology Letters, claims to have been written by 25 8-10 year olds. The paper, Blackawton bees, co-authored by Beau Lotto from UCL (and the 25 8-10 year olds), reports the data collected from an experiment it claims was designed and conducted by the primary school children. The paper includes no citations as it does not review any relevant literature, but presents the data in 'kids speak'. Nevertheless, Lotto claims in the prologue to the paper, the lack of scientific context does not take-away from the merit of the study, which he describes as scientifically and conceptually novel. Lotto goes so far as to describe scientific endeavour as a series of 'games' which even children can play and claims that the paper 'reveals science in its truest (and most naive) form, and in this way makes explicit the commonalities between science, art and indeed all creative activities'.
Judging by the response to this article on fb, many readers (including scientists) seem to have responded positively to the article, decribing it as charming and cute and applauding the way it is reaching out to children and getting them involved in science - something which scientists generally like to see. My response to the article was somewhat different. I found myself somewhere between 'You've got to be kidding!' and 'Well, I guess I won't be submitting to Biology Letters again'. I found the science to be underwhelming (at best), the abstract/background written by Lotto to be borderline offensive and the fact that the article is published in what should be a highly respected journal disappointing and potentially damaging to scientists everywhere.
I will start with the science of the paper. Although it is accompanied by a generous commentary written by Laurence T. Maloney and Natalie Hempel de Ibarra, which attempts to provide the credible scientific context the article itself is lacking, the absence of citations ought to have been sufficient for Blackawton bees to have been rejected in its current form. In the abstract, Lotto claims that the inclusion of citations would have been disingenuous, as scientific literature is not accessible to 8-10 year olds. Disingenuous, really? Since the students came up with the design of the project all by themselves, generated their own hypotheses, developed their own ways of testing for alternative interpretations and even wrote their own manuscript (as claimed in the abstract)? Sure they did. Just a coincidence then that they happened to come up with a theoretical question that could be answered using Lotto's famous bee matrix? Of course not. The children were led by Lotto through the design process and through the write up process. The children did not write the paper of course, but their conversations in small groups with Lotto were transcribed and (presumably) cherry-picked and pieced together to produce the final product. And what knowledge did Lotto use to inform his decisions as he guided the design process and led the write-up conversations? Knowledge gleaned from previous research on bee visual cognition - research that should have been cited. Of course, the fact that a qualified scientist (Lotto) also appears as an author on the paper, means that the inclusion of citations would have been anything but disingenuous.
The lack of citations, masquerading as something along the lines of childlike purity, allowed Lotto to claim that his findings were novel and important without having to actually describe how they were novel or why they were important. This may not have been a problem were there not a wealth of literature already demonstrating the ability of bees to merge colour and spatial information (in fact, we've known for quite some time that bees can merge pretty much all and any combinations of colour, visual pattern, spatial, temporal, circadian and odour information) when making laboratory foraging decisions. Any reader with a basic working knowledge of comparative cognition would struggle to appreciate exactly what the scientifically novel aspect of this study is supposed to be. Perhaps an expert on bee cognition may be able to detect which detail of the methodology represented a manipulation that had not been previously reported (if indeed there was one), but the majority of readers of Biology Letters would expect such information to be included in the manuscript itself, backed up by citations which one could look up should one have doubts about the novelty or interpretation of the data.
The abstract/background written by Lotto makes one comment, in particular, that I, as a scientist, find to be offensive. He writes that the 'true motivation for any scientific study (at least one of integrity) is one's own curiosity'. Is the implication supposed to be that a scientist motivated by something other than curiosity (like, oh I don't know, finding a cure for cancer) lacks integrity? Interesting perspective. I wonder why Lotto himself has bothered to publish any of his findings. If he is purely motivated by curiosity, then once he has the data he has satisfied his curisosity, hasn't he? Can't he move on and satisfy his curiosity about something else, without wasting time on such laborious, disingenuous tasks as writing up and publishing? (I guess he did get a class of primary school children to write his latest paper for him...). A lot of factors motivate scientists to continue their research (striving for recognition, a drive to succeed, a desire to help others or the world, a need to obey their supervisor until they can get their own lab!) and none of these motivations imply that the scientists or their data lack integrity.
I think that what I find most offensive about this 'integrity' comment is that it is embedded in a manuscript that lacks any kind of scientific integrity and is itself, nothing but a gimmicky publicity stunt. Does Lotto truly expect us to believe that curiosity about bee visual cognition was the major motivating force that led him to submit a 'kids speak' paper containing questionable findings to one of the most prestigious biology journals he could find? Does he also expect us to believe that each of the 25 school children made a genuine authorship-level contribution to the manuscript? I would expect that a scientist of genuine integrity, in possession of data that they believe is interesting and worthy of publication, would show that data the respect it deserves and write it up in a way that best facilitates its dissemination to the relevant scientific community. By presenting these data in 'kids speak', Lotto fails to not only show the data the respect they deserve, but also denies his readers and the general scientific community the respect they deserve.
Lotto's own descriptions of the children's contributions to the manuscript are also of questionable integrity. He claims that the children 'asked the research questions, hypothesised the answers, designed the experiments and drew the figures'. One can tell by looking at the figures that the children didn't draw them. They were clearly given printed sheets which the children coloured in appropriately. As for the rest of the claims about the children's contributions, given that Lotto is explicit about these children's lack of knowledge of any previous work, it is preposterous to expect the informed reader to believe that these children were responsble for all these steps of the design process. Given that, and in the absence of any genuine attempt by Lotto to detail the children's actual contributions, one is left doubtful that 'Blackawton bees' should be taken at face-value, even with what little face-value it has to offer.
Thus far my criticisms may seem to be levelled squarely and solely at Lotto. It is worth pointing out, however, that without the compicity of the editors of Biology Letters, this manuscript would never have seen the light of day. Editors, under normal circumstances, are the individuals within the scientific machine who are chiefly responsible for ensuring that the peer review process is conducted thoroughly and fairly for each piece of work that is ever published. Given that 'scientific excellence' is the first and foremost selection criteria listed by Biology Letters on its website, it is very difficult to believe that this paper was subjected to the same rigours of peer review as any other submission. For that, the editors owe the rest of us an explanation.
The issues I describe above may seem not so serious to some readers. Ok, so some reasonably unexciting data have been presented without citations and the author list contains a host of people who likely didn't contribute as much as an author ought. But does this really matter? Don't the novelty and wondrousness of having children collect data and publish a paper outweigh the fact that the paper itself is not what you'd typically find in Biology Letters? I mean, they're children, you can't expect them to conduct a proper study and write a proper paper! But this is exactly my point. If Lotto wanted to make a point about children being able to do real science, then why not treat these data as real science? Why not present them the way one would any other paper, subject them to the rigours of peer-review and then, once published, announce that the author list contains mostly children, who played a key role in the conception of the study and the collection of data? The answer to these questions, of course, is that publishing these data properly was not really an option for Lotto. A rigorous peer-review process would likely have resulted in these data being rejected from the average behaviour/cognition journal. With only 5 subjects and what essentially amount to null results in the second and third experiments, these data would have been difficult to publish anywhere. They would never have even been considered for a journal such as Biology Letters.
Now don't get me wrong, it's fantastic that children are getting involved in science. Primary school science education programs are immensely worthwhile. They foster interest and understanding of the world and even promote the beginnings of critical analysis and skepticism. These are all noble objectives. But just as the local under 10's football team isn't given prime time TV coverage on a Saturday afternoon (because they can't actually play elite football), the local primary school's science lesson doesn't warrant prime time coverage in an elite scientific journal.
By far, my biggest concern about this manuscript is not (just) its lack of scientific rigour, but what it implies about the scientific process. Lotto explicitly refers to science as 'little different' from playing a game in the abstract of this paper. He furthers his comparisons between science and play when he suggests that 'Blackawton bees' represents science in its 'purest' form as it has been done by children. Now hang on just a second. Science is not a game. To suggest that science is best, or purest, when conducted by children is a piece of sickeningly sentimental romanticism. But it is worse than that. It also implies things about the scientific endeavour that real scientists know to be untrue. Firstly, it suggets that the money (and there is loads of it) that governments around the world spend training research scientisits is not just a complete waste, but is counter productive. If you logically follow through Lotto's views you might think we should just let research students follow their naive curiosity and not taint their purity by forcing them to read, understand and integrate into their thinking decades of previous research methodologies, theories and findings. The other point implied by Lotto's comparisons between 'science, art and, indeed, all creative activities' is that science itself, the best science according to Lotto, is nothing more than a group of expensively-trained individuals spending tax-payer funded grants to satisfy their curiosity about 'how bees see' or 'how frogs smell' or some other topic that would seem equally arbitrary and pointless to the lay reader.
Training research scientists is expensive, time consuming and absolutely necessary. Conducting all kinds of scientific research is expensive, time consuming and absolutely necessary. Scientisits (including Lotto, I'm sure) know this to be true. The average tax-payer who funds these activities, however, does not. And this is the cost that the rest of us will pay for Lotto's journey into the magical land of Everything-done-by-a-child-is-pure-and-wonderful. This publicity stunt has already made its way into many popular newspapers and no doubt will be covered by many more. Because of this, the never-ending struggle scientists face to convince governments and tax-payers that the costs of scientists' expensive endeavours are justified and necessary for the continued development of our societies, economies and lifestyles just got a whole lot harder. If scientists are willing to disregard the scientific process in such a blatant way as 'Blackawton bees', how can we expect lay people to regard it with the respect it deserves?
Categories: In my opinion