This post was inspired by this other one, featured in WordPress’ Freshly Pressed section, on how should non-scientist read a scientific paper. While the approach presented therein is both valid and valuable, I’d like to address the way I think a scientist should read a paper, given the fact that we need to read a lot of them at all times. Each scientist has their own reading style, not to mention their own writing style, and while my CV could indicate I don’t know how to do neither one, here I present to you my scientific-paper-reading style which I consider to be the most suitable for me.
I’d like to start by emphasizing that I dive into scientific literature in a bona fide fashion. That is not to say I’m totally naive or even gullible, but even when science is all about questioning and casting doubt onto all sorts of claims, we can’t re-develop every bit of science we need. At a certain point we must start
*gasp* believing trusting other scientists’ claims. Reading in what I call bona fide is not mutually exclusive with critical reading. This sort of scientific trust is earned, to a degree, mostly by two indicators: Author’s preceding reputation at the time of publication of any given paper as well as the journal’s. Both indicators aren’t without controversy and flaw.
The way I read a paper is the following: I start with the Abstract, then follow with the Conclusions, then the Results section, sometimes I read the details of the methodology and seldom read the Introduction. Let me explain.
I read the abstract first because I read in bona fide as I hope the authors wrote the paper in bona fide. If properly written, the abstract should include all the relevant information as to what was done, why, and how but also point to the knowledge derived from it all: Their conclusions! and that is why I follow with that section. I’m interested in knowing what the authors learned and ultimately want me to learn about their study. Once again I’m reading in bona fide, so I hope they weren’t tempering their results to fit their preconceptions, that all experiments were thoroughly self-judged, validated, correlated, referenced and controlled. Recently, my sister Janet, who is a physicist working on her PhD in neuroscience, told me about some friends of hers who never (shall I say, never have I ever?) read the conclusions as to not becoming biased by the authors. To me it seems like too much work having to scrutinize every piece of data again in order to come up with my own conclusions when authors, collaborators, people on the hallway down the lab (optional), referees and editors (vide infra) have already (hopefully) done it (properly). Still I put on my scientist badge and question everything I critically read in the results section trying thus to understand how did the authors reached their conclusions and asking myself if I could come up with something entirely different. No? OK, how about something slightly different? Still no? Well, do I agree with the authors on their findings and their observed results? And so on. I like thinking that my critical reading process resembles the Self Consistent Field method which iteratively reaches the best wavefunction for a set of certain given conditions, but it never reaches the exact one.
The methodology section is a bit tricky, specially when it comes to computational chemistry. Back when I was a grad student, working in an inorganic chemistry lab, I’d only read the methodology if I had any plans of reproducing the experiment, other than that I didn’t care too much if reagents were purchased from Aldrich or Fluka or if the spectrophotometer was a Perkin Elmer one, I just expected authors to have purified their reagents prior to usage and calibrated all spectrophotometers. Now in computational chemistry I read about the methods employed, which functional and what basis set were used and why were they selected are my most frequent questions, but the level of theory is usually stated in the abstract. I also take a look at what methods were used to calculate which properties; these questions are important when we have to validate our trust in the results in front of us.
Finally, I seldom read the introduction because, if the paper is relevant to my own research, I don’t need to read why is important or interesting, I’m already sold on that premise! that is why I’m reading the paper in the first place! If both me and the author act in bona fide, we both already know what the state of the art is, so lets move on because I have a ton of other papers to read. Hence, I read the introduction only when I’m trying to immerse myself in a new field or when reading something that seems interesting but which has little to do with my area of expertise. There is another reason why I almost never read introductions and that is that, even when I try to work in bona fide, there are a lot of people out there who don’t. Twice have I received the reviews from a mysterious referee who believes it would serve the work a great deal to cite two, maybe three, other papers which he or she lists for your convenience, only to find out that they all belong to the same author in each case and that they are not quite entirely related to the manuscript.
In the title of this post I also try to address the writing of a scientific paper, although I’m not an authority on it, I think today’s key phrase is bona fide. So to young and not so young scientists out there I’d ask you to write in bona fide, please. Be concise. Be convincing. Be thorough and be critical. This is science we are doing, not stamp collecting. It shouldn’t be about getting all sorts of things out there, it is about expanding the knowledge of the human race one paper at a time. But we are humans; therefore we are flawed. More and more cases of scientific misconduct are found throughout the literature and nowadays, with the speed of blogging and tweeting, we can point at too many of them. The role of bloggers in pointing this frauds, of which I’ve written before here, is the subject of recent controversy and possibly the topic of a future post. We are all being scrutinized in our work but that shouldn’t be an excuse to make up data, tinker or temper with it, to push our own personal agendas or to gain prestige in an otherwise wild academic environment.
I for one may never publish in Science or Nature; I may never be selected for any important prize, but even the promise of achieving any of those is not worth the guilt trip of lying to an entire academic society. I try then, to always remember that science is not about getting the best answers, but about posing the right questions.
What is your own style for reading papers? Any criticism to my style? How different is the style of a grad student from that of a researcher?
As usual thanks for reading, rating and commenting!
If a mind is a terrible thing to waste, then wasting a collective mind is an even more terrible thing. During the past weekend the library at the institute of chemistry suffered a flood caused by a broken pipe just above it, which incidentally happens to be the lab were I used to work as an undergrad student. When it comes to scientific journals, our institute still relies a lot on paper issues for the oldest numbers; we can order them online but it’s just easier to Xerox it at the library if you really need to read that old reference.
This morning the librarians were appalled when noticed not only the huge puddle on the floor but all the books and scientific journals that were dripping water from the shelves. The broken pipe has been fixed and the water on the floor has been mopped. It is now the books the ones that suffer the aftermath of this accident. Not only saving the information was important; wet paper is a great culture media for fungi which in turn could pose a health threat to all users. The administrative staff immediately got to work in recruiting academics and students to help the drying process: “Heal a book!“, they informally called it. Everyone grabbed an item and with the help of industrial blow dryers – the kind we use in chemistry labs to dry wet glassware – and an extraordinary amount of paper towels, each person got to dry the journals page by page.
I got an item that corresponded to the British journal New Scientist, which consisted of about fifteen issues from the year 1980. When I noticed the title in my hand I wanted to switch it. Should we save first those journals with the highest impact factor? or should we work on those that are most relevant to our own research? Should we throw away Chemical Abstracts now that the whole database is online? After all, New Scientist is a magazine which summarizes research that has already been peer reviewed and published; it is journalistic work, not peer reviewed science. But I was afraid to look pedantic so I got to work on drying it.
Each person had their own technique. Some journals had their binding covers still in good shape so they were placed open standing on the floor in front of fans. Some placed paper towels carefully between pages and after a while they would remove them and then use the blow dryer. I thought that if I heated the edges of the paper and thus dried them, capillarity would drive the moisture in the innermost part of each page outwards. Didn’t quite work, at least not in a pragmatic time scale, so I went back to page by page.
I’m glad I did so. That way I was able to find some real pieces of history which could make any scientist nostalgic. For example: I took these photos with my iPod, and if you are by any chance reading this piece on an iPhone, you must find the following picture about Swedish research endearing.
Yes, online doodling games were already a thought back in 1980!
Are you subscribed to this blog? That means you got a notification by e-mail. So what? No big deal! Well, back in 1980 Britain was getting excited over a new form of comunication called the ‘Electronic Mail’ (available only at a couple of post offices). Besides, you wouldn’t have been able to get that message nor read this post on an HP Matrix Machine (you can’t even find a decent link in google about it nowadays!)
But scientists are not all about working, we like games too! So how about purchasing a ‘Hungarian Magic Cube‘ or a ‘Chess Computer‘?
We also love a juicy piece of gossip. For instance, did you know that John Maddox was a controversial editor for Nature back in the 70’s who, as a student, went into chemistry because if he’d gone into physics he could’ve been drafted by the army in WWII to work on radars? Well me neither. But it seems that we should have known who he was, and now we do.
There were many pieces of science news that nearly kept me in the library all night, if not for the fact that I had to drive 50 miles from Mexico City to my place in Toluca, but the one that captured my attention more than any other was the news of a European dream envisioned more than three decades ago; a dream from a group of scientists about looking for answers, like any other group of scientists, answers that are fundamental for the understanding of our universe and the understanding of matter, back when some of the biggest questions hadn’t even been fully posed, this group of visionaries agreed on taking the necessary steps to build an enormous subatomic-particle Supercollider for the European Center for Nuclear Research, better known as CERN.
Back in 1980 I was already alive but I was only two years old. I could barely talk and had no idea what the word ‘future‘ meant, let alone what I’d become when it reached me. Now, even if I’m not a particle physicist I get excited about the news regarding the finding of the Higgs Boson and even if I’m not an astronomer I also get excited about pictures from the Curiosity Rover on Mars. I am a scientist. One out of hundreds of thousands or perhaps even millions, and this is part of my collective memory, the memory of the work of those who paved the road for us, those giants upon whose shoulders we struggle day by day to stand with dignity and against all odds. But here is the thing: those giants are actually made of dwarfs, millions of them; millions of us. Thousands and thousands of papers written, reviewed and published; papers that collectively gather the scientific experience summed up in rigorous experiments both successful and failed.
Preserving the information in those wet journals is important despite the fact you can get them all online. I hope one day a bored chemistry grad student goes to the library and browses old issues of New Scientist and other journals just for fun; they’ll go for a trip down a collective Memory Lane which will remind them that if they can dream it in the present, they can make it come true in the future.
This title may sound like the one of an episode of the famous geeky sitcom The Big Bang Theory but it is not; it is in fact a far more interesting albeit less fun debate. First of all I want to make clear this post isn’t an attempt to further bash Dr. Schön err I mean Mr. Schön (sorry, I couldn’t help it) but the entire debate raises some interesting questions, specially those regarding the recent outcome of the controversy as well as the forthcoming aftermath, that are worth asking and of course, with some luck, answering.
A little background first: Jan Hendrik Schön (Germany, 1970) got a PhD in physics at the prestigious Konstanz University in his homeland; after that he got a job as a researcher at the even more prestigious Bell Laboratories located in New Jersey USA, where he made groundbreaking discoveries on conductivity, superconductivity, organic conductors and semiconductors. Ironically enough, his conduct was his doom (Sorry again, I couldn’t help this one either. ) Between 2001 and 2002 he published more than 60 scientific papers on these topics, 15 or so of which were published in Science and Nature. Similarities in the graphs led to other scientists to believe the data could have been manipulated which turned out to be the case! Little by little, both journals Science and Nature, as well as other important journals like Physical Review and Applied Physics Letters withdrew some of his papers; others remain under further investigation for their possible withdrawal. Dr. Jan Schön eventually had to come clean and confess his lack of scientific rigor and misbehavior. This and him being fired from Bell Labs could have been the end of the story but in no way was near it. In 2004 the University of Konstanz decided to withdraw his PhD degree to which Dr. Schön appealed on the grounds that his thesis work was performed without any data manipulation or any other sort of ethical misconduct on his part; finally today after seven years of back and forth lawsuits and appeals the state court ruled against him and the university stripped him from his degree. Dr. Schön is now Mr. Schön. Further details (since this post does not intend to be a repository of others nor to just inform you about the entire gossip) can be found in this Wikipedia article.
So what is this post about then? First of all I’d like to address the obvious questions: Was the University of Konstanz right or wrong about withdrawing his degree? Did they go too far? What are the implications for other scientists? There is still controversy among the scientific community about whether or not the University had any right to do it on the grounds of scientific misconduct on work that they did not fund nor had to do anything with. The matter is fairly obvious, the University of Konstanz does not want to be affiliated with Schön on any level since it will indirectly hurt their reputation. But is someone really thinking that this university is to blame? I hardly think so. Therefore on that side they could have let him keep his degree but on the other side one may postulate that credibility must be sustained throughout our careers just in the same way as MD’s can have their licenses revoked under malpractice. Granted, MD’s get stripped only from the right to legally exercise medicine not from their degree, meaning that in some cases they may recover that legal right instead of having to go to school all over again, although in the harshest cases this wouldn’t help either.
Definitely his conduct deserved strong actions from his employers and the scientific community, in the end almost 30 papers in very prestigious journals had to be withdrawn! He was trying to take the scientific community for a spin! And this is another thing I’m baffled about. Didn’t he ever think that such claims, such amazing claims, would attract the interest of a large number of the most prominent scientists working in the field of conductivity? As if there weren’t billions of dollars invested in those topics worldwide! If he was breaking new ground in organic superconductivity by proving some theoretical predictions, he would have gotten a Nobel prize for sure and that would have attracted a lot more people in trying to reproduce his findings in order to make their own little contributions therefrom. In fact this actually happened! Many laboratories throughout the world claimed to have failed in reproducing his results. I can only imagine those teams feeling frustrated for not being able to get the same numbers/trends in their experiments and clearly getting less and less frustrated when they found out they were not alone and that the number of their companions was growing larger. Had he published his results in some obscure, dubious-quality journal probably nobody would have ever found out, but then Lucent Technologies, the profitable company that runs Bell Labs would have not been happy in funding his research. This in turn raises yet other questions: How come nobody at Bell Labs was able to tell his results were all made up or even just tempered with? Private companies are not run as academic labs, publication of findings go through a lengthier process than in academia, which is normal since private research centers invest a lot of money in developing technologies which will generate enough profit to keep the company running and researching for as long as possible. They also have strict policies about data-recording and experiment-tracking procedures which apply to every researcher in the company; they are not subject to interpretation or to desire, they have to be followed to keep track of all the expensive research done within. The second question I got from this paragraph is about other frauds done in lesser journals which may go unnoticed because the work itself simply goes unnoticed by its own lack of merit. And from this, yet another question is raised and linked to another ongoing controversy: What is the future of peer-reviewing in journal publications? Surely no referee would have tried to reproduce his experiments and I’m sure a large number of data was requested by Nature and Science in order to deem the papers worth of being published (was his answer to this request “No problem!“?) Referees this quality were blindsided by reputations, Schön’s and Bell Lab’s; only until the final users (the readers) noticed similarities between data sets, and noise, in different experiments that the fraud rose to the surface, but noticing those similarities should have been the work of the referees! in fact that is their job! Aren’t they accountable by omission too?
This controversy is not a first, nor will it be the last. The most famous controversy of data tempering that comes to mind right now is the infamous experiment, or more accurately the infamous data selection in Robert Millikan’s experiment for measuring the charge of the electron. In this experiment only the “nicest” data points were used and although inclusion of the entire data set would have not affected the final value of the charge obtained, the statistical error would be increased to 2% instead of 0.5% as he presented it to the scientific community; a much “nicer” error. Of course I’m not trying to compare both cases which are completely apart; selecting data is not as serious as data manufacturing, but they are both just as unethical. Should have Millikan been stripped from his PhD degree and/or his Nobel Prize?
Mexico is not immune to this controversies either. In 2006 three papers authored by the famous Mexican Professor Eusebio Juaristi and his student Omar Muñoz-Muñiz in 2003, had to be withdrawn also on the grounds of irreproducibility from the Journal of Organic Chemistry, Tetrahedron and Tetrahedron Letters. There were serious errors in those papers which led to believe the student had incurred in scientific misconduct but apparently they managed to prove they were simply honest mistakes. Was Muñoz-Muñiz denied his PhD degree? No. In fact he now works as a researcher at Universidad Veracruzana in Mexico. Not a bad gig. I understand he had some problems getting into the National Researchers System which gathers us all researchers in Mexico as an independent entity at the same time that it collects data about the research done in the country. Data seems to be today’s secret word kids! Once again the whole ordeal could have been dealt with if a proper peer reviewing process had been carried out from the very start.
The conclusions: Ethical work, ethical reviews. Is it really that hard?2011, International Year of Chemistry http://www.chemistry2011.org