Science flexes its Bicep – perhaps prematurely

| 7 Comments

You may get an idea how science really works from an article by Boston science writer Kate Becker in today’s Boulder Daily Camera. (I think the column is exclusive to the Daily Camera.)

Becker describes the Bicep2 experiment, which looked for evidence of cosmic inflation by examining the polarization of the cosmic background radiation. The authors of the paper announced its conclusion before the paper had been submitted for review; since then, others have criticized their method and thrown the conclusion of the paper into doubt. Specifically, some think that cosmic dust may polarize the radiation in such a way as to give a false positive, in this case a polarization that mimics that of the cosmic background. The researchers have considered cosmic dust and disagree. At any rate, their article has finally been published, and you may find the abstract here. I read the abstract, but as Casca said, it was Greek to me, and I have no opinion concerning the conclusion. We will, as Becker notes, wait until other telescopes weigh in or the Bicep2 data are further evaluated.

Thus, if I understand correctly, cosmic inflation is still a theoretical concept, and scientists are trying to demonstrate its reality; no theory is presumed to be true in the way that certain religious texts are presumed to be true. However much the Bicep2 team would like to demonstrate the reality of cosmic inflation, they will ultimately go where the evidence leads them, in stark contrast to those who reject evidence because it does not appear to agree with their interpretation of a religious text. Indeed, that is why scientists immediately seized upon the data in order to see whether they could falsify it.

Becker concludes her article,

Some commentators are calling this [a] cautionary tale, a story of over-eager physicists rushing to make results public without sufficient scrutiny. These critics argue that the episode undermines public trust in science and scientists.

But I disagree.

After all, science isn’t just a series of right answers. (How boring that would be!) Science is a process. And now, we all have a front-row seat to watch it happen.

Unhappily, it seems to me, the commentators have a point. Whether or not the physicists were over-eager, and whether or not their results are upheld, it seems likely that episodes such as this one in fact undermine “public trust in science and scientists.” That is, even if science is working the way it is supposed to, people accustomed to having the “right answers” may not recognize the process and instead fixate on the uncertainty. Not all journals will accept a paper that has been previously published, even in a press release. I think it is better not to go public with a new result until the experiment has been thoroughly reviewed. If that means that the team was over-eager, then the team was over-eager.

7 Comments

If that means that the team was over-eager, then the team was over-eager.

Ergo, Jesus.

As always.[/UD mode]

Glen Davidson

Some of the problem is the effect that the Internet has had on highly competitive research groups trying to establish priority with an exciting result.

The same thing happened with supposedly superluminal neutrinos from the LHC. That “dramatic” result turned out to be caused by a faulty cable connection; it went away when the connection was repaired.

Many of these experiments are expensive, with heavy demands on time and manpower. The funding of these experiments is subject to political whims that could change on a dime if results are not forthcoming in the way politicians want them.

I suspect that all of this contributes to pulling the trigger on announcing results too soon.

I was once part of a military funded project within a major corporation in which the pressures of funding tempted some of the employees to announce results prematurely because they were anomalously “good.” I complained to no avail; and more careful work and analysis later revealed the results to be not as promised.

Under these kinds of pressures, it takes a lot of team self-discipline to do the grungy work of checking results and accounting for systematic effects before blurting out a public announcement.

I think we saw a little better discipline in the announcement of the Higgs.

Of course this happens only in Science — except when it doesn’t.

Mike Elzinga said:

Some of the problem is the effect that the Internet has had on highly competitive research groups trying to establish priority with an exciting result.

The same thing happened with supposedly superluminal neutrinos from the LHC. That “dramatic” result turned out to be caused by a faulty cable connection; it went away when the connection was repaired.

I don’t think that’s a fair description of the OPERA team’s conduct. From what I recall, the original announcement was made only after a long period of in-house troubleshooting, and was carefully couched in terms of asking for outside troubleshooting. In fact, this was in the original paper: “Despite the large significance of the measurement reported here and the stability of the analysis, the potentially great impact of the result motivates the continuation of our studies in order to investigate possible still unknown systematic effects that could explain the observed anomaly. We deliberately do not attempt any theoretical or phenomenological interpretation of the results.”

ksplawn said:

I don’t think that’s a fair description of the OPERA team’s conduct. From what I recall, the original announcement was made only after a long period of in-house troubleshooting, and was carefully couched in terms of asking for outside troubleshooting.

Yes; I remember that; and I think those caveats were prompted by the criticisms coming from other physicists.

I was keeping up with reports within the physics community. Many physicists were quite critical of and uncomfortable with the way those results were announced and published.

There is a fairly reliable grapevine within experimental communities, and it is made up of people who have some familiarity with the “human dynamics” within various experimental groups. As I recall, there was some indication of conflict within the group about announcing and publishing those results. Hints of this even appeared in APS News.

I personally think that it is okay for the public to get a look at some of the inner workings of frontier research. As Matt notes from Becker’s article, such a front-row seat affords the public a more realistic look inside these research efforts. Research is not easy; and it doesn’t always proceed in a straight line.

In fact, it is good for students to learn about this process in order to gain a more realistic picture of how research proceeds. People who have ideas that they think should be tested need to understand what that involves; and that includes understanding all the time, budgetary, and political pressures that all experimental groups feel in addition to the routine processes of developing and placing the experimental handles on a theory.

Incidentally, the reports within the physics community on the Bicep2 results are not hyping them in any way. They are simply noting that there are other experiments currently in progress that will be producing data that will be needed to crosscheck the Bicep2 results. Because of the difficulty and expense of these experiments, there is a cluster of experiments in progress that produce data bearing on the same questions. That has always been understood from the conceptual beginnings of these experiments.

The same processes are taking place in the search for dark matter; there are several different types of experiment taking into account various systematic uncertainties.

I suspect that if there is any “sensitivity” to allowing candid looks into the workings of frontier research, that sensitivity may be spurred by the kvetching of sectarian groups like the ID/creationists. Those people are always sitting on the sidelines criticizing and razzing science and scientists; and they do this while claiming to know all about science and not being able to do any of it at even a middle school level.

Over at his “preposterousuniverse” blog, Sean Carroll promises to “weigh in about BICEP”, real soon now.

http://www.preposterousuniverse.com[…]in-your-face

George

Thus, if I understand correctly, cosmic inflation is still a theoretical concept, and scientists are trying to demonstrate its reality

I believe this is incorrect, if by “theoretical” you mean to say that it’s something like string theory in being “not yet tested” against reality. Inflation theory predicted what the structure and detail of the cosmic microwave background would be, predictions confirmed by both the COBE and WMAP missions.

Inflation also correctly predicted flatness and no exotic artifacts (monopoles, etc.). Both of those claims were already suspected at the time the theory was developed, so inflation was more post-dictive than predictive about that. Nevertheless, the large advances we’ve had in astronomy in the last 30 years means those claims could have turned out to be wrong, and they didn’t, so maybe we should award the theory an extra half a point each for those post-dictions. ;)

Leave a comment

About this Entry

This page contains a single entry by Matt Young published on June 27, 2014 2:54 PM.

Understanding creationism, V:An insider’s guide by a former young-Earth creationist was the previous entry in this blog.

Photo Contest VI: Finalists is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Categories

Archives

Author Archives

Powered by Movable Type 4.381

Site Meter