Sunday, June 27, 2010

Hockey Stick Science


A.W. Mortford has a book out titled The Hockey Stick Illusion. It highlights how modern science is done (I read its 400+ pages in two days because it was fascinating). The main issues are not abstruse statistics, but rather detailed, parochial empirical issues.

Recent warming only seems alarming if recent temperatures are outside of normal historical fluctuations. As the medieval warming period when Vikings settled Greenland was obviously very warm, at least in Greenland, one might think that current temps are not that alarming. Thus, in 1998, when Michael E. Mann, Bradley and Hughes published a paper documenting that current temperatures are many standard deviations of their average since at least 1000 AD, it became the signature graph for the Global Warming Community.

Tree rings, or isotopic composition of ice cores (the ratio of 18O to 16O) and other things are related to temperature, and these are the types of things used to estimate temperature prior to 1880. As the 20th century temperature increase that has everyone worried is only 0.6 degree centigrade, one needs some serious precision to claim that temperatures in the past 1000 years did not vary above this level. There's no fundamental law that related tree rings or oxygen isotopes to temperature, these things just have an imprecise theory and some empirical support, but it's not calibrated like some calorimeter. To think you can know the temperature in 1100 with the kind of accuracy that Mann et al present is really absurd.

The problem is there are many temperature proxies, various tree rings, ice cores, all with different results (over 400 of them). Mann et al eventually used 112 (or 159) of them for their paper, which allows for a lot of cherry picking. Further, some series are truncated, some extrapolated, using seemingly innocuous phrases like "if records terminate slightly before the 1980 training interval, they are extended by persistence". That's one bizarre way to treat missing data. They also extrapolated certain time series that did not start or end at convenient times, all with a bias towards their end ('We have to get rid of the Medieval Warm Period' said one infamous email).

There's lots of fun data issues and rhetorical strategy presented in this book that highlights how real science is done. You have two sides with pretty strong end-views--global warming is unprecedented, or not--and while both claim to simply be interested in the objective truth, after 10+ years invested in one conclusion it defies credulity to think a researcher can address this question objectively any more. Basically, we have two sets of partisan scientists presenting their case, like paid lawyers.

As David Goodstein notes in his recent book On Fact and Fraud: Cautionary Tales from the Front Lines of Science, a great quote from the great Richard Feynman:
The first principle is that you must not fool yourself—and you are the easiest person to fool.... After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.”

The winner of this debate will be those who fooled themselves the least. Like a financial economist rigging his backtest, this may generate a publication but in the long run the data are what they are, and its best to have the facts on your side because eventually the facts win. Very few are committing conscious fraud, but rather, fraud of the more common sort, that of where a seemingly innocuous inaccuracy saves tons of explanation in their mind.

As Oscar Wilde noted, education is an admirable thing, but it is well to remember from time to time that nothing that is worth knowing can be taught. Big debates are usually not centered on not singular facts or theories, but their many observations, knowing which are relevant, which are not. Knowing how to weight correctly is mainly an exercise in meticulous research and wisdom, and it especially helps to have correct or at least popular a priori prejudices.

19 comments:

Anonymous said...

funny thing about economists: they don't know jack about the economy (witness failure to foresee financial crisis, witness failure to agree on how to respond, witness absurd assumptions in mainstream theory, etc.) yet they like to tell practitioners of other disciplines how to do their jobs (witness freakonomics, robin hanson, and this post).

Anonymous said...

A few more thoughts: you may be correct, and this book's author may be correct. But for a non-climate scientist unfamiliar with the material and without the time to evaluate the evidence himself, a thinking person must decide whom to believe. Do I believe 98% of the world's climate scientists on the issue? Or do I believe an economist and an accountant? Fortunately, that's a simple dilemma to resolve.

Further, the idea that climate science is bogus involves an elaborate conspiracy. A full 98% of working climate scientists (per recent PNAS article) accept the hypothesis. How would they all get on board with bogus findings? What motive would they have?

One proposed claim is that they raised their discipline's profile and got all sorts of additional grant money. If that's why they all agreed to buy bogus science, why don't other scientists do similar things? Where are the astronomers warning of hundreds of meteors posing imminent threat to Earth? That would certainly grant them additional grant money and prestige.

Climate change may be incorrect, but I doubt strongly that an accountant and an economist are going to be the guys who figure it out. This seems particularly true when you consider that the economist works in a field that can't figure out that asset price changes aren't normally distributed and that economic actors are not rational optimizers.

Robert Johnson said...

Anonymous, I happen to agree with some of what you're saying about economists, and with the gist of your appeal to authority argument. However, I think Eric has presented arguments here that can't be dismissed quite so easily. Selecting from among data and claiming precision that is not supported by the data are serious problems for scientists working in all fields. The question is whether Eric is right to ascribe that kind of behavior to Mann and other climatologists. You reference an article that claims 98% of climatologists agree on...something. Care to elaborate?

Ken Coffman said...

For those who have not read this beautifully-written, easy-to-read and meticulously researched book, I am selling copies in the USA...with free shipping and a free bonus CD with supplemental material.
https://www.stairwaypress.com/shop/the-hockey-stick-illusion-climategate-and-the-corruption-of-science/

Jack Hughes said...

A full 98% of Homeopaths support the consensus view on homeopathy.

If I want the truth on homeopathy I consult specialists who have been working in this field for decades - not outsiders.

Anonymous said...

Robert

Eric can't claim they selected from data: he doesn't know. He says not using all 400 temperature proxies "allows for cherry picking." There's no allegation that cherry picking occurred.

Eric also doesn't know what precision the data warrants. All he says is that a 0.6-degree centigrade anomaly requires "serious precision." He has no clue how precise the data analyzed allows. Presumably, it's sufficiently high. That sort of statistical analysis would be elementary and would be easily replicated or shown not to be replicable. Maybe Eric would feel better if we converted to Kelvin, and it was a 273.75-degrees Kelvin anomaly. That doesn't sound like it needs as much precision. Smirk.

Further, Eric, given his credentials, probably doesn't have a clue about how precise tree rings, ice cores, and other temperature proxies are for creating historical temperature times series. This is the basis for my argument by appeal to authority. Neither Eric nor the accountant author of the book he's plugging likely have the expertise to critique the methods used in the climate change research.

It's clear that Eric hasn't attempted to replicate Mann et al's findings, and I doubt the author of the book he's plugging has the expertise to do so. Unless they did, they would just be speculating about faults that real professionals undoubtedly explored analytically.

Further, when Mann et al originally published their findings, the best way for a climate scientist to make a name for himself would have been to refute it. It would have been easy then, if it were false, because the discipline hadn't accepted it. The burden of proof was on Mann, yet it hasn't been debunked in the field, it's been embraced.

Here's the article I referenced:
http://www.pnas.org/content/early/2010/06/04/1003187107.full.pdf+html

I'm thinking Eric should stick to financial economics, a field in much worse shape than climate science.

Anonymous said...

Tree ring data is not a good indicator of past temperature.

This is why Mann ditched the later data in order to "hide the decline" and splice on non tree ring data as the more modern data did not go where he wanted it to go.

So where does that leave the validity of the previous tree ring data?

Doug

Eric Falkenstein said...

Well, clearly I won't settle the debate, I have neither the resources nor inclination. But here's an example of why I'm skeptical of the assertion that we are several standard deviations above historical temperatures for the past 1500 years (or so).

tree rings. For the past 150 years we've been measuring temperatures directly with various instruments. And generally trees seem to follow temperatures , growing more during warm years and less during cold, until around 50 years ago. Then, as scientific instruments registered rising temperatures many trees started growing less.

So, why does one presume that these same tree rings will faithfully represent temperatures in 1300, when they don't for the latter part of the 20th century? Who knows what kinds of arboreal plagues, droughts, solar flares, ozone holes, and whatnot could not have been relevant back then? Because certain trees did match temperatures for 100 years? So, if the pattern works for 100 out of 150 years, it works with 0.2 degree precision back in 1300? I think that's not very scientific.

Lastly, there's the issue of integrity. Mann et al have shown extremely tendentious reporting--withholding data in many cases, developing a calculated rhetorical strategy of 'not responding' and hoping criticism fade away, as opposed to actually addressing the issues. There is only one reason to withhold data, because you don't want to be embarrassed. No one lies like the indignant, and the Global Warming clique is filled with righteous indignation.

Anonymous said...

Call me anonymous2 :)

Anonymous 2:06pm says:

Further, when Mann et al originally published their findings, the best way for a climate scientist to make a name for himself would have been to refute it. It would have been easy then, if it were false, because the discipline hadn't accepted it. The burden of proof was on Mann, yet it hasn't been debunked in the field, it's been embraced.

It seems that you haven't heard about the subversion of the peer review process detailed in the Climategate emails. Any climate scientist trying to publish a refutation would have been subject to intense harassment.

Eric also doesn't know what precision the data warrants. All he says is that a 0.6-degree centigrade anomaly requires "serious precision." He has no clue how precise the data analyzed allows. Presumably, it's sufficiently high.

Again, you display quite entertaining ignorance. One tree-ring data set (anomalously) shows the 11th century to have been the coldest of the past era. Another reconstruction shows a dramatic fall in temperature since 1960.

These issues are collectively called "the divergence problem".

Most of the tree ring data sets do not continue to the present day. The sceptics keep asking that they are updated, but apparently there isn't a budget or any interest in this.

pesadilla said...

Anonymous, you are trying to defend the indefensible. By appealing to authority, you only weaken your case. Science, as you well know is not about consensus.
I think that you might be playing devils advocate in order to provoke others to react to your observations. As you well know, there is no empirical evidence to support your position, only adjusted data which has been tortured to the point of submission.
Computer modelling is not real science, it is biased and has been incapable of predicting anything of value in the 25 years or so that it has been available. All the empirical evidence is in conflict with these models. Where is the model that predicted what has happened since 1995.
Read the e-mails again and then try and make an objective assessment of what is really going on here.
Your only possible defence is the precautionary principle and that is the "drowning man clutches at a straw" defence.

Robert Johnson said...

I, for one, am inclined to concede the point that Anonymous 1 raises that without reviewing the evidence it's impossible to make an informed judgement about whether the recent warming is significant (or anthropogenic). Unfortunately, most of us are not going to be able or willing to do that, so we end up relying on authorities, choosing who to believe based on whatever understanding we can bring to the discussion.

At the same instant, Pesadilla is right: Science isn't about consensus, though consensus can be seductive (groupthink).

So maybe the most important question is, how can we evaluate expert opinion without needing to become experts ourselves? Does anyone have a quick and dirty (though hopefully relaible) method for checking who is full of **** and who isn't?

David said...

Robert
Try looking at the competing blogs and see how they treat people who disagree with them. A real scientist will seek to engage with and persuade those with different viewpoints: those with something to hide will edit, delete and insult their visitors and egg their supporters on to do the same. Compare the tone at Climate Audit, Lucia's Blackboard and Real Climate. Also apply some basic logic of your own and learn some simple facts: how much is the sea level rising? How accurate are tree rings as measures of temperature? Are tropical storms getting worse? The "consensus" gang and their supporters love to dismiss their critics as non-climate scientists and to encourage you to think that the subject is too complicated for you to understand without their priestly intercession. Instead, just believe in yourself.

Anonymous said...

The key offence revealed in the emails is the subversion of the peer-review ("cheer"-review) process, and the habit this allowed the Hockey Team to acquire, of persistently and wilfully neglecting the null hypothesis. Here's what Michael Kelly, of the Oxbrough Committee, wrote about the (carefully selected) papers he was given to read:

“(i) I take real exception to having simulation runs described as experiments (without at least the qualification of 'computer' experiments). It does a disservice to centuries of real experimentation and allows simulations output to be considered as real data. This last is a very serious matter, as it can lead to the idea that real 'real data' might be wrong simply because it disagrees with the models! That is turning centuries of science on its head.”

He concludes with:

“My overall sympathy is with Ernest Rutherford: "If your experiment needs statistics, you ought to have done a better experiment."”

I leave it to readers to reconcile the Oxburgh Report as published with Kelly’s contribution to it as best they can.

cheers,

Tom Forrester-Paton

Anonymous said...

Sorry, that should be "Oxburgh"

Anonymous said...

Anonymous has a rather high opinion of climatologists, indeed a totally unreal opinion.

The study of the climate has inputs from:

Mathematics
Physics
Dendrochronology
Geology
Statistics
Biochemistry

and probably dozens more. There is no one man who is an expert in all these fields and therefore I suggest that there is no such thing as a climatologist.

Stan said...

Someone above asked how can we evaluate the experts without becoming experts ourselves? Answer -- the same way juries have to every day in trials.

First, what is the burden of persuasion? The more draconian the proposed policy, the more certain we need to be.

Second, do the experts follow the scientific method?

In climate science, the clear answer is no. Not even close. 90% of the instruments flunk basic scientific standards for siting and the scientists never even bothered to check. It took an amateur to ask the question. This level of incompetence is simply breathtaking.

The first principle of forecasting is that a model has to be verified and validated to be useful. None of the climate models have passed this first basic step. Yet we are asked to spend trillions on their guesstimates.

Good science requires transparency and replication. Climate science has neither. Phil Jones, one of the most influential climate scientists, illustrated the intellectual bankruptcy of his profession when he refused to share data with another scientist on the grounds that the other scientist would try to find something wrong with his work!!

Those few influential climate studies that skeptics have been able to examine carefully have contained all manner of butchered statistics, bizarre assumptions, and garbled computer code. The climate databases are a mess because quality control has been a disaster. The IPCC, claimed by climate scientists to be the gold standard of scientific assessment of the climate, has been exposed as a fraudulent sham. The list of lies, misrepresentations, and incompetent work products goes on and on.

One doesn't need to be a climate science expert to understand that the field is a methodological disaster area. One merely needs to have a basic understanding of the scientific method and be knowledgeable of all the ways the field falls short.

The extraordinary policy responses demanded by alarmist scientists are exceeded in magnitude only by the enormity of the incompetence and methodological shortcomings underlying their 'science'.

Robert Johnson said...

David, thanks for the blog suggestions. I'll check 'em out.

Stan, I agree. Whether the policy recommendations make any sense is even more pertinent.

Drewfus said...

Its pretty simple really. The climate scientists can do all the 'agreeing' they like, but if mother nature disagrees with them, its all over bar the shouting.

Anonymous said...

Well said Jack Hughes and Stan.

I feel sorry for the "THE SCIENCE IS SETTLED" people. So desperate for social validation that they refuse to open their minds to heretical possibilities. They probably also have to pretend they liked Brokeback Mountain.