Mike Butler: Hide vs NIWA

A detailed and somewhat technical battle is being fought over the introduction of the Emissions Trading Scheme, little of which is finding its way into the mainstream media. John Boscawen and Rodney Hide of the ACT Party are leading a campaign against the ETS, arguing that it is senseless for New Zealand to proceed with such a scheme on its own since the additional costs incurred would disadvantage economic competitiveness. ACT argues that scientific basis for such a scheme is far from settled and has become mired in political self-interest, and riddled with scientists manipulating data.

Hide’s detailed criticism of New Zealand' National Institute of Water and Atmospheric Research (NIWA), to Waikato Federated Farmers on Thursday drew a swift reply from NIWA’s chief climate scientist Dr David Wratt. Here are Hide’s allegations and Wratt's response:

Hide:
What then about the science?
This is the other necessary condition for us staying fully involved in the Kyoto process.
Is the science unassailable, is the matter settled?
Well no, it never was. I am not suggesting that there is not a respectable body of science suggesting we should be concerned about the potential for global warming from the activities of mankind. Because there is.
But there is also a respectable body of science suggesting that the risks have been greatly exaggerated, that our level of understanding of global climate is weak. What we have known for some time is that the institution of the IPCC has become driven by political agendas, rather than by science.
We know of the notorious hockey stick graph, of data manipulation to eliminate the Medieval warm period and produce a sudden dramatic spike in temperatures in the past century.
We have heard of the numerous projections of imminent disaster that had no basis in established science. In short, the credibility of IPCC process has been destroyed by the political agendas that have infiltrated it.
Now Climategate has left the science looking even more suspect than it already was - and it was by no means secure before Climategate. The hacked emails, documents and computer codes from the University of East Anglia Climate Research Unit (CRU) are now in the public domain.
They reveal a systematic attempt to manipulate the historical time series data, together with arbitrary adjustments to the computer codes which produce the averaged and smoothed temperature data.
CRU officials, from the top down, have conspired to block official information requests, influence and control the peer review process, and avoid accountability. They claim to have lost the original, unadjusted data, which means it cannot be cross-checked in the usual mode of scientific peer review.
And independent reviews of their methods confirm that the CRU was disorganised and lacking in competence in statistical methods. Given that most of what they do is database management and statistical analysis, this is an astonishing revelation.
Their data is the crucial input to the computer models on which projections of rising global temperatures are based. Those projections have led the global community to consider introducing globally binding ETS systems to reduce emissions.
You know the old saying about computer models: garbage in, garbage out.
These global emission targets, if met, will have a substantial negative impact on world economic growth and incomes, and thus on global poverty. These are issues, therefore, of the utmost importance.
Climategate has massively damaged confidence in the methods, processes, and basic data on which all the analysis is based.
Well, you might say, at least it couldn't happen here. Well here's a curious thing.
New Zealand' National Institute of Water and Atmospheric Research (NIWA) is responsible for maintaining our climate records, having taken over from the NZ Meteorological Service in 1992 when NIWA was established.
In the early to mid 2000s NIWA produced a graph showing New Zealand was warming even faster than global averages.
Around that time the government started exploring options for reducing emissions - carbon taxes and ETS arrangements. Helen Clark, then PM, later announced her Party's intention that New Zealand should lead the world in fighting climate change.
The NIWA graph was most helpful.
This new official NZ temperature series - which was based on only seven measuring sites - showed a warming trend of 0.9 degrees centigrade per century. The Climate Conversation Group (CCG) and the NZ Climate Science Coalition (CSC) combined to investigate the New Zealand temperature record.
They started with the official graph and the raw temperature readings, both published on NIWA's web site. There are big differences between them. The raw data shows basically no trend since the 1850s, whereas the adjusted series shows the 0.9 degree rising trend per century.
But there was no explanation for the differences on the web site.
Most of us know there are many good reasons to adjust a series of temperature readings: there may be gaps in the series, the site for the temperature readings may change, the environment around the site may change (eg trees or roads or parking lots), or what was once a rural area becomes urban over time.
These can all be valid reasons for making adjustments. The technical term for the documentation of these adjustments to raw data is a Schedule of Adjustments (SOA). It's quite normal to have an SOA.
In fact it is essential - that's how you keep track of the reasoning behind the adjustments, and this is what allows other scientists to cross-check the quality of the work. Replication of results is an essential part of good science.
And here is where it gets interesting.
NIWA started to get evasive when asked for the SOA. In answer to requests they talked about the standard reasons for making adjustments, and they referred to numerous papers which describe different methodologies for doing so. They said the information could be found in obscure papers from 20 and 30 years ago.
The one thing they did not do was simply hand over a schedule of adjustments. It wasn't a state secret. The climate science coalition duly followed up the references. But there was no SOA in any of them.
In frustration they contacted me, and we started asking questions of the Climate Change Minister, and then the Minister for Research, Science and Technology.
We tried oral questions in Parliament. We got evasive answers. So we peppered them with written questions. Again, evasion.
We got the usual references to academic papers scattered around the internet, and one on restricted access at a university library. We managed to get that one on interloan, electronic access only, but were only allowed access for a limited time.
It was like trying to get information out of the Kremlin, but not as cooperative. Coming after the scandalous behaviour and sloppy processes revealed in the United Kingdom at the climate research unit, we started to think the unthinkable.
Are they trying to hide something?
Maybe we have our own CRU scandal?
All we wanted was a schedule of adjustments for the official climate series, so that scientists - not politicians, not political parties - could check the analysis.
All we wanted was some basic science. As scientists, NIWA should have been welcoming scrutiny of their data and methods. Instead they were blocking us at every step.
It eventually became clear that NIWA simply did not have a well-organised database, with an accessible or up-to-date, or defensible schedule of adjustments.
One of the reasons for the embarrassment and defensiveness apart from hiding sloppy and inadequate scientific procedures is probably that NIWA along the way realised that it actually has a statutory duty under the Public Records Act to maintain full and accurate records.
They are in breach of their statutory obligations.
The Seven Station Series seems to have been introduced into NIWA's records, without formality, on some date in the early 2000's when James Salinger was Principal Scientist of the Climate Group. There does not seem to be any supporting documentation, it appears not to have been checked or assessed or peer-reviewed.
Was it approved by the Chief Scientist, and on what basis? If so, show us the documentation.
What we know is that the raw data show no rising or falling temperature trend. The rising trend in the official Seven Station Series is a result entirely of adjustments to the data.
You might say the warming was man-made. Analysis of the data shows that 90 per cent of the adjustments favoured a rising temperature trend.
A sceptic would naturally be getting suspicious at this point.
Conceivably these adjustments might be entirely reasonable. But we won't know, and can't know, until NIWA is able to publish an authoritative schedule of adjustments, and until independent scientists are able to check that work.
As the pressure was building from the Climate Science Coalition (CSC) on the Seven Station Series (SSS), NIWA released in December last year a new Eleven Station Series (ESS) which showed a similar rising temperature trend to the Seven Station Series.
The CSC analysed this data, and visited a number of the measurement sites. There have been site re-locations for these instruments; some have multiple and lengthy gaps in the data; some are "class 4" stations which have an error potential as high as 2 degrees. The CSC has grave doubts over the consistency of the series, as new stations are added from quite disparate locations.
When the 7 and 11 series data are overlaid, the differences are more pronounced than the similarities. Warming periods do not coincide.
There is a happy ending, or at least the prospect of one. After all the blocking, the defensiveness, the evasion, NIWA has - to its credit at last now announced that it will have a fresh start.
Essentially NIWA has conceded it does not have a schedule of adjustments. It is conducting a full review of the temperature series, and has committed to publishing a complete set of data and documented adjustments - in short, a schedule of adjustments.
At last.
Why does all this matter?
Well, it doesn't much matter what New Zealand's temperature trend is in respect of whether or not global temperatures are rising or falling. Our data doesn't strengthen or weaken the case for emissions control policies.
But the science does matter in respect of the politics of global warming. The science is used to motivate political agendas, to scare people into accepting changes that they would otherwise resist.
And how come our climate scientists didn't notice any of the problems with the IPCC reports? Were they asleep at the wheel or did they just decide to keep quiet about it?
All of this matters greatly for the integrity of science in New Zealand. It matters whether or not our taxpayer funded scientific institutions behave with some integrity, as well as doing good scientific work.
It matters when they consider themselves some sort of secret society, not open, not accountable.
And it matters if they disregard their statutory obligations to maintain adequate records.
There are heads that should roll in NIWA.
They have misled Ministers.
They have performed with reckless disregard for basic scientific standards and for their statutory obligations.
They have blocked perfectly reasonable requests for information.
And they have covered up the inadequacies of their own performance.

Wratt:
There were many lines of evidence which pointed to climate warming in this country, Wratt said in an NZPA report.
These included ship-based measurements and shrinking glaciers as well as land-based temperature measurements, such as those in Niwa's longterm 7-station series (of adjusted data) and an 11-station series (of unadjusted data).
Niwa has previously said the temperatures compiled from the seven locations -- Auckland, Masterton, Wellington, Hokitika, Nelson, Lincoln and Dunedin -- show "striking agreement" on temperature fluctuations between years and also on long-term trends.
They use data adjusted to made to allow for changes over the decades, such as when the position, type or location of thermometers was altered: Dunedin has the longest history, but its climate readings have been taken at six different sites.
The 7-station series indicated a rise of 1.1degC per century, and the 11-station temperature record showed a rise of 1.3degC per century.
Wratt rejected Mr Hide's allegation that warming periods shown by two key sets of measurements did not coincide. He released a graph of the two sets of measurements overlaid to show they had very similar trends.
"The warmer and cooler periods clearly do coincide," he said. This was despite the fact the graph was based on separate sets of data, only one of which had been adjusted.
Wratt rejected Hide's statement that he and a lobby of climate sceptics, the NZ Climate Science Coalition, had not been able to document details of the adjustments to raw data because there was no "schedule of adjustments".
Niwa had posted its 7-station temperature series online, and Wratt said that a schedule of adjustments required to create the series was also available from the web page.
"Niwa makes all its original climate data freely available over the web for anyone to analyse," he said. This had been the case since July 2007, and the adjusted 7-station series was made available to members of the Climate Science Coalition a year earlier than that.
Wratt said international climatologists accept the need to make adjustments in data --which was not the same as fiddling or manipulation -- and Niwa kept all the original, unadjusted data secure in the national climate database, which anyone could access free over the internet.
Niwa was further documenting the 7-station series, and would post those details on its website.

Sources:
Niwa says Hide's climate science attack wrong, NZPA May 6, 2010, http://nz.news.yahoo.com/a/-/mp/7186552/niwa-says-hides-climate-science-attack-wrong/
Self-Inflicted Wounds and Sloppy Science,Speech by Hon Rodney Hide, ACT Leader, to the Waikato Federated Farmers AGM, at Hamilton Airport Conference Centre, Hamilton, Thursday May 6, 2010