Uncharted Territory

November 22, 2009

Beware Proxy Problems and Correlation Cons

Filed under: Complex decisions, Global warming, Healthcare, NHS, Politics, Reflections, Science — Tim Joslin @ 7:48 pm

I receive email notifications of new posts on the Realclimate blog, a forum for discussion of the science of climate change, run by real climate scientists! Usually there is one post every few days, so I was slightly surprised to be notified about a second post last Friday (20th).

I was stunned when I saw that the second post was on the topic of the release of internal emails from the Climate Research Unit (CRU) at the University of East Anglia (UEA) in Norwich, England.

A single passage in tens of megs of emails has become the focus of the mudslinging. The Realclimate guys have this to say:

“No doubt, instances of cherry-picked and poorly-worded ‘gotcha’ phrases will be pulled out of context. One example is worth mentioning quickly. Phil Jones in discussing the presentation of temperature reconstructions stated that ‘I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.’ The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term ‘trick’ to refer to a ‘a good way to deal with a problem’, rather than something that is ‘secret’, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the ‘divergence problem’–see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend [sic] not using the post 1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.”

I’m afraid the explanation of the use of the word “trick” makes me squirm! And to say the data were manipulated “to hide the decline” is unfortunate to say the least. The Guardian notes that “[t]he scientists [sic] who allegedly sent it [the ‘trick’ email] declined to comment on the email.” Well, if you ask me, they ought to be commenting, PDQ.

The ought to comment, because it’s important to get to the bottom of the issue. I have no doubt we are seeing spectacular climate change. What bothers me, though, is how little we know about past climates.

A couple of weeks ago I mentioned my puzzlement that the scientists are now saying that temperatures were considerably (3-4C) higher than at present during the last interglacial. Then, last week, I read this in the Telegraph:

“Louise Sime, lead of the British Antarctic Survey study, looked at ice cores to see how temperatures changed during periods of high carbon dioxide[.]

She found that during the last period of high CO2, 125,000 years ago [125 kya], temperatures were up to 6C higher than present day levels.

Such a hike in temperature could lead to a rise in sea levels of between 4 to 6 metres over hundreds of years as the ice sheets melt.

‘We didn’t expect to see such warm temperatures, and we don’t yet know in detail what caused them. But they indicate that Antarctica’s climate may have undergone rapid shifts during past periods of high CO2.’

Dr Sime said the study suggests that current high levels of CO2 could also cause a rise in temperature. She said further research could predict the affect on sea level rise.

‘If we can pin down how much warmer temperatures were in Antarctica and Greenland at this time, then we can test predictions of how melting of the large ice sheets may contribute to sea level rise.’ “

It might be worth pointing out that the “high CO2” 125 kya was nowhere near as high as it is now – 300ppm tops, compared to ~390ppm today (and the other greenhouse gases [GHGs] we’ve emitted make the present situation even worse).

The point is that if the climate system is more sensitive to elevated CO2 levels than we think, we have to revise our targets, as I’ve pointed out before.

If the more recent Medieval Warm Period (MWP) and Little Ice Age (LIA) were real events then we need to find out exactly what caused them. I suspect changes must have been triggered in the ocean circulation. Maybe we simply haven’t yet been cooking the planet long enough to disturb the system, or maybe, as I suggested before, the continued warming counteracts the planet’s normal negative feedback response to a period of warming.

What bothers me about the CRU leak is that it makes no sense at all to me to use a proxy for the temperature record when you actually have an instrumental record (or can even construct a record from historical documents). The instrumental record should be used to determine which proxies are valid for dates earlier than you have records for. It sounds as if the link between one possible proxy (Briffa’s tree-rings) and temperature doesn’t hold, so that particular proxy should simply be discarded altogether, not just for the period from 1961.

Lots of proxies have been used to reconstruct past temperatures, which is clearly a seriously complex and difficult exercise. Maybe the scientists need to explain a little more clearly exactly what these “temperature” series tell us.

The CRU hack controversy is a bit of a shame because it’s completely overshadowed the earlier Realclimate post on Friday. A Problem of Multiplicity currently has just 28 comments compared to 913 and counting for The CRU Hack, but in fact makes a much more important point.

If I interpret it correctly, A Problem of Multiplicity basically points out that if you compare enough sets of data you’re bound to find some correlations. This is not entirely disconnected, of course, from the problem of reconstructing past temperature records, though the area of research being criticised is the persistent attempt (often associated with a global warming scepticism agenda) to identify possible effects of solar cycles on climate.

I remembered A Problem of Multiplicity when I read Ben Goldacre’s Bad Science column in yesterday’s Guardian. It’s obvious, now Realclimate has pointed it out, that just by chance any drug is going to be associated with some side-effects. What’s needed is to use this initial detection as a hypothesis, and examine an entirely different set of patients to see if a statistically significant correlation is found. Tricky business. Maybe the NHS (or other organisation representing patients, not Big Pharma) should provide a website detailing exactly how possible side-effects have been determined. Because if you worried about everything listed on the leaflet in the packet you’d never take anything.

Back to climate. I don’t envy the scientists their job in trying to get their message across. I’m beginning to suspect the message needs to be a lot simpler. Especially when they’re up against this sort of thing from David Bellamy:

“I’m sceptical about man-made climate change. There’s absolutely no proof that carbon dioxide will kill us all. It’s not a poison, it’s the most important gas in the world. Carbon dioxide is an airborne fertiliser. How can farmers grow increasing amounts of food without a rise in CO2?”

Quite easily. Plant growth is rarely limited by CO2 availability, since they have adapted to the level that’s been in the atmosphere for the past 20 million years or so. Much better ways of improving plant growth are to improve the availability of other factors, e.g. water and mineral nutrients.

As I see it, responsible citizens have a choice. They can either accept the scientific consensus or they can delve deeply into the science themselves. I’m afraid I don’t see a lot of middle ground.

If people do decide to get to grips with the science they won’t be unduly alarmed by the dumb things scientists sometimes do. Just like the rest of us. Let alone the myriad mistakes regularly made, just as a random example, by economic policy-makers, their political masters and, of course, bank executives.

Informed responsible citizens will also realise that science is never the finished article, but continually evolving. Quite interesting really.

Unfortunately, as the science gradually changes, so must policy. And it seems to be becoming fairly clear that our targets for safe CO2 (and other GHG) levels are far too optimistic.


October 28, 2009

NHS Nutters

Filed under: Healthcare, NHS, Politics, UK — Tim Joslin @ 10:48 pm

The British electorate would be well advised come next June to forget about traditional political affiliations and whether or not we “need a change” and choose their next Government on the basis of the apparent sanity of its leadership.

Unfortunately, the British electorate is unlikely to take my advice and will instead elect a party which admittedly does boast some front-bench talent. But the wit of William Hague and wisdom of Ken Clarke will have to play second-fiddle to the Etonesque eccentricity of Messrs Cameron and Osborne. The key word in their bizarre philosophy is “localisation”.

Here’s a headline: “Tories vow to save money by scrapping national NHS database“. Having spent billions so far, this would seem a remarkably drastic step. The Tories are responding mainly to concerns about patient confidentiality, it seems. Instead of a centralised database, “Stephen O’Brien, a shadow health minister, said that the Tories would instead decentralise IT provision in the NHS, allowing trusts to buy their computer systems provided that they were compatible with others in the health service.”

Now, it seems to me that one reason the NHS IT project has overrun is because it is so monolithic. It would have been much less risky in many ways to set standards so that disparate systems could communicate, and implement them locally. But I see why a “key part of the programme involves the clinical records for every patient being stored on a ‘personal spine information service’.” If you have local systems, the data exchange issues are much greater.

One of the major benefits of computerisation is that patients’ records can be called up wherever they happen to be. And there’s also a huge benefit of computerisation in that the data is so much easier to “mine”. I read today of a careful study on migraine-sufferers. “Link between migraine and stroke” said the Independent. Worrying, so I read further:

“Young women who suffer from migraines with visual disturbances and who smoke and take the contraceptive pill are at a higher risk of stroke, research suggests.

Migraine doubles the chances of a stroke if accompanied by aura (temporary visual or sensory disturbances) according to the research, published online in the British Medical Journal.

Other factors that heighten the chances of a stroke include being younger than 45, a smoker and using contraceptive pills containing the hormone oestrogen.

Researchers led by a team from Harvard Medical School said there was no evidence of an increased risk of stroke among people having migraine without aura.

About one in five people suffer from migraines, with up to a third having an aura. The authors pooled the results of nine previous studies on the link between migraine and stroke to come up with the findings. “

Maybe the drugs prescribed for the migraines have caused the strokes – especially as my impression is that not so many years ago you needed the aura to call it a migraine and qualify for the drugs. Who knows? Hopefully the medical sleuths are on the case.

Anyway, my point is that with a comprehensive national database – however it’s implemented – you wouldn’t need elite medical professionals to determine correlations of this kind. School-level statistics skills would be sufficient. The big pay-off from an NHS database is this sort of data and the resultant medical progress. You want to work the database. That, I suspect, is why a centralised design was chosen. (And I haven’t even mentioned Shipman – detecting similar individuals would surely be worth the cost of a few computers).

How do the Tories deal with this issue?, I wonder. This is what their report has to say (apologies on their behalf that parts of the key passage are barely comprehensible gibberish):

“In order for patients to reap the benefits of information technology in relation to their healthcare, there must be a change in the way information technology is supported: the Executive must not regard health informatics [they’re using this grandiose term confusingly – to me at least – to mean all NHS IT services, whereas I’d use it specifically to refer to knowledge engineering] as a tool to extract data [as if it’s being stolen – isn’t language great?] from the National Health Service but as a way of organising health and social care information around the needs of the patient [as if sharing data across the system isn’t in patients’ interests]. Systems must be able to deliver clear benefits to the care of the patient and the work of the clinician in delivering this care. They must not be seen by clinical staff as solely [sic – really, would they?] systems for data collection. The dataset mentality – where the bulk of data collected bears not [sic, I presume they mean “no”] relevance to patient care – should be abandoned. Clinical systems should be built to focus on the patient, not the disease, procedure, specialty or service providing care. These requirements should be met by developing appropriate views on the patient-focused record according to the context in which the patient is seen.”

Actually, it’s not just the words I’ve highlighted which are gibberish, the whole thrust of this paragraph is completely barmy. Call the men in white coats. Several of the individual sentences make little sense, but I can find no way of avoiding the conclusion that the Tories want to throw out many of the enormous benefits of computerisation. Cameron apparently elaborated the full mad starey-eyed “vision” at the Tory Party conference:

“Now I want you to imagine how we’d have gone about [updating NHS computers], if we’d had the chance.

We would have said: today, you don’t need a massive central computer to do this.

People can store their health records securely online, they can show them to whichever doctor they want. [I love this bit – not if you’re in a coma, you can’t].

They’re in control, not the state. [Puerile, absolutely puerile].

And when they’re in control of their own health records, they’re more interested in their health, so they might start living more healthily, saving the NHS money. [Yeah, well, now I know the doctors are going to wait for me to come round so I can log them on to my medical records, I’m certainly going to do everything I can to avoid entering hospital horizontally].

But best of all, in this age of austerity [oh, please], a web-based version of the government’s bureaucratic scheme services like Google Health or Microsoft Health Vault cost virtually nothing to run. [Probably ‘cos they’re crap as Gerald Ratner would have said].

So this is where some really big savings could be made.”

Well, exactly. I can save the tube fare if I walk 8 miles to central London.

David Cameron, of course, is a direct descendant of Thatcher – as if through the intervention of some kind of incubus – and for her “there was no such thing as society”. This is now dressed up as a philosophy of “localisation” – the heading in their report is “8 action points to bring about localisation in NHS IT” – who could possibly be against that? Well, I am, if it means that I don’t benefit from the medical experience of others, and the outcome of any treatment I have is not used for the common good. Hey, why not go the whole hog and give up on the blood transfusion service? We could all have our own private blood-banks.

My experience is that the NHS is already far too localised. GP surgeries are a law unto themselves. Take registering. My local GP, first of all, only accepts registrations up to 4pm, despite being open until 5; they require two proofs of address + photo-id; and, worst, refuse to let you take a form to fill out and bring back when you get there and find you don’t have the right id, so you find when you eventually do come to fill it out that you don’t have all the information required (NHS number and former GP surgery address). Plus, they’re just plain rude with it. Our taxes are paying for an army of these receptionists, who have no medical training whatsoever. This is where we should find our first cost-savings when we finally get our records out of the 1940s, off paper and onto magnetic media.

My recent experience of trying to register with a new doctor (having moved recently) agitated me so much that I thought I’d try to find out how I should be treated.

Well, try as I might, I could find nothing on the NHS website. Surely, I thought, GPs surgeries benefit hugely from being part of the NHS, there must somewhere be a detailed set of obligations as to how they should go about their business. In the end, I thought, I’ll drop them a line. But this is what I found:

“Since April 2009, the NHS has run a simple complaints process, which has two stages.

1. Ask your hospital or trust for a copy of its complaints procedure, which will explain how to proceed. Your first step will normally be to raise the matter (in writing or by speaking to them) with the practitioner, e.g. the nurse or doctor concerned, or with their organisation, which will have a complaints manager. This is called local resolution, and most cases are resolved at this stage.
2. If you’re still unhappy, you can refer the matter to the Parliamentary and Health Service Ombudsman, who is independent of the NHS and government. Call 0345 015 4033”

This is simply not fit for purpose. Neither of these options is appropriate.

Here’s the first paragraph of “Things to think about when you’re complaining” (from the NHS complaints page):

“If you decide to make a complaint it’s important to consider what you want to happen. Are you content with an apology, do you want action to be taken against a member of staff, or do you want a change to the system? Whatever action you’re seeking, make this clear.”

Well, someone, somewhere has realised that I might want “a change to the system”. So why has my ability to complain been “localised”? My only recourse it seems is to write to the Department of Health.

For the record, I would have thought the GP surgery registration process should follow a clear set of rules so as not to exclude those unable to attend during the day and expedite the registration of new arrivals to the area.

Otherwise, the right to choice in the NHS is somewhat limited.

Blog at WordPress.com.