Because of the weather this July, I’ve decided to work at home today. I’m keeping one eye on the window and another on the Met Office’s snazzy new weather radar map (select location, tab:map and button:rainfall) – no, I don’t know either why they spent no doubt a fortune changing something that was entirely adequate. And I suspect it uses more of my scarce computing resources than the previous version, which would have informed me perfectly well that I’ll have to bring the washing in in an hour or two. It’s been a rare opportunity to get that chore done!
It’s been nagging away at me to follow up my previous post which noted how much rain we started having as soon as drought was declared.
It’s not original for me to note the failure of the Met Office’s 3 month predictions. We had the wettest June as well as the wettest April on record, of course. Watts Up With That saw the same Met Office report for April to Jun (AMJ) (pdf) that I did and adequately contrasts this with the weather that actually occurred in April. Not A Lot of People Know That, Autonomous Mind and Saxon Times also make hay, figuratively speaking only of course.
I questioned the wisdom last time of extending the region of official drought when it was already raining, around April 16th. It seems this may have been done at least in part on the basis of the Met Office’s AMJ precipitation report (pdf). Here is the crucial graphic:
What this shows is series of scatters (the columns of x’s and +’s) of the precipitation in April on the left and for April through June on the right that actually occurred in previous years and was predicted for 2012 by a series of model runs.
Now, “scatter” is the operative word, since the model predictions are all over the place. What this tells me is that the models used have very little skill in predicting precipitation for a given year. That is, they don’t really do any more than predict the rainfall we generally experience. Perhaps the Met Office refuses to accept this and believes its models do predict something meaningful one to three months ahead.
Nevertheless, as shown by the graphs on the right of each plot, the model predictions were slightly biassed towards drier conditions in 2012.
It’s also possible to note that the models seem to be reasonably well calibrated in that they predict the sorts of amounts of rainfall that occur in the real world. To say anything more, we’ll have to assume that the models are perfectly calibrated, that is, they aren’t biassed towards predicting less (or more) rainfall than actually occurs, i.e. they don’t always tend to predict less (or more) than average rainfall for a given month regardless of the initial conditions.
Why then, should the models have forecast less rain in 2012, when, as we know, there was rather more than usual?
Here’s my hypothesis: it’s all to do with persistence of the initial conditions. The point is that weather conditions tend to continue for varying amounts of time. If it rains today it’s more likely to rain tomorrow than if it’s dry today. And the models are capturing this aspect of the weather.
An analogy for the financially inclined would be momentum investing. Shares (and other financial instruments) appreciating in value tend to continue to do so. Many investors base their strategies on this simple fact, and indeed it works well. Until the bubble bursts, of course.
This persistence in the weather system is exactly the reason why I was so critical before of the decision to extend the drought when it was actually raining. The Environment Agency should have waited to see how long that rain was going to last. They could at least have given themselves a sporting chance of not looking like idiots by waiting until a few dry days were forecast before imposing (or causing the water-supply companies to impose) water-use restrictions.
The timing of when forecasts are prepared is therefore important. The 3-month AMJ outlook (pdf) was issued on 24th March, presumably based on model runs up to a week or two earlier. The initial conditions for these model runs would have implied dry weather in late March, since the forecast for up to 10 days has some reliability. And, because the models have a tendency to predict persistence, they would, on average, have tended to predict drier than usual conditions into April. After that I expect they would have tended to regress to the mean – as evidenced by the more average weather in the 3 month plot than in the April plot in the graphic above.
In other words, it may be possible to predict, at least qualitatively, just by looking out the window, the results of running suites of these long-term forecasts.
Running the model repeatedly doesn’t yield the truth, but merely reveals any systematic bias – in this case inherent in the methodology. All reminiscent of the modelling of floods in the UK I looked at a while back, where the exercise simply reinforced the underlying physics that a warmer atmosphere holds more water, which was the systematic bias in that case.
By way of validation of my hypothesis, we can look at the forecasts for rainfall in March through May. According to the Environment Agency report, “Drought prospects for spring and summer 2012” (pdf) referenced by the BBC on 12th March when the first hosepipe bans were declared, the latest 3 month Met Office forecast was taken into consideration (see Chapter 2).
The Met Office forecast of precipitation from March through May (MAM) (pdf) is dated 24th February and we can see a similar pattern to that for AMJ:
Note again the scatter and the regression to the mean for the longer-term forecast.
Here’s another one, for July through September (pdf), but this time when the initial conditions were set during the wet June:
Surprise, surprise – now we’re expecting the wet weather to continue.
PS Don’t worry, I brought the washing in a few minutes ago – there’d been just a few drops of rain. It’s falling steadily now, though.
Uncharted Territory 