Transport planners using models that are 'not fit for purpose', says report

Transport plans heavily influence urban planning and design policy, yet major transport spending decisions across England may be being made using analysis from computer models that are not fit for purpose, according to an audit of regional and sub-regional models released by the DfT.

See experts talking about the role of modelling and people movement in case studies on RUDI

The review, led by respected modelling expert Denvil Coombe, examined 30 models and concluded that 21 were either unsuitable or had unknown suitability for testing any of four policy interventions (highways, public transport, parking and road pricing).

Coombe emphasises that the report represents a ‘snapshot’ of the state of play when the review was conducted in autumn 2008, in collaboration with ACS Transport Consultants and MVA Consultancy.

'Of the models which I currently regard as unsuitable, I am aware that several are undergoing substantial programmes of enhancement and I believe that others could be enhanced with relatively modest effort,' he said.

In a single page commentary published alongside the report, the DfT said it recognised that the review had 'highlighted a number of deficiencies in many models which need to be addressed'. Where a model cannot be demonstrated as being fit for purpose, 'the Department would have serious concerns over the use of the model concerned,' it added.

The research team compared models with Government guidance. In some instances they were unable to assess how compliant the models were because documentation was 'incomplete, unavailable... [or] unclear'. Among the problems identified by the team are poor model validation techniques and model structures and elasticity values that do not correspond to guidance.

Coombe told RUDI sister magazine Local Transport Today: 'The most important message I have is: consultants and clients should do more to keep themselves abreast of the guidance and to ensure that they follow it as far as it is practical to do so.'

Asked about the implications of the findings for scheme appraisals, he said: 'The sensitivity of the appraisal results to deficiencies in the model, some of which may be unavoidable because of limited data availability or budgets or timescales, should be demonstrated.'

The modelling fraternity had been braced for the report after Coombe revealed some of his headline findings at last year’s transport modelling forum organised by Local Transport Today. At the time he did not identify the models which he was referring to by name. The final report, however, includes critiques of each of the 30 models by name and recommends how weaknesses of particular models can be overcome.