Sir Richard Branson, founder of Virgin Group, and the runner up in bidding for the West Coast franchise, was due to take the Department for Transport (DfT) to court the following day as he claimed the bidding process was flawed, particularly in the assessment of risk and the level of financial guarantee First Group was being asked to make.

Basically, the two franchise bids by First and Virgin were remarkably similar, with steadily rising premium payments to the DfT except during the last two years of the franchise when First was offering substantially more than Virgin. Branson, who lost out twice in bidding for the East Coast Main Line franchise, first to GNER and later National Express, under what he claimed were similar circumstances, only to see both franchises fail, feared a similar situation on the West Coast. Naturally First Group strongly defended its bid and stood by its passenger and revenue forecasts.

Up to September 12, both McLoughlin and his predecessor Ms Justine Greening, said that the franchising process was robust and that the DfT would robustly defend its decision to award the 13-year four-month franchise worth £5.5bn (net present value) to First Group.

However, flaws concerning the level of risk in the franchise bids, together with mistakes in assessing passenger numbers and inflation, and the financial guarantees requested, meant that McLoughlin had no choice but to cancel the whole 15-month competition and reimburse the four bidders, expected to be around £40m. He also announced that the DfT will no longer contest the judicial review of the franchise bought by Virgin.

On top of this, the DfT has wasted several million pounds of taxpayers money in assessing the bids, and could face claims for damages from bidders, especially First Group which suffered a £230m drop in its share value following the announcement that it will no longer take over the franchise from Virgin. McLoughlin now has to decide who will run the trains after December 9.

McLoughlin has also halted the bidding process for three more franchises currently underway, and suspended three DfT officials, while an enquiry into what went wrong is conducted, with an initial report expected by the end of October. In addition McLoughlin has asked Mr Richard Brown, currently chairman of Eurostar and a career railwayman with experience as both a British Rail manager, and a franchise manager, to report by the end of the year on more fundamental issues concerned with franchising, namely the assessment of risk, the whole bidding and evaluation process, and how to resume franchising. If Brown recommends changes, these will inevitably take time to implement and translate into new invitations to tender for West Coast and the three franchises on hold. This could force the DfT to negotiate extensions to these franchises, as well as other franchises in the pipeline.

The DfT has a stark warning on its website to franchise bidders: "Franchisees must build resilience into both their operational and financial plans to deal with the changes in the economic environment to which a passenger rail operation may be subject." The DfT also warns that will it not bail out franchisees that get into financial difficulties. Hence the need to agree deals that will survive fluctuations in the economic cycle.

The West Coast franchise was supposed to be the first of a new breed of franchises with a longer life, typically up to 15 years rather than seven which is the norm today.

Franchises let between 2004 and 2009 contain a revenue risk sharing mechanism which offers protection after the fourth year if revenue fails to reach the target set. This is the so-called "cap and collar" arrangement, and 11 out of the 16 franchises are already receiving financial support from the DfT. The DfT was unhappy with this, and decided that revenue risk sharing mechanisms will in future be designed on a franchise specific basis. For instance the new West Coast franchise was to contain an adjustment mechanism linked to fluctuations in GDP growth which would have cushioned the franchisee against a major downturn in revenue due to exogenous economic factors. Likewise the DfT would share profits in excess of forecast.

Is the decision to cancel the West Coast contract, and put the other tenders on hold, merely a result of flaws in the system and mistakes made by officials, or does it point to deeper problems? This is an ideal opportunity to consider whether more fundamental changes are needed.

When franchising first started in 1996, the 25 franchises were of three basic types: regional requiring subsidy, London and southeast England commuter services which might require subsidy initially but which had the potential to make premium payments, and inter-city which were expected to become profitable. Franchisees were free to innovate and introduce new services above the minimum requirement even outside their area. This led to considerable innovation with many services being introduced between places which had never been connected directly by rail before. Franchisees were not expected to invest a great deal, as the trains would be leased from rolling stock leasing companies (Roscos) and the infrastructure was the responsibility of Railtrack.

But over time, the DfT has modified the franchises to eliminate competition between them, thereby ending services outside of the franchise area, and to introduce an element of cross-subsidy by merging loss-making regional or commuter services into profitable inter-city operations. Great Western, for example, now comprises all three. As a result, there are now only 16 franchises for England and Wales.

The DfT has also tightened its grip on the operation of franchises to the extent that it is now very difficult to adapt train services to meet changes in demand, let alone introduce new ones. And to make matters worse, the DfT is entering into new and expensive train deals which will lock operators into using a specific type of train for years to come with no control over the cost.

Is it reasonable to expect companies to forecast passenger demand and revenue, as well as the effects of inflation, changes in the economy, and competition from other modes well into future? How many people predicted the current economic downturn and financial crisis five years ago, let alone 15 years ago?

Two of the franchises currently on hold – Great Western and Thameslink – are extremely complicated. Great Western will be affected by the Crossrail project at the London end, the reconstruction of Reading station, electrification of part of the network, and introduction of the controversial Intercity Express Programme (IEP) fleet during the life of the next franchise. Thameslink will be affected by major construction work, and the incorporation of the Southern franchise and part of the Southeastern franchise at some point during its life.

Is the one-size-fits-all approach to franchising still appropriate? The government wants to reduce the cost of running the national railway, but it currently costs up to five times more than under British Rail. Only a proportion of franchises go the full term meeting their financial commitments to the government, and so the promised flow of funds back to government rarely materialises. Should other types of contract now be considered for regional services which will never be profitable? The concessioning system, common elsewhere in Europe, is designed to reduce the burden on taxpayers while offering passengers a better deal, and it seems to work.

Do we actually need franchises for profitable intercity services? East Coast, which is currently being run by the DfT, operates in competition with two open-access operators, First Group's Hull Trains and Grand Central, now part of German Rail's Arriva Group. Another Arriva subsidiary is seeking access to the West Coast route.

The introduction of direct competition on Italy's high-speed network has encouraged the incumbent to up its game considerably, and increased the size of the overall rail market to everyone's benefit. Why not try something similar in Britain? Wouldn't it be better to have regular payments from operators, all operating on an equal footing, rather than the high risk strategy offered by the current flawed franchising system? The government will also generate income out of the operator's profits.

Clearly, it is time for a fundamental rethink of franchising in Britain, and now is an excellent time to do it. But will ministers be bold enough to take the plunge?