Stephen Littlechild, Professor emeritus at the University of Birmingham, fellow of Judge Business School at the University of Cambridge and a top regulator from 1983 to 1998, explains how politicians and regulators have, by misunderstanding how markets work, regulated to boost energy firms’ profits at the expense of higher bills for consumers.

Britain’s competitive retail energy market was the first in the world, and for many years the most competitive. It had the most active suppliers, and the most active customer switching. This competition and choice brought better offers for customers. It may not seem like it because of recent energy price increases. But these reflect increases in fuel costs like gas, higher costs of renewable energy and other obligations on suppliers, not a lack of retail competition.

In fact, retail competition was sometimes too fierce, witness the problem with doorstep mis-selling. But Ofgem took action to fix that problem.

Retail profits in the domestic sector used to be minimal; Ofgem calculated that many were negative. New entrants came into the market, but until recently most found it tough to survive.

Retail competition has been enhanced by a dozen switching sites. Each seeks the best way to attract users, to offer the simplest calculations, to include the most relevant information and the clearest comparisons, to facilitate subsequent switching. No other country can boast as lively, innovative and effective market for information and assistance to energy customers as Britain.

So what went wrong? Someone or something had to be blamed for the energy price increases. Ofgem was unable to find evidence of market failure. It concluded that the problem was customer failure. Customers were paying high prices because they were unable or unwilling to understand suppliers’ offers. So the market had to be simplified.

An increasingly bizarre series of proposals and directives has emerged from Ofgem, Government and now Which? magazine for dumbing down the retail market. All are well-intentioned, none shows any understanding of competitive markets, none will increase customer engagement, and all will make customers worse off.

Ofgem was first. It noticed that suppliers based in one area were offering lower prices to customers in their competitors’ areas. In 2009 it decided that requiring suppliers to charge the same price to all customers would bring the benefits of the lower prices to all customers – Right? Wrong. Suppliers predictably found it more profitable to raise their low prices to new customers than to lower their prices to existing customers.

Customers suffered because the low-price offers were withdrawn. They also began to lose interest in switching supplier: the switching rate has since fallen by nearly a half. But suppliers did not lose out from this reduction in competition. Quite the opposite: Ofgem’s calculations show their retail profit margins increasing to an all-time high: from minus £10 per dual fuel customer in May 2009 to about £50 from 2010 to 2012 to £100 now.

In 2011 Ofgem proposed that all suppliers should offer the same monthly standing charge – which Ofgem itself would specify. It overlooked – or didn’t care – that this prohibited tariffs with no standing charge, which are popular with pensioners. And that Ofgem would now be jointly responsible for setting energy prices. Ofgem withdrew its proposal.

Meanwhile, suppliers found other ways to compete – for example by offering lower prices online. Customers benefited – until Ofgem decided that this made the market too complicated. In October 2012 Ofgem proposed that suppliers would be allowed only four tariffs per fuel. This of course is tough on customers with minority tastes, like green tariffs, or even tariffs with no standing charge.  And innovation will cease if a supplier can only innovate by withdrawing an existing tariff that supplies about a quarter of its customers. But now it’s simplicity that counts, not the availability of products that customers want.

There are other petty restrictions. Discounts must be the same each year, expressed in pounds not percentages. If this restriction had been in place, it would have banned the best offer in the market earlier this year. And in future it may not be viable for suppliers to offer discounts that don’t use percentages to tailor the discount to the size of bill. But the availability of good offers is no longer a relevant consideration.

At the same time, another bright idea popped up at Prime Minister’s Question Time. Just in time for the County Council Elections. The campaign leaflet says “Conservatives in Government have forced energy companies to put customers on the lowest tariff”. Leave aside that this is not yet enacted, and still just an idea that Ofgem might reluctantly trial. Leave aside too why Conservatives in Government and not Ofgem are now regulating energy companies. Let us just ask: what does it mean and is it a good idea?

If it means that energy suppliers will be forced to put their own customers on to the lowest tariff offered by their rivals, is there the remotest chance of this working? Suppose it means that energy suppliers will be forced to put customers on the lowest tariff they themselves offer. But if a supplier offers a discount on its standard tariff coupled with an exit charge of £50, do we really want to force that supplier to put all its customers on a tariff that locks them in? And if a supplier wanted to offer a discount for new customers, but was forced to put all its existing customers on the same discounted tariff, isn’t it obvious that it would be more profitable not to offer the discount in the first place? Once again, the proposal will drive out the best offers.

The latest bonkers suggestion is from Which? magazine. It says that Ofgem’s proposals for simplifying the market don’t go far enough. The government should require single unit prices for each energy tariff, like petrol prices on a garage forecourt. Simplicity is flavour of the month, and petrol is a competitive market, so what’s wrong with this? Lots.

First, Which? seems to be asking for a single uniform price across the whole of the country. But distribution network charges vary considerably across the country. To impose a uniform retail price or network charge would require massive geographic cross-subsidisation between network operators and between customers that would be neither workable nor obviously equitable.

Second, forcing all tariffs to have a zero standing charge would mean that suppliers would not be allowed to offer a lower unit price to larger customers that are more economic to serve, and suppliers would no longer be interested in attracting smaller customers. The likely impact on different kinds of customers has not been considered.

Third, limiting the variety of energy products so that customers are faced with only one price per supplier would enable and encourage suppliers to coordinate prices. If one supplier breaks ranks then other suppliers will either follow or that supplier will fall back into line. Customers will find that suppliers offer similar prices almost all the time. Where then is the incentive to engage in the market?

All these schemes assume that regulators and governments know more about customers than those who make a living by discovering and providing what customers want. These schemes won’t really simplify the market and they won’t persuade customers to engage more. But they will restrict competition, and customers will be worse off because the best offers will disappear. Suppliers will find it costly to comply with the proposed 126 pages of new regulatory red-tape, but the costs will be passed through to customers and the suppliers will grumble all the way to the bank.

Britain’s claim to have a leading competitive energy market and regulatory body is no longer tenable. “If it ain’t broke, break it” may sound like action, but it will not, ultimately, be persuasive to customers.