It's rather comforting to find one of your own heartlfelt prejudices backed up by a man with a Nobel, as I do here with Gary Becker's thoughts on the existence of bureaucracies:
Whether an organization is “efficient” cannot be defined in any absolute sense, but only relative to feasible alternatives. Therefore, it is reasonable to conclude that a large bureaucratic organization is efficient if it manages to thrive in a competitive sector; that is, a sector with easy entry of organizations with different decision-making structures. For if potential entrants were more efficient than the bureaucratic organizations, they would enter the sector and out-compete the bureaucracies.
Banks, oil companies, and manufacturers of large building equipment, to take a few examples, are in industries without major artificial restrictions on entry of competitors. Large bureaucratic firms, such as Caterpillar, JPMorgan Chase, and Exxon, persist profitably in these industries, sometimes alongside much smaller firms, like small banks, small equipment companies, and wildcat oil drillers that are generally more nimble. The persistence of these large bureaucratic companies suggests that their net advantages, taking into account their greater rigidity, are sufficiently great to enable them to survive the competition of smaller and more flexible firms. This is an application of the “Survival Principle” approach to efficiency developed decades ago by the Nobel economist George Stigler (see his article, “The Economies of Scale”, Journal of Law and Economics, October, 1958).
We might want to make sure that these firms are not using size in order to influence the regulatory environment, thereby gaining rents of course. But yes, this is a point I have made here several times, that as long as we have a market in organisations then we don't have to worry too much about which type of organisation is doing whatever job.
We can also extend this a little by thinking of Coase's "Theory of the Firm" where he points out that whether something is done by a firm, rather than a network of contracts (and by extension, by a firm, a bureaucracy or whatever) shouldn't particularly bother us. For, given the technology available at that time and place we'd expect competition to give us the most efficient method of performing that task. The organisational form therefore becomes a function of the technology available, market competition being the thing that adjusts such organisations.
Make sure we have ease of entry into a sector and we can pretty much leave the rest of it alone.
We're still left with the problem of why there's only one government at any time of course but that's a rather more intractable problem. Perhaps best solved by making sure that we've ease of exit from its clutches.