As we all know little snippets of information get bandied about in the public conversation and then take on the aura of incontrovertible facts. George Osborne, in a political speech, claims that raising the minimum wage does not cause job losses. That is taken to be true by all too many, when the actual official report accompanying the speech insists that the latest rise will cause 60,000 job losses.

"See, Osborne said it" is not what we would take to be a great standard of proof but it's enough for some people, sadly.

What really interests us here though is what people will take as the standard of success. To take an example from this morning's papers, Laurie Penny tells us that gender quotas in organisations make those organisations better:

Plenty of people do, according to another study, this time from the London School of Economics. It showed that companies with a strict 50-50 gender quota performed far better, partly because it meant men were expected to work harder to prove themselves – and fewer mediocre men ended up in positions of power.

Well, no, that's not what the study shows actually. It's about gender quotas inside a political party:

Our study provides a unique window on quotas and, at the same time, pushes forward the measurement of competence in political selection. It uses the fact that, in 1993, Sweden’s Social Democratic party voluntarily introduced a strict gender quota for its candidates.

We wouldn't insist upon it but we'd be willing to wager that political parties work a little differently from organisations where output can be measured in detail.

However, we will admit that their measure of competence is interesting. By looking at the wages of candidates (adjusted for experience, job, education etc) they show that insisting upon 50/50 quotas raises the general level of candidate. The assumption is that someone earning over the odds in the private (well, they include public sector workers as being "private" here but still) sector is likely to be more competent than someone not.

Hmm, OK. But let's go back to some measure of the actual output of the organisation. We seem to have evidence of the quality of the inputs increasing. But what of the quality of the output?

Well, the year after this change marked the peak of the vote for the Social Democratic Party in the national parliament elections. It's fallen from the 45% achieved in 1994 to 31% at the last election. We're deeply unsure that that is a useful measure of an increase in the performance of the organisation. The point of political parties being, we're really quite sure of this, to gain and exercise power.

No, we most certainly do not claim causality here but we do still think this is a useful example of how these stories don't get the examination they probably need. The claim being made is that gender quotas make companies more efficient, the proof being offered is that a political party with gender quotas loses 33% of its vote. The more detailed look tells us that inputs seem to have increased in quality while output has decreased.

These stories do need a little more examination, a touch more investigation into what is success, don't they?