What kinds of inequality should we be worried about?

Most political theorists are egalitarians of some sort. While I personally find Derek Parfit's argument in "Equality and Priority"—that egalitarianism sometimes says making others worse off without making anyone better off is good in one way, or even required—extremely convincing, and hence call myself a "prioritarian", I have trouble dealing with the arguments in Michael Huemer's "Against Equality and Priority". Nevertheless, I am very sympathetic to the basic claims of luck egalitarianism, i.e. that those advantages in life that are down to pure luck are undeserved. Combined with the fact that others are in desperate need, there is a strong case for redistribution before other complicating factors are brought in. But even egalitarians should (and often do) favour wealth or income inequality in the three following cases.

1. When inequalities come about as a result of different levels of effort. Some people are born with vast natural talents (e.g. Wilt Chamberlain) while others are not, or their talents are not in such high demand by the market. Some are born to dedicated, loving parents while others are raised in far less supportive environments. No one could claim they bear most of the responsibility for their genes or upbringing. But even if we could even out the differences in income or wealth due to different upbringings and talents, we'd want to leave in the differences from different levels of work. This is because leisure can be seen as a form of income, as it adds to utility. To give those who take more leisure the same money income as those who take less would be subverting equality, rather than enforcing it.

2. When inequalities derive from differences in job satisfaction or riskiness. People who do more dangerous jobs are paid more. This is exactly what economists would expect; extra money compensates the worker for the extra risk of injury or death. But it's also what we should want. A more satisfying, less risky job (like teaching or creating art) should pay lower by justice, and this is one of the really good and egalitarian elements of the market economy. This ties in with the previous point as one extremely undesirable element of certain high-paying jobs is the extreme hours they demand. If typically people's willingness to do extra hours begins to decline at an accelerating rate, we would expect high hours occupations—in a just, egalitarian system—to be paid disproportionately well.

3. When inequalities are necessary, due to the infirmities of current human nature, to produce a greater total pot to help the needy. A prominent element of John Rawls' A Theory of Justice, the book that kick-started the recent era of social justice theorising, was the difference principle, the idea that inequality in society should only be as much as is necessary to make the worst-off person as well off as possible. But driving inequality any narrower would harm the worst-off and would thus be unacceptable. Of course, as G.A. Cohen shows in "Incentives, Inequality and Community" and his book Rescuing Justice and Equality, this isn't a demand of justice—it's just a practical consideration when we have the welfare of the badly-off in mind. But in the real world pragmatic considerations are often appropriate, and it may well be that certain inequalities in a given society can be practically justified on these grounds.

The really interesting question is this: how much of the inequality in real-world capitalist societies is down to these three legitimate sources, and how much is down to undeserved luck?  The key difference between 'bleeding heart' libertarians and traditional left-wingers may come down to this crucial (empirical?) question.