"The genuinely prominent defectors from libertarianism in recent years, have actually gone not to the right, but to the center and left. The most notable are probably Jerry Taylor, Will Wilkinson, and some of their associates at the Niskanen Center. I took issue with Taylor's rejection of "ideology" here, and Wilkinson's views on democracy and libertarianism here and here. Taylor and Wilkinson are important figures, and we should take their critiques of libertarianism seriously (as I have tried to do). But, so far at least, their shift has not triggered a more general exodus from libertarianism.
Various measures of the number of libertarian-leaning voters in the general public show that their numbers are roughly the same as they were 15-20 years ago (somewhere between 8 and 20 percent, depending on which measures you use). The number of self-conscious, rigorously consistent libertarians is surely much smaller. But the same can be said for adherents of other ideologies. Many studies show that most voters don't take a carefully consistent and rigorous approach to political ideology, and often don't even understand the basics of those world-views."