Why the term 'ivory-tower intellectual' is an insult

Over at Wizbang there's a post on a symposium held at Swarthmore College, 'symposium' being a place where intellectuals superior to all others(they'll tell you so) gather to decide just how the world should be run by either themselves or someone they select. Here's the quote that inspired my title:
"At the conclusion of the first session, after a rousing discussion focused on promoting hardcore redistribution of wealth in the world, Mr. Barber opined:

The last point I want to make for now has to do with the nature of the discussion we've been having and what might be the political viability of the debates that we so-called "public intellectuals"engage in. When I try to imagine an African-American, unemployed guy from Detroit or Arkansas, an out of work (what did Howard Dean call him?) confederate guy with a shotgun in his pick-up truck, sitting here and listening to us talk, I realize that a number of the things we say in good faith and with impressive intellectual clarity and historical perspective are nevertheless politically unviable as ways of talking about the world. We haven't yet found a language that takes the vitally important, moral, philosophical and historical points we're making here and translates them into a language that ordinary Americans will understand."

I'll let that speak for itself, straight from the world of overeducated fools.