As it happens, The Peter Principle was the first book I ever read about business, back when I was about 15. I can't remember where I found it, perhaps left lying around by a relative or in the 25 cent bin at the used bookstore where I bought all my fantasy novels. It made a great impression on me that still lingers. As the New Scientist piece points out, academic research since 1969 supports the basic finding that many managers are incompetent. They get promoted because they are good at the jobs they used to do, which may have very little to do with managing. My first summer jobs were as a groundsman at a series of apartment complexes, and I observed several times that being a good maintenance man has little to do with being a good manager of a big complex. I also met one guy whose career had been a yo yo of promotion and failure, because he just wanted to be a maintenance man and yet he was so good at it that people kept promoting him to manager, something he hated. By the time I met him he was vowing never to accept any promotion ever again.
The idea that high-level incompetence is inevitable was formulated in the 1969 best-selling book The Peter Principle: Why things always go wrong. Its authors, psychologist Laurence Peter and playwright Raymond Hull, started from the observation that while jobs generally get more difficult the higher up any ladder you climb, most people only come equipped with a more or less fixed level of talent that corresponds to their intelligence, knowledge and energy. At some point, then, they will be promoted into a job they can't quite handle. They will, as Peter and Hull put it, "reach the level of their own incompetence". And there they will stay, fouling up operations until they either retire or some egregiously inept act gets them fired.
The problem is what they get up to in the meantime. "They end up distracting us from their crummy work with giant desks," says Robert Sutton of the Stanford Graduate School of Business in California. "They replace action with incomprehensible acronyms, blame others for failure, and cheat to create the illusion of progress." Meanwhile, Peter and Hull concluded, the actual work gets done by those who have not yet scaled the summit of their own incompetence. That would be you and me, then.
The problem as I see it is not just that the work gets harder as you rise up the ladder, but that it changes. Being a manager is just different than being a maintenance man, or an archaeologist, engineer, electrician, waiter, or any number of other fields within which managers are generally promoted from within. Motivating people, allocating resources, planning for the long term, and other functions of management are special skills, and there is no reason to assume that they will be distributed in the same way as creative talent or an ability to fix things. Running a whole company in an intelligent way seems to be such a rare skill that almost nobody has it. When a field is dominated by ever-changing fads, nonsensical slogans, and slick consultants, you can tell that we don't really know anything about it, and this is more true of management than anything else.