When us humans come out of a suboptimal experience, we tend to overcorrect in the opposite direction in our next experience. Repeated overcorrecting results in penduluming (which is often sub-optimal behavior).
For instance, let's say you dated a workaholic investment banker and the relationship goes sour (for reasons not qualifying you to be part of Dating a Banker Anonymous). The next person you may be inclined to date might be a starving artist – someone completely in the opposite direction. In reality, if you dated that investment banker but you just wished they worked a bit less, you may want to date a trader, lawyer, or accountant.
In dating, many people are always looking for the opposite of the person they last had a relationship with.
Rather than completely overcorrecting, a repeated iterative correction, without going to the extremes, may prove more fruitful. In math, this idea is represented as a limit of a function oscillating and eventually approaching an asymptote; in physics this is known as damped oscillations (for all the math and science geeks).
In the physical world for instance, repeated or extreme overcorrecting while driving can cause a car to roll. The idea is to make gradual correction in the opposite direction towards that happy medium.
The past 16 years of picking Presidents, demonstrates that the nation has an inclination towards penduluming. In 1992 we were concerned that President George H.W. Bush was out of touch with the public (e.g. not knowing the price of milk and never seeing a scanner before). And so this country chose a guy named Clinton who was a man of the people and loved McDonald's. The nation was then tired of this guy cheating on his wife and so we yearned for the stability in the prior administration, thus we pendulumed to a George W. Bush who was pious and talked about his love for religion. And this past election we elected someone perceived as very different from Bush.
Even national security policy can vary widely. World War I was a total quagmire because most countries succumbed to a bizarre notion that the first country to strike would have an overwhelming advantage. Of course, this did not happen and the world saw years of trench warfare with little territory gained. Seeking to avoid this horrible kind of war is what led to Chamberlain’s policy of appeasement. He did not want to rush into things which turned out to be the wrong thing to do. Later, World War II turned into the complete mobilization of the nation but a huge cost of 400,000 lives and required the attention of almost the entire country. So when we got into a war in Vietnam we pendulumed to a more limited engagement — the thought was to rather fight them when they were distant and weak then when they were closer and stronger. Clearly that didn't work out, so we followed the Powell Doctrine of overwhelming force in the first Gulf War. But, of course, Vietnam taught us that occupation was a likely failure so we left Saddam Hussein in power. Then in the most recent Iraq campaign we went back to a more limited engagement … but we decided to be an occupying force yet again. It appears our leaders are oscillating back and forth and cannot decide on the best strategy.
And it is not just military leaders, voters, and hopeless romantics that pendulum. We all do it in business. You may join a work environment with no process that is too chaotic, and then move to a new company which might quickly overwhelm you with the initially desired bureaucracy. Or if you tried to outsource your lead gen and it did not work you might decide to never outsource anything ever again.
The human species as a whole is very fickle about what we want. We think we want one thing and when the outcome does not totally work out, we often go for something completely different. We’re constantly taking excessive measures in an attempt to correct or make amends for an error, weakness, or problem, while running off of very little data. And so ideally you would want to collect additional data and make iterative corrections. Just don’t swing too far to the left or right.
Special thanks to Rob Reid for his ideas and Vivek Sodera for his edits.
This is very interesting, the pendulum effect, it reminds me of the ‘cobweb theorum’ in economics, where suipply and demand are out of sync and prices radically see saw up and down, because they are ever seeking to correct for one another.
I have an axe to grind, I’ll be frank. I see the use of military intervention versus humanitarian and diplomatic intervention as a common example of a swinging pendulum, which desperately needs ‘to be in sync’.
So I wrote a book about it, and yes, this is a plug, but the proceeeds go helping me run my 501c3 (charity for those non-Americans). It’s at Amazon.
‘The Competition for Hope’ is book about the race between education and disaster. The education is about equipping young men for sustainable jobs versus the disaster when they lose hope and pick up a gun.
The book is for an international audience, but uses Afghanistan as one of several examples. The British Army (which I admire) has been in Afghanistan for seven expensive years and has lost many brave men. The repeated refrain from soldiers on the ground has been the inability to ‘stabilise poverty stricken captured territory’ in order to keep it. Therefore it has needed to be recaptured again – costing more British lives.
In Afghanistan US $16 billion of humanitarian aid has been wasted or never turned up – leading to needless deaths of our soldiers and others.
The Competition for Hope reminds us that in the American War of Independence many young British soldiers defected to the American side with the lure of sustainable jobs in far away Kentucky. Today in Afghanistan, a valid humanitarian effort to lure young men away from fighting with the reality of sustainable jobs, should significantly help a credible counter-insurgency campaign.
The book points that the solution is to empower existing leaders at grass roots levels with the cash and skills to set up and run employment programs. What is required is the imagination to know the problem intimately, the humility to appreciate new skills and sufficient cash to implement sustainable job programs – for young dangerous men.
As Winston Churchill said: “Action this Day.”
You couldn’t be more on target. San Francisco politics follows your theory to the t. Just look at the way the election of supervisors has changed from district to citywide and back, again and again… The Police Department is no different. We swing back and forth from leader to patsy as the mood suits. The current administration is a pretty good example of this swing. I can only assume that when April comes around, we will see a totally different face at the helm. Thanks for your insight.
Interesting premise about relationships but it just doesn’t hold water. There is no psychological research that shows a pendulum effect in picking potential mates. The preponderance of studies show our choices of mates, friends and lovers is influenced by our family of origin, cultural and socio-economic upbringing.
The workaholic banker and the starving musician may both be cut of the same cloth in that they may be self-referenced, narssistic and absorbed in their own interests and not available for a relationship regardless of how much money either earns. A woman may be continually choosing male behavior patterns she was raised with and vice versa for men.
There were a lot of reasons why we didn’t go into Iraq the first time, not the least being George Bush the first didn’t have the temerity to do it without the backing from European allies. He resorted to his old CIA tactics of inciting the Shites to revolt and then failed to back them up which resulted in the Iraq – Iran war.
As for Vietnam, I wouldn’t call 60,000+ dead American soldiers a small localized war. We were fighting the Chinese indirectly and it spread into Cambodia. There was no viable reason that the drafted Boomers could buy into as to why they were fighting that war. Pretty much like right now in Iraq where Bush the Second tried to gin up WMDs then Liberty and Democracy as the “good” reasons to die for.
That this war took the public longer to wake up to was because of no draft and the covered up the casualties. But both wars are more similar than different in that they were and are a confection of the military industrial complex fought for no apparent reason except to go to war. In fact the last one we started, how incredible is that?
It is just too simplistic to apply one mathematical model whether it be the pendulum effect, Elliot Wave Theory or whatever and then attribute everything that happens to the model. This is when revisionist history occurs as evidenced in your brief but delusional journey through modern American warfare and mine.
I have had clients who couldn’t stand a chaotic work environment move to a bureaucracy and thrive but it had to be the right bureaucracy that fit their style of rigidity. It would be nice if life was so easy to describe in black and white but it isn’t.
However a colleague told me recently that the entire economic crisis was caused by supernatural powers and that I should just get over worrying about it.
thanks Auren. I think you should title this post, “Goldilocks and the three bears for Adults” 🙂
I’ve often noticed this pattern as well. I’ve pondered this much. I usually refer to it as “patterns” or “cycles.” While I understand your warning about overcorrecting too much, I’d also warn against correcting too little (i.e. staying at an extreme for too long) – in fact, I’ll argue that its much worse than over-correcting, because you loose valuable steam. (I assume this is what happened to people that decide to reinvent their lives, and start the pendulum over). Anyways, its impossible to know where the sweet-spot is without overcorrecting. Goldilocks wouldn’t have known “just right” without overcorrecting. Indeed, you don’t know how to over-correct from a fishtail unless you practice it some to get a feel for the friction between your car and the surface, and the momentum characteristics of your car.
I tend to think human learning is an amazingly optimal learning system – being able to find the sweet-spot after only a few swings of the pendulum – maybe even as little as 2. (I’ll strongly note that I mean this is optimal on a personal level – not a macro level involving lots of group-think- like how wars are fought) I’ve certainly found I usually achieve a “good enough” solution after a swing to each extreme – for the easy decisions in life.
I think the suboptimal results you refer to stem from 1) not realizing I’m at an extreme 2) being to stubborn to accept/change my position. That IBM CEO line about “to succeed, double your rate of failure” comes to mind. more on that below.
My conclusion here is that one’s ability to suceed derives from the speed at which you 1) realize you’re at an extreme, and 2) decide to change course. In general, I think my brain decides which is an appropriate course to take once the choice is made to abandon the … max(theta) position of the swing. nerd alert.
ahh, help, tangents are overwhelming me: This also has implications for decisions made too quickly (hastily) – if you’re buying stocks, maybe a quick decision makes you a genius, or maybe you were a fool. Hm, the free market works on a pendulum, huh? If you’re switching relationships quickly, maybe you’re not learning as much as you should be (for the pursuit of optimal-time-to-stability). Some people are probably better at predicting where the pendulum’s equilibrium is given a few extremes. Sometimes you don’t have to live the extreme, you get to observe others. For instance, I’m fairly sure I don’t want to smoke crack, after observing others.
The common human issue of “not letting go of a bad thing” derives from the inability to #1 or #2. Taking this idea a step further:
Note to be argumentative or anything, but … the pendulum analogy is inappropriate, and thus misleading for one reason: we think of pendulums as being acted upon by a fixed gravitational constant. I’ll argue that the gravity pulling on your pendulum is anything but constant. Human’s pendulum’s “gravity force” is made up of many many factors: emotions, experience, ability to project an outcome, how hectic your life is, how hectic your /day/ is, legal or social bounds to your condition, … (e.g “I just want a cigarette damnit, I know it wil kill me” “I’m stuck in this marriage” “I hate my job, but it pays the bills”) … being “stuck” at any of these extremes is just a result of the gravity acting on your pendulum – and it changes. Maybe you stop smoking when you feel more like living, or decide to re-invent a marriage, or finally realize you /can/ change jobs.
For the sake of clarity, some examples I think of when pendulum-ing: These examples are where I struggle between too much and too little swinging – a direct result of my intelligence probably.
1) while debugging code – where I start debugging a problem, how much I poke around in an area until realizing the problem is elseware. (sometimes its hard to look at the big picture again after poking around in an area.) In general, my notion of “its worse to loose pendulum-steam too early, than to swing for a little bit”
2) driving/over correcting
3) golf – a lot of the learning of how to hit the ball deals with tiny little tweaks to hone in on the right way to do it. Golf is hard because its hard to save the sweet-spot in muscle memory well enough. Any golfer will tell you “I think I figured it out!”, and maybe they have for a few shots. But it never stays.
Anyways, I think the overall lesson should be to “pendulum faster”, while working to realize extremes, and acting to make a change when you realize.
Just curious, did the pendulum at the California Academy of Sciences start this discussion? I keep thinking about it. (its powered though, so its kinda cheating – or maybe there is another analogy in “powering your pendulum” 🙂
-Abram
You are right, it happens in the stock market, in relationships, in voting, and in business. The stock market and voting involve millions of people making decisions so you can’t attribute the swings to one irrational decision by one person. Maybe there is some logic to the irrational behavior.
I used to warn my search clients about this. I described it as the “Never make the same mistake twice” rule which always leads one to the corollary “Make a new mistake each time.”
This happens a lot in recruiting! When recruiting the replacement, one is hyper-vigilant about avoiding the issues presented by the previous employee, yet blind to the issues which the new candidates might present.
Interesting. I hadn’t heard the term
“penduluming” before but the concept has been around for a long time in terms of biological phenomena such as insect outbreaks, diseases, etc. When there are new introductions of insects or diseases and few, if any, parasites or predators to bring the newly introduced organism under “control” we usually see huge increases in population desity and concurrent swings in population density much as your diagram shows. Eventually, the population(s) come into equilibrium with the environment and the problem disappears until some perturbation of the system occurs and the organism once again starts oscillating wildly.
Nice post – I agree completely! One of the most important behaviors for entrepreneurs is to learn to manage their emotions and try to keep even-keel in the highs and the lows. You make the worst decisions when you are in an emotional state. And so many CEOs are in that emotional state during these tougher times.
In developing a new behavior, sometimes it’s useful to use the pendulum approach (behaviorally practice the opposite extreme) in order to establish a golden mean.
For example, people with drinking problems can find it useful to abstain entirely for a substantial period of time before they succeed with moderate drinking. Similarly, shy individuals who practice therapeutic shame-attacking exercises (purposely behaving outlandishly or shamefully in public), then more easily learn to act appropriately assertive in social situations.
I don’t know. Maybe — I don’t know too many people who’ve dated i-bankers then exchanged them for starving artists. I know a lot more girls who complain about certain characteristics of their i-banker boyfriends, then wind up choosing to date another i-banker who ultimately has very similar characteristics. I’m not sure what you call that. How many DABA girls have hung up their designer dresses to elope with struggling musicians? If any, its the exception not the rule.
But if I take a weak opposition to pendulum theory in relationships, I tend to reject the theory almost completely in politics:
In the 2000 election, for example, 50% of the nation voted to elect Gore — a politician with nearly mirror policies to that of Clinton. And if there were any year in which pendulum theory should have applied, it should have been 2004, but it did not. Even in 2008, nearly half the elctorate voted for the same sort of leader we’ve had for the past 8 years. The type of president we choose tends to be more a function of the primary process that selects extremes of party representatives and pits them against one another in the general election. The effect may masquerade as penduluming, but there is more at work that produces these results.
Even if we call the effect a pendulum, its certainly a much more conservative adjustment we make in politics than dumping an i-banker for an starving artist — to make that analogy true, we’d have to elect a long-haired communist peacenik after a flag-waving conservative hawk. In the context of the political spectrum, the choices Americans make seem closer to the more reasonable, incremental shifts you describe — from i-banker to accountant.
Those are some interesting observations Auren, although I’m not sure the psychological research would support them 🙂
Many theories of learning argue that humans and animals do tend to behave like a damped oscillator. We make a very big adjustment the first time we correct a behavior, with progressively smaller adjustments on each successive correction. Unfortunately, that kind of behavior might not scale up to political decisions, where we get to make a choice so infrequently, and have less opportunity to learn from our mistakes.
Note that the damped oscillator also “overcorrects” in the sense that it corrects to the mean and then beyond. So what we are trying to determine is whether the correction is further from the mean than the original starting point. In other words (to push your analogy beyond it’s breaking point) the question is not whether Clinton was to the left of the mean compared to George H.W. Bush, since that is also true of our damped oscillation. Instead, the question is whether Clinton was further left of the mean than George H.W. Bush was right of the mean. I think that’s a tougher argument to make.
Don’t have a ton of time right now to elaborate, but in fact it was the Korean War (the war we always forget) rather than WWII that swung the US policy pendulum towards limited engagement. Korea was fought as a UN coalition in theory (in fact, it was more like the “coalition of the willing” in Iraq Part 2 with the US doing most of the fighting) and it was our general failure there (given that the war technically still hasn’t ended 50 years later) that convinced the American public and subsequent administrations that – to paraphrase The Princess Bride – “you never fight a land war in Asia.” The American public was tired of war in general and war against communism in Asia in particular. But even the limited effort in Vietnam by JFK led to full scale failure by Johnson in the end. Strikingly, no matter how you feel about war in general, taken in context our “huge” losses in Iraq are actually remarkablely small. Something like three months in Okinawa cost the United States on the order of 12K dead and 60K wounded. For one tiny island. That bloodbath convinced Truman to drop the bomb. Put in perspective, the seven-odd years of occupying the entire country of Iraq has had “few” casualties. Our armed forces have been considerably more successful – at least at saving American lives. Then again, it’s hard to compare the human suffering of war quite so objectively. As Stalin said, “one death is a tragedy; one million a statistic.” But I digress. Given the state of Iraq as a military success and a political failure, one wonders where next the pendulum will swing. Total war?
Not sure I agree. Even accepting your examples and the premise that this is an accurate mathematical model, small iterative changes isn’t always the best strategy to find an optimal solution. In fact, your recommendation is exactly the wrong strategy when trying to find an globally optimal solution in a neighborhood of locally good solutions.
In Evolutionary Computation we call this exploration vs. exploitation. Exploitation (which you advocate) is good at refining a solution, but will get stuck in local optima, missing better solutions that lie elsewhere. Exploration is good at finding a new, previously undiscovered solution somewhere else in the domain, but struggles at refining that solution. For most complex problems, a mix of reproductive strategies that employ some exploration and some exploitation often yield the best solution-finding performance.