Complex And Inefficient Systems

Simplicity by *chuscli
Simplicity by *chuscli

“Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius – and a lot of courage – to move in the opposite direction.” ~ E.F. Schumacher(?)

In discussion with a colleague, I mentioned that WordPress.org was able to provide very nice functionality with excellent usability through php, mySQL, and Apache. She said these were simpler technologies not particularly suitable for today’s complex corporate environment where more complete frameworks are required to satisfy all of the business requirements. I challenged her by saying that some of the blogs get millions of daily hits with this simple architecture while some corporate systems stall after a few hundred users. She told me that the data model in a blog is much simpler than in any corporate system. I know. I am just not willing to take this man-made complexity for granted. Call me obsessive compulsive when it comes to keeping the world simple but what I have seen and continue to see is simply excessive complexity.

In addition to the computer on which I’m typing this text, many structures around us are systems of varying complexities. Some of the complex systems make the computer seem like a basic child’s toy. Everything can be viewed as a system (often with many nested sub-systems) but here are some examples.

  • The human body is perhaps one of the most complex systems with interlinked sub-systems made up of cell-structures, interfaces, etc.
  • A corporation is a human system that can be as simple as a piece of paper and as complex as today’s multinationals are.
  • Software, the Internet and all other computer systems are built up from many sub-components which further rely on sub-systems to carry out their tasks.
  • Banks and markets are also systems playing their part in a larger global exchange, transfer, and conversion system.
  • Houses with their electrical, plumbing and other sub-systems can be user-friendly, complex, with interfaces, inputs and outputs.
  • Government is a complex multipurpose human system with impacts on all other systems.
  • Universal health care initiatives are also systems with many participants.

I could list endless examples but just like an average gasoline engine, I hypothesize that systems built, managed, operated, maintained by humans operate at some very low level of efficiency. I am not just talking about productivity and other similar measures. I am suggesting that in aggregate we build inefficient systems and structures. I see this in computer systems all the time. I would further suggest the larger the system, the higher its complexity, and the lower its efficiency. If I had multiple lives, I would devote one to the study of this subject with hopes of finding a set of principles to guide new efficient system architecture and construction. Why do we need to do this? With the amount of inefficiency growing with complexity, we may find ourselves at a point where the maintenance of the system takes more effort (human lives) than the value of the system warrants (Soviet government). We may find ourselves unable to properly regulate (think AIG) and maintain the system. We may find ourselves challenged with enforcing ethical principles (Madoff) in the system. We may find ourselves slaves to these suboptimal systems that we have imposed on ourselves. When the systems choke us, we end up with a revolution. When the systems clash, we end up with wars.

Could the fundamental source be our insecurities and yearning to appear more intelligent, more capable, stronger… strong enough to handle complexity? Or is it that we are the inefficient, ineffective ones who are capable of creating nothing more than a highly ineffecient system? Is it our methods of education? Or perhaps for our systems to be efficient, we would need to become machines, losing all creativity. Could it be that each time we step into a system, we feel the need to make changes thus destabilizing the entire structure?

10 Comments

  1. Jeff – Natural systems have many advantages over systems engineered by humans. Thanks for the visit.

    Don – Funny but they certainly speak volumes! Thanks again!

  2. I am fond of the saying, “when all else fails, read the directions.”

    Here are some other possible explanations — for your excellent post:

    1) Behind every new complex, man-made system (from Ipods to CDO’s) is a salesman who gets paid based on his ability to sell it. And a team of professionals whose existence is based on supporting the complex system.
    2)On the buy side, how many purchasers are motivated by “keeping up with the Jones'” (i.e. status) versus utility?
    3)Reading and writing is a complex and difficult activity, but because we learned how to do it as children, we take it for granted. The knowledge, effort and time required to run and maintain a typical PC is substantial, but more burdensome because we’ve learned to do it as adults AND it’s constantly changing.
    4) As you correctly observe, from my systems engineer point of view, EVERYTHING is a complex system. However, one does not need to understand molecular biology to enjoy (or despise) an Egg McMuffin; anymore than one needs to understand particle physics to use a flash drive; or to understand macroeconomics and payment systems to cash a social security check.
    5) Complexity is one byproduct of the specialization of labor that our civilization’s developed. Yet all great systems share the traits of simplicity for end users, idiot-proofness, robustness and redundancy.
    Back at NASA, we called that “Fault Tolerant Systems.”

  3. Rocky – thank you for your excellent points. Regarding 2), “keeping up with the Jones” is highly dependent on our perception of them. During the Russian revolution, the proletariat was sold sacrificies with the promise of utopia. Regarding 3), all systems always change but some change faster than others. Some systems change by design (elections). Language, as a system (or interface), changes quite a bit. Regarding 5), have you ever come across any estimate of the cost of building “fault tolerance” into a system?

    Also, the cost to replace or modify an existing system is often an order of magnitude higher than the cost of building a new system (Big Dig in Boston for example). Thanks again for the visit.

  4. LD:

    To answer your question on 5, there is a concept called “Six Sigma Quality”… which was (ironically) devised by Motorola. Basically it means that the item will only fail after six standard deviations (> 99.997% reliablity). Depending on the system, that may be 6 sigma hours, 6 sigma iterations, 6 sigma stress. This is a common approach to designing disk drives and other mission critical stuff.

    An example: if I drop my laptop computer from my desk, it will probably break. But if I bought a “ToughBook Computer,” it will survive that stress. For this “quality,” the Toughbook costs roughly 4x the price of the laptop. That is an anecdote which answers your question. This is entirely a function of design/cost/reliability.

    An engineer can build a system which can withstand 7 sigma, 8 sigma, 9 sigma events. At some point, however, there is an exponentially- marginally diminishing return on the improvement. (Although plaintiffs’ lawyers will argue otherwise 😉 Additionally, there will ALWAYS be some unanticipated stress that will get ya. (For example, the computer may be able to withstand a rooftop drop, but the software virus that you caught from Facebook will erase your hard drive.) 😉

  5. Rocky – thank you for this great explanation of six sigma. I have heard that Dabbawalas are a good example of it in a human system. Do you think our current financial troubles are the once-in-a-lifetime that has pushed the system beyond the 99% reliability? Also, what is the probability that the trillions being spent will result in a six sigma system of never having a similar recession again?

  6. LD:
    At the risk of potentially alienating you, and thousands of other people, I will bluntly state that the Social Sciences are not a science in the same sense of the Physical Sciences. Economics and psychology use certain scientific methodologies for proving/disproving hypotheses, but (at the current time), they lack the underpinnings in atom physics, chemistry and biology to establish 6 sigma confidence levels. Perhaps at the 2-3 sigma level. But not at the 6 sigma confidence level.

    While odd and improbable things do happen in the natural world (meteors, volcanoes, earthquakes), they happen without human participation. When living organisms that have free will interact with natural systems (whether they be amoebas or humans), the probabilities become much less useful.

    Without weighing in on the pros and cons of trillions being spent, I am highly confident that both unexpectedly great and unexpectedly horrible things will occur again in the next 50 years.

  7. Rocky – I can’t disagree with anything you say. Statistics is very useful when used properly and very dangerous in ignorant hands (minds). My mind (sometimes dangerously) applies concepts across multiple disciplines but I enjoy that as a way of learning. 🙂

Leave a Reply

%d bloggers like this: