Images (top) self-explanatory from Charles Crowley at http://www.cs.unm.edu/~crowley/Complexity/index.html
(bottom) Stephen Wolfram's four classes of cellular automata illustrating diversity with small initial differences or perturbations.
Studies in Chaotic Systems, which are sub-cases of Complex Systems, show four types of movements: (1) movements that eventually become extinct of (2) movements which produce regular cycles (3) movements which grow structurally but the growth is not totally predictable (4) movements which dissolve into utter chaos. It is category 3 that is most interesting as it seems to represent the state which Life in this Universe follows and may explain how interesting, complex systems, such as living things, arise in a world where one of the most fundamental principles is that entropy, that is disorder, always increases. [Second Law of Thermodynamics].
Experiments carried out with Cellular Automata [which are grids created on a computer and populated and allowed to grow through time according to certain rules] confirm that there is a region between Order and Chaos that seems to be the sweet spot for evolution, diversity and progress. Scientists from Christopher Langton, the pioneer in Artificial Life to Stephen Wolfram, inventor of Mathematica the software and author of A New Kind of Science, have explored this region intensively and tried to model its mathematical properties. Langton developed a value called Lambda to try and locate this region, but found that it is not a Universal value that can be generally applied. It all depends on the path taken towards the chaotic stage.
For our purpose though, shorn of all the mathematics, we can safely say through experience that Life is best on the edge of Chaos. Take Jazz music and its improvisation as an example. Jazz musicians improvise within a structure-the chord progression of the song. They are free to go anywhere they like, as long as they stay within the underlying structure. The best and most interesting improvisational ideas are those that walk on the edge of Chaos between total Order and predictability [such as playing root notes or arpeggios of the underlying chords] and utter chaos such as playing in a totally random way without taking into consideration the underlying structure. And like the Category 3 complex cellular automata that evolve into so many varieties even from the same basic rules but with infinitisimal initial differences, jazz improvisation takes you on an infinite variety of paths to achieve the same objective. I was suprised to find that I was not alone in having this view of Life and Jazz as best when balanced on tyhe edge of Chaos. I found a 183 -page .pdf on exactly this subject by Professor Morris Holbrook of New York University, who wrote this article for the American Markering Review in 2003: http://www.amsreview.org/articles/holbrook06-2003.pdf
Its the same with anything in life. The best jobs are those that are challenging but not too challenging as to be stressful. The best novels are those that are not predictable, and yet have some order. The best gym workouts are those that leave you sufficiently exhilarated but not have you gasping for air or leave you with sore muscles. Even in cuisine, the best recipes are those that have an underlying theme, but within that theme display the creativity of the chef. And like the cellular automata we can achieve the same objective in so many different ways. So the question is : how do we find for each of us, our sweet spot in life ? The adage that everything is good if done in moderation will help us find the sweet spots in each facet of our life.