Tuesday, July 25, 2006

The Nature of Knowledge

Following my previous article on Life as a Complex Adaptive System [CAP], it is only natural that I should explore the nature of knowledge in a data-driven context

When a CAP acquires data through its sensors [in human beings: our ears, eyes, nose, tongue etc]. it needs to process this mass of data to turn it into information. Information differs from pure data. By being organized in a contextual format, information yields insights. Some of the things the human brain tends to do to data are:

Categorization. Human beings are fond of sorting things into pigeon holes. Thus, this is a Monarch butterfly and that is a Swallowtail butterfly. This is an insect and not a reptile because etc etc. Categorization implies groups of objects with common properties. When we categorize, we have fewer objects to deal with and Life becomes more simple. Sometimes categorization can be quite arbitrary. When is a human considered tall or fat, ugly or beautiful? The definitions differ from person to person, from place to place and from time to time. But nevertheless, categorization is an important data management tool for a CAP. Categorization involves the creation of classes or clusters with strict boundaries. These boundaries are sometimes too sudden when a gradual approach would have been more befitting. Most things in this world are not black and white, but shades of gray. The creation of more categories may help to mitigate this problem. For example if something is classified not merely as Positive or Negative, but with 5 categories: Very Negative, Negative, Neutral, Positive and Very Positive, it helps a bit. In recent years the use of Fuzzy categories based on Fuzzy Logic, has helped to smoothen the transition from one category to another. In the real world, such technologies encrypted in semi-conductor chips and sensors has also resulted in elevators that are less jerky, car engines that are smoother, and washing machines and rice cookers which wash cleaner and cook nicer rice..

Relationship analysis. The end result of all analytical activities is to help a human being describe the relationship between objects. An object has no meaning by itself. It must be viewed in the light of its place among other objects. Here we use object as an abstract noun, in the widest sense of the word. Humans will try to express and measure the relationship and changes in relationship between an object and other objects. Changes in relationship between objects may be correlated. We make decisions regarding an object based on our analysis of the relationship between it and relevant objects. In cases where the relationship cannot be measured or expressed in a quantitative way, we make human judgment. Even though our technologies for measurement of the degree, distance and speed of relationships between objects has improved a lot, there is a huge area where we have to make decisions based on non-quantitative human judgments.

Pattern recognition and Prediction. Many decision making activities are based on of pattern recognition. By looking for patterns in data, we hope that these patterns will be the same for the future, or a discernible pattern for the future can be determined? A pattern is a form or template that repeats itself, although in the real world, most patterns cannot be so strictly defined as they possess a lot of noise that tends to cloud the main pattern. Why do we need to predict? We need to make a prediction so that we can plan and anticipate. It is the human way of dealing with the uncertain future, and probably the most important activity that differentiates us from the other animals. Our predictions will in many cases not be 100% accurate. But we can plot several possible outcomes of a scenario, and decide what to do when each of these scenarios take place. We can also assign a probability of our prediction coming true or estimate the accuracy of our predictions. We have been doing this intuitively with our marvelous brain for thousands for years although we may not realize it. When a Masai warrior faces an attacking lion his brain automatically analyzes the probability of various scenarios. Will the lion keep on charging, it is just feigning, what would be the degree of injury, can the Masai scare away the lion with a shout, will a spear stop the lion? Drawing upon his knowledge base and accumulated experience of his community, the Masai warrior decides what to do. Today it is often possible to build mathematical models that represent the relationship between variables and do a prediction based on that. Many activities of modern man involve planning and scheduling ,which requires prediction at least on probabilistic terms. Without prediction Companies cannot forecast their sales and produce the right amount meet demand. Without prediction, projects of all nature could not be planned. Information technology has greatly enhanced our task for collecting data, analyzing it and predicting with it. RFID sensors embedded in everything from clothing to baked beans enable the monitoring of store sales and makes re-supply a more efficient task.

Having processed the data into information, the next step is to transform the information into knowledge. Turning information into knowledge adds even more value to it. The are many definitions of knowledge. Wikipedia defines knowledge thus: Knowledge is the awareness and understanding of facts, truths or information gained in the form of experience or learning (a posteriori), or through introspection (a priori). Knowledge is an appreciation of the possession of interconnected details which, in isolation, are of lesser value.

But I prefer a more data-driven definition like: Organized or contextualised information which can be used to produce new meanings and generate new data.. The essence of it is that knowledge is obtained by synthesizing information. Knowledge builds upon a knowledge base. At its most generalized, it means the store of all human experience, from which we learn and from which we build upon to create new knowledge. A good example of the difference between data , information and knowledge and wisdom can be seen with this illustration from the Online Dictionary of Computing: 1234567.89 is data; "Your bank balance has jumped 8087% to $1234567.89" is information; "Nobody owes me that much money" is knowledge; and "I'd better talk to the bank before I spend it because of what has happened to other people" is wisdom.

99% of all new knowledge is built on synthesis of past knowledge , leading to new insights that creates the new knowledge. Nobody would intentionally reinvent the wheel. Ph. D students write their thesis based on an study of the existing literature of the subject. Invention of new technologies is based on the study of existing technologies. As an abstraction, Knowledge cannot be destroyed. It can be lost but it cannot be destroyed. Thus the sum of all human knowledge keeps on growing. With the creation of the Internet and Search Engines like Google, the distribution of Knowledge has increased by thousands of times. And a world where everyone has more Knowledge can only be better.

No comments:

Post a Comment