A Thought About Technology Innovation

Frequency of English Words in Wikipedia Entries (in 2006)

"Wikipedia-n-zipf" by Victor Grishchenko - Victor Grishchenko [1]. Licensed under GNU Lesser General Public License via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:Wikipedia-n-zipf.png#mediaviewer/File:Wikipedia-n-zipf.png

A plot of word frequency in Wikipedia (November 27, 2006). The plot is in log-log coordinates. x  is rank of a word in the frequency table; y  is the total number of the word’s occurrences. Most popular words are "the", "of" and "and", as expected. Zipf's law corresponds to the upper linear portion of the curve, roughly following the green (1/x)  line.

When you are writing, most thoughts will be expressed with common words. Sometimes you will use less common words, but words that are in your working vocabulary. Sometimes you will use words you have read and understood but not used before -- words you then need to express a thought precisely. Sometimes you will go to a thesaurus or ask a friend for a word that you need and can not find without help.

People have analysed large bodies of text, as was done in the graph above. If words are ranked according to the frequency that they are used and ordered by that rank, then the graph of frequency shows the expected pattern. More than that, there is a highly regular graph and the distribution follows a power law, called Zipf's law.

A conceptual model for the process is that at each point you draw a word from your vocabulary; you select the word that expresses the concept needed. Some concepts occur frequently in your writing, some occasionally and some rarely. The probability that you will draw a specific concept and word is proportional to how often you use that concept and word. Sometimes you will reach out to add a word to your written vocabulary to express a concept new to your writing or to better express a concept you have tried to express before. You do so a set of possible words adjacent to your working vocabulary. (I am here drawing on an idea from the evolutionary biologist, Stuart Kauffman, who has discussed the concept of "the adjacent possible") Having used a new word, you are more likely to use it in the future, and the more frequently you use it, the more frequently you will use it afterwards.

Technological Innovation

It has been suggested that something similar happens in technology choice. There are some tools and techniques that are used widely in a society, in part because they work to achieve widely shared objectives or solve widely shared problems. Sometimes someone will reach out to use an unusual tool or technique. More occasionally someone will innovate, transferring a kind of tool or technique used elsewhere -- from one class of the adjacent possible technologies. On other occasions someone will combine ideas that exist in the environment to create a new tool or technique. Such invention can also be regarded as derived from a class of adjacent possible technologies.

I had the distinct privilege of working one summer (when I was right out of school) with Mark Kac, a very distinguished mathematician. He is famous for the following quote:

There are two kinds of geniuses: the ‘ordinary’ and the ‘magicians.’ an ordinary genius is a fellow whom you and I would be just as good as, if we were only many times better. there is no mystery as to how his mind works. Once we understand what they ’ve done, we feel certain that we, too, could have done it. it is different with the magicians... Feynman is a magician of the highest caliber.

Perhaps we can think of innovations as characterized by the degree of originality needed by a person to make them. Many innovations would be available to most of us. Some would be available to many of us.

I think of Thomas Edison inventing the light bulb. It was not a new thought that running enough current through a filament would generate light. The problem was to find a way to create that light without destroying the filament in the process. In retrospect it seems fairly obvious that the filament would not burn up if there was no oxygen to support the combustion, so putting the filament in a bulb with no oxygen seems an "adjacent possibility". Edison then had his assistants try many possible filaments. (Edison allegedly said, "I have not failed 1,000 times. I have successfully discovered 1,000 ways to NOT make a light bulb.") That suggests that, at least in the invention of the light bulb, Edison was a genius of the first Kac kind. His innovation in that case was in the selection of a possibility that would be sufficiently close to be available to many, once we had seen what Edison had done.

Then there are the magicians of technological innovation. Nicola Tesla might have been one.

Tesla's invention of alternating current motors and the alternating current electrical system (which beat out Edison's direct current system) seem to me inexplicable. Even having taught electrical machinery at Berkeley I found it hard to visualize the fields involved -- and have no idea how he invented a technology for their use.

I like Brian Arthur's book on the nature of technology. I think it too suggests that technology is a growing body of knowledge, and that technological innovation is usefully regarded as drawing from fields of adjacent possibilities.

A Mathematical Model

This post was based on a recent paper in Nature which provides a model for novelties and innovations. While it does not address technological innovation specifically, its use of Zipf's Law, Heap's Law, and the Polya urn model seem potentially useful if applied to quantitative understanding of the introduction and spread of new technologies. (One thing that the model does not seem to deal with is the replacement of a technology by a newer and better one.)