A Thought About Frequencies and Information

Zipf's Law

Source: "The Mystery of Zipf"

Source: "The Mystery of Zipf"

Zipf, and many others noticed that the more frequent words tended also to be the shortest words - "a", "an", "and", "the". Longer words are found less frequently. I quote from a recent article:

Zipf believed that the relationship between word length and frequency of use stemmed from an impulse to minimize the time and effort needed for speaking and writing, as it means we use more short words than long ones. But Steven Piantadosi and colleagues at the Massachusetts Institute of Technology in Cambridge say that, to convey a given amount of information, it is more efficient to shorten the least informative — and therefore the most predictable — words, rather than the most frequent ones.

 An Extension of This Thinking

I have been thinking about what I read for knowledge. It seems to me that I can classify the things I read by length.

• Very short items such as headlines, titles, Twitter tweets, Facebook posts, etc.
• Paragraph length items such as lead paragraphs (who, what, when and where), abstracts and Zunia item descriptions.
• Longer items, such as Wikipedia articles, journal articles and magazine articles.
• Books.

It happens that I have some problems with my eyes, and I am reading less now. As a result, I tend to watch TV programs and streaming videos more to gain knowledge. Here too there is a similar classification:

• TV news tends to have many items, each treated for a minute or so.
• YouTube and other online sources have many interesting videos (TED comes to mind) that are ten to 15 minutes in length.
• Book TV, CSpan's American History TV, the History channel, and many online college courses tend to have hour long segments.
• In some cases such as PBS series or college courses longer videos are provided (albeit divided into segments).

Clearly there is less information in the shorter things but I tend to see a lot more short things than long ones. This I don't get much information out of a headline or a TV news item, but I read a lot of them. I get a lot of information out of a history book, but I only read one or two a month. I get a lot of information out of a PBS series such as the one on World War II or on the Prohibition era, but I don't watch many of them.

I suggest that we organize our search for knowledge according to a seldom articulated rule -- that we want relatively shallow knowledge covering relatively large areas, and relatively deep knowledge on relatively narrow areas.