Tag Archives: information theory

Isaac Asimov and Information Theory

I have been reading a lot about information theory these last two months. In the course of this reading, the same people keep showing up again and again. Vannevar Bush, Alan Turing, Claude Shannon, J. C. R. Lichlider, Marvin Minsky, Norbert Wiener, and John McCarthy to name just a few.

It is the last few that caught my attention. Having read much of what Isaac Asimov wrote over the course of his life–including his 3-volume autobiography, which I’ve read at least 14 times–the last three names were already familiar. Marvin Minsky, Norbert Wiener, and John McCarthy were mentioned a number of times in the second volume of Asimov’s autobiography, In Joy Still Felt.

Asimov knew them in his years living in the Boston area. All three worked in information theory at M.I.T. Asimov, who had quite the ego, said of Minsky that he was one of two people that was smarter than Asimov himself. The other was Carl Sagan. At a party for Asimov’s 50th birthday which both Sagan and Minsky attended, Asimov wrote that “Carl did not fail to point out that I had in the same room with me the two men I conceded were more intelligent than I was.”

Minksy was involved with robotics at the time. Norbert Wiener coined the term “cybernetics.” He also tried to get Asimov to collaborate on a mystery with him. McCarthy worked with Minsky on artificial intelligence.

Thinking back on this, it seemed that Asimov’s interaction with these men was purely social, and a matter of proximity, and knowing the same people. What is remarkable to me is that, knowing these people at the forefront of information theory, I can’t think of a single instance where Asimov wrote about information theory in the way he wrote about other sciences. He had the best and the brightest in the field over to his house, but as far as I can tell, he never showed any intellectual interest in the theory.

Sure, Asimov wrote about robots and the Three Laws, but that is not information theory. Asimov wrote about entropy in physics and chaos theory, but not about the parallels between entropy and information. He wrote popular pieces about using computers, but I could find a single essay in the 399 monthly science columns he wrote for the Magazine of Fantasy & Science Fiction from the late 50s until his death in 19922 that went into any detail on information theory. The closet I can come are 2 essays in the 1970s.

The first, “The Age of the Computer” is really more about the impact of computers on society, not information theory. The other, perhaps a little closer, is “The Ancient and the Ultimate” is about how information is contained (book form or digital).

I can’t explain this lack, especially given his camaraderie with Minsky, McCarty and Wiener. Asimov admitted that there were certain fields he simply didn’t understand. Economics was one such example that he gave. Could information theory have been another? After all, he did admit that Minsky was more intelligent than he was. By implication, could that mean he just didn’t get information theory?

It’s too bad, really. I would love to read an F&SF-style essay on information theory written by Asimov.

Hard Books to Understand

I have just finished James Gleick’s fascinating book The Information on the history of information theory. It is a rare milestone book for me in that it is one of three books that have really pushed my ability to comprehend complex subjects to the limits.

I say one of three books. I went through through the list of books I’ve read since 1996 to be sure. There are only three books (including Gleick’s) out of about 1,070 so far that I immediately recognize to be in this category.

The first of these was Consilience by Edward O. Wilson–a book that was recommended to me over 21 years before I finally got to it. The book deals with the theory of how all subjects are interrelated. At least, I think that it what it was about. Maybe it was more about taxonomy, or meta-taxonomy. It was a tough read.

Not long after that, I am upon an even more difficult read, indeed, one that I consider the most difficult book to comprehend that I have read. This was The Origin of Consciousness in the Breakdown of the Bicameral Mind by Julian Jaynes. I was turned onto this book by Robert J. Sawyer, who has mentioned it frequently as an influence upon his own writing. Indeed, a few months after I finished it, I saw Rob at a convention and told him how difficult I thought that book was.

Jaynes’ book is a theory that consciousness as we think of it arose much later than people thought–indeed, he argues that it is relatively recent construct, going back to ancient times, but not before. This was difficult to comprehend (my own consciousness was not really up to the task, I guess) but it was a fascinating argument (if I understood it correctly).

Gleick’s The Information has now made this elite list. Of the three, it was the most comprehensible, but I had to strain to understand it. I had to pause, and re-read passages, and visualize the concepts, and really think about them before I felt like I had a grasp on them.

Do you see the pattern? All three books are about theories of information. I find this fascinating, since, as a developer, I work with practical information theory every day. I find reading about it endlessly interesting, and yet, it is an incredibly difficult subject for me to understand. My theory is that the more abstract, the more difficult a subject is to comprehend. I’ve read math books (A Tour of the Calculus by David Berlinski and essays on mathematical subjects like When Einstein Walked with Gödel by Jim Holt) that have been fairly abstract and yet comprehensible. But information theory is so abstract that I find it exceedingly difficult to understand. Of course, the delight in reading about this stuff is in large part coming away with a better understand of it.

I’ll add one honorable mention to this short list: How the Mind Works by Steven Pinker. The book was recommended to me a few decades ago by one of the smartest people I’ve ever met. When I finally got around to reading it, I found it to be a challenge. But not quite as much as these other three. When I scanned my list, I hesitated on this one, but finally decided that it didn’t quite make the cut.

I’m kind of fascinated by the concept of books that are challenging to read–from a comprehension standpoint. Are there books that you’ve found to be challenging? I’d be interested to know what they are. Drop you suggestions in the comments.