Collective Intelligence in Action, by Satnam Alag

After getting a preview of Gnuplot In Action by Philipp K. Janert, I wandered around Manning Publications Co. to see what other books they had available. Collective Intelligence in Action was coming out in August, 2008 and was available through the Manning Early Access Program. That sort of book always gets my attention; people who don't know about collective intelligence talk about smart mobs all the time. When you see a book with Collective Intelligence in a title, it means that someone isn't just parroting a diluted meme. Collective Intelligence, however, has deep roots and is a playground of ideas that are often ignored in a period of buzz about what recent people have written only because they did so after the Internet and Web 2.0 came along.

And so, I got a copy of Satnam Alag's unedited draft of Collective Intelligence in Action - and I was not disappointed. In fact, I was pleasantly surprised.

At the very start of the book (Chapter 1, 'Understanding Collective Intelligence', the author wrote something that few people seem to recognize:

...These four basic conditions are: “Wise crowds” are effective when they are composed of individuals who have diverse opinions; when the individuals are not afraid to express them; there is diversity in the crowd and there is a way to aggregate all the information and use it in the decision making process...

Implicit in this is a phrase often lost among the meme-trodden: wisdom. Wisdom of crowds, as described in that short paragraph, is a drum that I beat recently, before I even gained access to the book. It implies that there is a direction of the collective intelligence; where intelligence itself may be a scalar quantity, wisdom tends to be more of a vector quantity. Without some sort of direction, we would all probably be primordial stew. As simple as the use of the phrase 'Wise crowds' was, the subtle nudge of direction implies a wisdom on the topic. Recursion is a lovely thing.

As the table of contents (below) indicates, the book starts with an introduction explaining Collective Intelligence and is followed by sections on gathering intelligence, discovering intelligence lurking in what is gathered, and applying the intelligence. There are abundant Java code injections where applicable, but the code flows with the writing of the author. Some familiarity with Java, beyond being able to spell it, is suggested but not necessary; a programmer of any language should be able to parse out the flow from the diagrams and code.

Part one might be easy for people to skim and avoid getting too deep into - but there are some really useful nuggets in those first 6 chapters that are worth a little extra time.

Part two steps up the pace. Discovering intelligence from the data - something that is overlooked in much of the talk of of Collective Intelligence - is a true treasure for those unfamiliar with data mining and text analysis. Some of the magic done by wizards behind the curtains of Web 2.0 are easily explained. The diagrams are quite helpful, but from Text Analysis onward will require a bit of a mathematical background to fully come to terms with.

Part three brings it all together in a way such that it is greater than the sum of the two other parts. Code, diagrams and the author's voice are clear and left me wondering what else is going to go into this book this August.

Overall, I am very impressed with this work and even excited about the book - something which may something about myself as much as the book. It is technical, it is theoretical - but most importantly, it is practical and focused throughout a diverse group of disciplines that, through the author, are integrated very well.

Since it is an unedited draft, I will not give the book a formal review at this point. But I will say that it is what many people have been waiting for - an explanation for how these things work. Great stuff. Collective Intelligence in Action is worth an early look - or waiting for.

Table of Contents

1. Understanding Collective Intelligence

Part 1: Gathering Intelligence

2. Learning from User Interaction
3. Extracting Intelligence from Tags
4. Extracting Intelligence from Content
5. Searching the Blogosphere
6. Web Crawling

Part 2: Discovering Intelligence from Data

7. Data Mining: Process, Toolkits, and Standards
8. Building a Text Analysis Toolkit
9. Clustering
10. Making Predictions

Part 3: Applying Intelligence

11. Intelligent Search
12. Building a Recommendation Engine

Appendix A: A Detailed Look at Web 2.0