Review: The Master Algorithm (P. Domingos)

I recently finished reading  The Master Algorithm (Amazon) by professor Pedro Domingos, which is an overview of the current state and near future of machine learning research, targeted for general audience. It can be warmly recommended for everyone interested in machine learning and artificial intelligence. It requires no special background knowledge, although a basic understanding of programming, optimization and principles of artificial intelligence is helpful.

The author Domingos works as a professor in the University of Washingon in Seattle, and is a very respected authority in the field of machine learning. In my own upcoming doctoral thesis I cite many of his scientific articles. Comparing to many other experts on this field, his writing is very clear and easy to understand. Also The Master Algorithm, published a bit over a year ago, is very clearly written, and it is targeted to those not familiar with the field and its jargon. A lot of insight can be gained from the book, provided that the reader has the motivation for thinking through the ideas and thought experiments. The book has received some well-earned praise from such important experts as Judea Pearl and Sebastian Thrun. The focus on general audience means, that many complicated ideas on a difficult field have been simplified greatly, but there is an excellent collection of further reading for those who wish to study the details.

Domingos organizes the scholars of machine learning into five ‘tribes’ or schools of thought, and they are researchers and developers relying on either logical deduction, neural networks, Bayesian statistics, genetic algorithms or analogies for creating learning systems. The advantages and disadvantages of the different tribes and their algorithms are described intuitively, and finally the idea of developing a ‘master algorithm’, which combines the best ideas of the five tribes, is proposed and developed further. This kind of ‘general learner’ could accept any kind of data, learn the learnable from it, and apply the learning to anticipate similar situations in future. The proposed ideas and goals are somewhat realistic, although a bit simplified and overly optimistic.

Today, this topic is naturally very timely and important, and it’s difficult to avoid exposure to it, since the first truly intelligent systems are entering the public stage. Some examples include IBM’s Watson, Google’s AlphaGo, self-driving cars such as those sold by Tesla, and digital assistants such as Apple’s Siri, Google Now, Amazon’s Alexa and Microsoft’s Cortana. Some people probably wonder, what kinds of benefits and business opportunities could be enabled by these technologies, while others observe the progress more cautiously and worry about the possible threats and harmful effects.

In the conclusion of his book, also Domingos discusses in my opinion very thoroughly the social aspects of artificial intelligence. He arrives at very similar optimistic conclusions as myself: artificial intelligence will become an extension of us humans just like all other technologies, such as smartphones and search engines, have already become. Humans will work in cooperation with intelligent systems and devices. These will make our lives easier in many ways. Of course it is inevitable, that there will be less and less economically productive work left for humans to do. This is why we as a society must move towards softer values and be prepared to share more and pay extra for services provided by humans. Both Domingos and I support a basic income that guarantees a certain level of sustenance for everyone.

In my personal opinion, technology should be seen as an achievement of all of humanity. Even though some people may become rich as a result of their own work and entrepreneurship, no one can anymore claim to have achieved everything alone. Behind all success there is the work of generations of humans and the fruits of the global economy. Hard work and risk taking should be rewarded, of course, but wealthy people must be prepared to share more to others as well. The value of a human being cannot be measured by economical measures alone.

With his students, Domingos has developed a relational learner called Alchemy. My own dissertation research is close to this, and I hope to introduce this method in this blog later along with some kind of practical demonstration. The same I wish to do naturally with my own methods as well.

This is the first review article I have written, and it starts a new series of blog articles. I hope to keep publishing it monthly.

(Updates 18 Oct 2016 6:24 pm: clarification to sentence structure, some minor corrections)

next steps

During the last week, I have been mainly taking care of various practical things necessary when starting a business, and sketching in my mind a picture of my business practices. Now I thought about sitting down for a while to describe, what kinds of things I will discuss in this blog in the near future. My intention is, besides commenting on various current or otherwise important and interesting topics, also to showcase my own activities and skills in the form of various projects. In this article I will sketch out some kind of a map of the terrain I am planning to explore in the near future, and hopefully succeed in sparking the interest of my potential supporters and customers.

books and writings

Because the long term success of my business will be founded on special expertise, I have set aside some time for reading and thinking. The expertise must be maintained and further developed. Naturally also finishing my dissertation will require some time and energy, but this topic I will discuss later. During the following weeks, I am going to write some reviews and commentaries on some important books that I have read and that I can recommend to others as well. Some of these will be more suitable to the general audience, others more technical or mathematical.
To cite a few examples, I will present such authors as Pedro Domingos, David Vernon, David Mumford, Jeff Hawkins, Ray Kurzweil and Marcus Hutter. Central topics will be various ideas and techniques related to artificial intelligence and intelligent systems. I will also write about current topics, news and breakthroughs and comment on them.

projects and experiments

In my previous article I made a somewhat provocative and even boastful claim of being a person, who would in principle be able to implement alone a complete system from servers to several client applications. This kind of a person is often called a full-stack developer, though the typical meaning of this term is a bit more narrow and refers to a person with much more experience in such tasks. I do not market myself as a full-stack developer, but I do not wish for this claim to remain as light-hearted bragging, so I will prove it. My plan is to implement a system like the one I described, phase by phase, and document the process in this blog.
The goal of this series of article is, on the other hand, to make it clearer for someone who doesn’t know this field very well what kinds of things  are required to implement modern information systems. On the other hand, I want to describe the rapid development of technology during the last 20 years; after all, I have witnessed first hand the times when modern tools and frameworks were not available, and large parts of the systems had to be coded and configured manually with great effort. Nowadays we are in a situation, where it is actually possible for one person to carry out all steps on their own, in reasonable time no less.
In the articles I will discuss topics such as virtual machines, Docker, REST, JSON and Android. I will deviate from the mainstream in such a way, that I am going to use functional programmin, so I will write about such topics  as Haskell and Yesod. All this will, however, be just a framework for the main topic, which is of course learning and intelligent systems. This means, that I will write also about relational learning, in addition to neural networks and Bayes nets. Using technologies such as these in business requires a foundation and supporting framework, and I wish to demonstrate that I have sufficient understanding of the relevant techniques. Of course, I will require for my own purposes, such as technical demonstrations, some kind of basic system that is under my own control. I will describe the process of developing this framework in this blog.