Our Final Invention: Artificial Intelligence and the End of the Human Era Kindle Edition

I knew I was reading the wrong book when the author breathlessly describes the terror that an AI "1,000 Times Smarter than Stephen King" could bring to bear on humanity. If THAT'S the extent of AI's intelligence, I'm no longer concerned! ha. Don't misunderstand, I am in complete agreement that artificial general intelligence (AGI) definitely poses an existential threat to mankind. It could conceivable mean the end of our species. But if you're looking for a thoughtful, scientific analysis of the threat and how it may manifest itself, this is not the place to find it. As a Cognitive Scientist with some experience coding AI, I expected a lot more from this book. However, being writen by a documentary filmmaker, it simply can't overcome the author's lack of knowledge or understanding of the subject. It is basically a series of magazine articles wherein the author interviews "experts" in the field and basks wide-eyed in their splendor, all the while not really understanding what they are saying. Barrat doesn't understand that there is no scientific way to quantify "1,000 times smarter" than a human. He predicts that the entire human brain will be mapped and understood in the next year or two (wildly inaccurate). This demonstrates a juvenile and shallow mental model of intelligence. The brain is incredibly more complex and difficult to understand or model than Barrat realizes. He also fails to make convincing arguments for exaclty HOW the AGI will manifest embodied cognition (i.e. exist in a mobile physical state such as a body) so that it can replicate itself, acquire resources, and dominate mankind. Ultimately, when the author says he "fears" that this and that will happen, I have to ask: "Why should I care what you fear? - you don't seem to understand the issue!"

James Barrat, as a filmmaker and journalist, has layman's style that makes the topic accessible to non-experts. Some reviewers have commented that his lack of subject matter expertise is reflected in the weakness of his arguments. I disagree, Barrat went to great pains to interview many of the big thinkers on AI and its potential impact on society and humanity - including optimistic boosters, skeptics, and doomsday alarmists. While he ends up closer to the alarmists, I feel he got there by the strength of the arguments he heard along the way. If a machine can attain human level intelligence and is able to recursively improve its own code and therefore the rate at which it learns and becomes more intelligent, there are reasons to believe such a machine could exceed our intelligence many times over in a very short period of time. Well just unplug it, you say? The AI already thought of that and stored itself all over the place. It is not a machine anymore, it is an unbounded sentience. If we had it contained on one system completely cut-off from all networks, what strategies might it devise to convince us to connect it to the wider world? Promise a cure for cancer? The solution to clean cheap energy? Reversing climate change? if only it had a little more computing power.... Also consider this - what would a super-intelligence want, what would be its ambition? Just survival? Expansion? As Barrat cleverly puts it, maybe it has different uses for our human molecules than we do. And finally, consider that probably the most aggressive investments in AI are happening largely in the dark, by DARPA and defense contractors in the US and by God knows who in other countries. So why would we assume that the first sentient human-level AI will be friendly? These are the kinds of questions Barrat grapples with, and it is a heck of a ride.

The author is a film documentarian venturing into speculation about potential impacts of artificial intelligence from research to implementation. Specifically he evaluates likelihood and threats of developing AGI (Artificial General Intelligence) and eventually ASI (Artificial Strong Intelligence). His observations are based on extensive interviews including those with Kurzweil, Yudkowsky, Omohundro, Vinge, and Dyson among others. My initial reaction to this book was skepticism because not a scientific technologist. I expected that he may miss more subtle but important technical steps being taken on this road to artificial intelligence (AI). The further I read the more it became clear he is providing some pointed observations derivative of his experience as interviewer for documentaries. In general his conclusion is that AGI and ASI constitute existential threats as a function of the rapidity and manner in which they are developed. The process of development is not clearly established because of a diversity of technical opinion regarding both feasibility and impact. The range of opinion is very broad and nuanced. At one extreme is Ray Kurzweil whose many books on technology generally are most optimistic as among a group of those researchers with knowledge and experiences in this technological future. Though most optimistic he is also highly qualified not only as an analyst of tech trends but also developer of tech tools that, before his time, were regarded as difficult if not impossible. Among these is the optical character reader and some preliminary work leading to SIRI. He topped up his views with the most recent book “How to Create a Mind”. Though a summary of technical concepts it possesses many realistic elements in the work of such as Jurgen Schmidhueber and others working with neural nets. If Kurzweil is at one extreme Yudkowsky and Vinge are probably at the other. Both express sceptism AGI or ASI development will prove benign venturing opinions that work toward artificial intelligence should be severely curtailed to the extent of stopping short of artificial strong intelligence (ASI) specifically. In between these two extremes there are examples of opinions falling over a fairly wide range of future possibilities - increasingly probablities. The algorithmic avenue is already demonstrating some of the potential of AI. There are probably few finance and investment firms without one variation or the other of algorithmic high speed stock analysis and trading systems. These evince many elementary ingredients one may expect to see in future AI. So technically thorough as a matter of fact they operate relatively free of human interaction in producing recommendations for investments, effectively making ‘intelligent’, i.e. statistically valid, ‘decisions’. In meantime the advances continue unrelenting toward a distant ASI/AGI future. The time frames, for example, between IBM Big Blue and Watson are shorter than forecast, and end products as powerful as planned and then some. Still neither of these developments is more than steps on a road to AI while also being quickly followed by other developments such as recently announced SYNAPSE development by IBM. All closer steps to technological ingredients on the AI road to human future. There is some movement among AI researchers that a congress should be convened of the sort genetic researchers held in Asilomar California. That is, a convention to establish ground rules and limits on directions of AI research. One of the cautions about development progress of AI-like tools is based on the important role played by DARPA (Defense Intelligence Research Projects Agency) as it provides a large percentage of funding for various projects underway including an annual robotics competition to observe advances approximating many human qualities of movement. Clearly this agency has a mission antithetical to a purely humane result of AGI/ASI. After all DARPA is in the business of developing ‘weapons’ for military use – a not altogether benign mission in technology except perhaps as seen from point of view men at arms. The author mentions impact ASI and AGI will have on employment. His pessimism is mirrored in an Oxford University study concluding advancing tech developments pose an explicit threat to an estimated 47% of the 702 employment categories of the US Department of Commerce. While this report is an estimate it nonetheless raises the same sort of questions about computers in general, ASI and AGI in particular, and their impact on society. The report has recently been augmented with estimates of tech influence on employment in many other countries of the world. Another Oxford author is John Bostrom who outlines in great detail a road from our present to some future of AGI/ASI. A more recent development centers around Musk and Tegmark motivated by concern to fund and form an institute for evaluating threats and benefits. There is a persistent sense of threat from computers, automation and robotics dating from decades before the present. More recently this sense of threat seems to be accelerating concern about our human future with highly developed robotic associates. Barratt is a lucid presentation of the issues from a non-technical point of view.

Private Frazer: We're all doomed. Doomed!A well written but verbose account on the theme with variations of the rise of AGI/ASI and the inevitable demise of homo sapiens (probably). A kind of Frankenstein's monster lurking in the shadows in the not too distant future. These ideas are described to the reader ad nauseam. Frankly, once I got the theme (first few pages) I got bored of the variations/repetition and gave up about half way through. On the other hand, if you're a doom and gloom merchant, this will be right up your street...I was left wondering, though. What if ASI, instead of wiping us all out for its own ends, decides to seek enlightenment and becomes a Buddha instead?

Written by someone who is passionate about AI and understands the subject very well. Barrat may at times sounds slightly hysterical about the risks but what if he's right? I venture on a more optimistic side of the debate but he's produced a great piece of work that should be read by everyone interested in the future of AI and the human race.

Fascinating and slightly scary, this book deals with the development of AI (Artificial Intelligence) and should be a warning to humanity to start debating the ethics and establish a 'Robot Manifesto' before we rush into creating entities without safeguards or control systems to protect humanity.Well written and a book you should read if you are concerned about the future, yours! and ours as a species.

One of the most important books that you must read - to gain insight and understanding of the potential risks we may face if we rush headlong towards development of powerful AI, without due care & precautions/ protection in relation to the possible negative & disastrous consequences.

Great book. Very disturbing. Make me to think about our future.