The Edge Question of the Year 2015

WHAT DO YOU THINK ABOUT MACHINES THAT THINK?

What’s wrong with turning over the drudgery of thought to such high-tech marvels? Nothing, so long as (1) we don’t delude ourselves, and (2) we somehow manage to keep our own cognitive skills from atrophying.
(1) It is very, very hard to imagine (and keep in mind) the limitations of entities that can be such valued assistants, and the human tendency is always to over-endow them with understanding—as we have known since Joe Weizenbaum’s notorious Eliza program of the early 1970s. This is a huge risk, since we will always be tempted to ask more of them than they were designed to accomplish, and to trust the results when we shouldn’t.
(2) Use it or lose it. As we become ever more dependent on these cognitive prostheses, we risk becoming helpless if they ever shut down. The Internet is not an intelligent agent (well, in some ways it is) but we have nevertheless become so dependent on it that were it to crash, panic would set in and we could destroy society in a few days. That’s an event we should bend our efforts to averting now, because it could happen any day.
The real danger, then, is not machines that are more intelligent than we are usurping our role as captains of our destinies. The real danger is basically clueless machines being ceded authority far beyond their competence.

Daniel C. Dennett

previously, The Edge Questions of the Years 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, & 2014