Daniel Suarez - Daemon: Bot-Mediated Reality

Hello! I’m Alexander. I’m the executive director at the Long Now Foundation. Stewart unfortunately can’t be here tonight. He is accepting an award in Chicago, but I’m glad that all of you are in light of the opening games tonight of the Olympics. Clearly, some of you made it out instead of watching the smog in China. I’d also just like to thank those of you who did come to the Mechanicrawl. Actually, can I see hands how many people came? Oh, quite a few, awesome. So we had over a thousand people. We had no idea how many people would come and it was great over a thousand people come and check out all the amazedness that is in San Francisco’s Fisherman’s Wharf that nobody ever gets to see except for the tourists of course. Tonight, Peter Schwartz will be doing the Q and A after the talk, and I just wanted to give a little bit of the back story. It’s actually nice that Stewart isn’t doing this because Stewart is actually a bit of the fairy godmother of this story which is…He read a blog review by Rick Klau of this book. The book was sent to him by Daniel Suarez’s wife and total cold call used the thing that everybody says it doesn’t work and he actually read it, which nobody ever does and then he reviewed it, and he gave it a really great review. And Stewart saw it and then bought the book and put it on the shelf and there it sat for many months. And finally, he pulled it off the shelf when he was looking for a new novel and he read it and he was amazed. The book was really pushing new ground. It was doing it in a new way, not with necessarily new technology but by combining the current technology in a totally fresh and interesting way and in some cases, a pretty scary way. Since that time, he also introduced Daniel to Peter Schwartz who connected him to Walter Parks of Dreamworks and so there is now a Dreamworks movie deal on its way anyway and a two-book deal with Dutton Publishers. So the books that we have are the remainder copies from the original purchase we have. They’re all signed out there, but they’re the last ones of this self-published edition. So if you don’t get these, you are not going to be able to get the new edition until January 9th. They’ll be available here and in our store afterwards and until we’re out, you will be able to get them. So Daniel tonight is going to be talking about the kind of themes that he worked on in this book. He’s not going to be necessarily talking about the book itself but like Neuromancer and Snow Crash, this type of book when you read it, you realize you’re reading something new and important and you will be riveted and he puts together the technology in a way that I think is very important for our civilization the way the Vernor Vinge is now working on as well and here to talk about that tonight is Daniel Suarez. Thank you Alexander. Good evening everybody. Now some of you might be wondering what qualifies a novelist to speak on the topic of Bot-Mediated Reality, so I’d like to address that first off. Who the hell is Daniel Suarez? Well, I’m a software guy. For the last 17 years, I’ve been designing and developing database management systems for large companies. Now, I’ve been doing this in many industries, defense, finance, logistics, entertainment, and for the last seven years with my own company. Now, over the course of my career, I’ve noticed society’s collective pursuit of hyperefficiency. This had been a great concern of mine because it’s been a pursuit of efficiency at the expense of almost everything else. Now, efficiency manifests itself in several ways. First of all, centralization that’s companies merging becoming ever larger, achieving economies of scale and fewer people making decisions that affect larger numbers of people and that begets a monoculture that’s vast uniform networks, not just digital but agriculture, industrial, financial, and media. And part of this is also an elimination of redundancy that is trimming fact. Now in nature, there is a use for fact, is to deal with disruptions. Big business tries to minimize the amount of fact and part of it also is a drive for ubiquitous connectivity that is unifying all of these vast networks to achieve centralized data management and analysis and as all of these things in the pursuit of efficiency that make it possible to achieve widespread automation but I mean software automation or bots and that’s what I’m here to talk about tonight. Because bots are increasingly what we rely on to make decisions for us. As a matter of fact, we’re increasingly taking our instructions directly from bots and sometimes, just on faith, it takes a while. I think this is in the Olympic Village, I’m not sure. Now my book Daemon, for those of you who aren’t familiar with the scenario of that book, the story is one of a game designer who designs an array of bots that scan the internet for the appearance of his obituary and when it appears on his death, his bots launched into action and begin to dismantle civilization as we know it. Now, as far-fetched as that sounds, it didn’t escape my notice that IT security experts didn’t immediately dismiss this scenario and that should tell you something about the state of our culture’s infrastructure because I don’t think it would take a singularity or a greater than human machine intelligence to trap the human race. I think it could be done with much simpler systems. Because one of the things about narrow AI which is what bots are is that they are great at allowing a very small number of people to run the underpinnings of society. Well the great mass of people makes few or no decisions of consequence which is one of my big concerns. In fact, I think soon if not already, it might be possible for our society to run completely on autopilot, nobody at the wheel. So you may think, “Wow damn, that does sound pretty unlikely†but I’ll go over a couple of ways that might happen because to me that’s the fundamental question of our time whether technology, Oh sorry I’m bellowing here, whether technology will liberate us or enslave us. And I think we need to get busy answering that question and soon because bots are not going away, they’re going to be with us as long as civilization is here so we need to get this decision, is this done right. So before I continue, let me clarify some terms. When I say a bot most of you already know that it stands for robot and I don’t mean this and I don’t mean this and there are no circumstances do I mean this, fabulous dance from the eighties. What I’m talking about is this, it’s a software bot, and it’s a software application designed to accomplish a specific purpose. This is the source code for any commerce bot that scrapes pricing information out of websites and accumulates it into a database probably to be picked up by still other bots. So a bot does one a role of these three things. It searches for information, retrieves information, and acts upon information. And although that these seem very harmless, these are really the building blocks for the whole biological world because cells retrieve chemicals and enzymes and pull them into the cell to act as catalysts to create all of the activity in a multicellular organism. So these are really as I said the building blocks for the natural world. So together they’re very powerful and knowingly are now, we’ve replicated this in the digital world. Now, if you look at the last item act upon, that’s the key one because that might get his headed back in this direction at some point but unfortunately not even Danny Hillis or Alexander Rose can build one of this yet. So that leaves us with digital bots from now and digital bots are powered of course by AI and Artificial Intelligence basically breaks down into two mainly branches and I’m going to buzz through this because I’m sure you’re all familiar with it but there is Artificial General Intelligence Now, that’s strong AI and that is difference from narrow AI and that this doesn’t exist yet, but there’s question about when it might arrive. Now, a lot of the theories like Vernor Vinge’s Singularity Theory pontificate whether are all of our networks might combined and soon create some greater than human machine intelligence. Likewise, Ray Kurzweil talks about his the belief that by the end of this century machine and human intelligence will merged into one, and then of course there’s Kevin Kelly in his one machine concept that all of the processors on the world are starting to equate to neurons in the human brains but the key thing about this is that it has been 30 years away for about 50 years already. So the only people making any money on earth are the science fiction authors and Hollywood which leaves us with weak or narrow AI. And these don’t even try to approximate even animal intelligence. They do one thing very efficiently and they’re everywhere. They’re all throughout society. As a matter of fact, they power the modern world and that’s because the modern world consistent largely of data. We’ve created mountains of data and they aren’t enough people to go through it all and bots are supremely efficient at pouring through mountains of data that the modern world creates and that’s what gives them so much power over us as people. Finance, telecom, security, and manufacturing all creating these data it’s worth noting that bots determine whether you get a mortgage such as FICO score. As a matter of fact bots might determine whether you get a job interview because resume scanners look for keywords but bots also affect us in a much more personal level even than that. Radiology and MRI, these films are examined prescreened by bots for follow up by doctors if an anomaly is detected. So instead of hiring a hundred doctors you can hire ten. It’s hard to imagine the more direct impact on your life as a matter of fact you better hope they’ve written these bots well but of course bots are also scanning for all of your medical records to see whether you need to have your medical insurance cancelled. So they’re mining all of these data but they're also listening to us as well and speaking to us if you called an airline or credit card company recently you probably spoken to a bot actually within the confines of a given business transaction, bots can understand and speak reasonably well with humans. As a matter of fact, typical synthetic speech voice based entirely on text, “synthetic voices seem rather fanciful.†Now, I have one with the Kentucky accent but that seemed sorts for here but these voices allow a cost reduction of up to 90% in call centers and that’s the key once again is that bots are more efficient than human but of course bots are also listening to us in other ways. Bots are largely responsible for the removal of the limitation on wiretapping because bots can be put onto the line to listen for key terms and once again notify a human being for follow up. So that’s the key thing they removed any limitations on wiretapping but bots are of course also watching us. The ballooning number of surveillance cameras out in the society is made possible largely by software because there aren’t enough Orcs in Mordor to watch all of these computer monitors. million in the city of Shenzhen, China and its software that watches these monitors looking for suspicious behavior like a sudden grouping of six or more people they can There are five hundred thousand surveillance cameras in the city of London and two increasingly recognize us by the way we walk, by our facial features and also they can identify people by race. So bots are really an unblinking eye and ever listening ear and of course they make key decisions about our lives based on our data. Well that’s why it’s worrying that are collective control of the internet seems to be slipping somewhat. Now this graph shows the growth of malware that is botnets in the last two years. Now during this time there was consorted effort by government, universities, private industry to contain the spread of botnets that is armies of zombie computers controlled by bots, and yet during this time, these botnets spread 2162% and that’s despite the consorted effort to stop them. So I would argue that we’re in Darwinian struggle with narrow AI, and that nature is currently selecting for bots and against humans and one reason efficiency. We’re just not as efficient in a structured society as bots. It’s partly due to a confluence of processing speech, storage, space increase, the spread of the internet and bandwidth that's made this possible. And anytime you bring a human in to do a job they bring a whole bunch of other stuff in with it like emotions, philosophy, goals, which bots do not bring. So having a big brain as we do is no guarantee of success in nature. In fact most organisms on earth do not have a big brain and do quite fine. Parasites for instance don’t even have a central nervous system much less a brain and they out number all other organisms of earth three to one. So having a big brain is no guarantee of success and as a matter of fact humanity in general is not guaranteed a staring role in nature’s pageant. That’s why it is important to understand the scope of this situation. Just look at the numbers. This is the growth of the internet over the last 15 years come from 1995 projected out to 2010. From 16 million to 1.6 billion people, well that’s people using the internet but of course has that network expands so does the environment for programs. So this really isn’t a perfect measure of the growth of the bot ecosystem. A better measure would be hard dress base on all of planet earth. Fortunately, someone is keeping track of that as well and then the next 5 years, it’s projected to go up a thousand percent which is much steeper than the growth in human users and we all know that our hard drives are expanding in size and memory and that creates a vaster environment for bots to run in. So we must be getting some benefits out of this if we’re building this vast network for these synthetic organisms. Well, pardon the purple it’s actually blue on my screen. Human race is expanding in numbers and that’s good we learn how to fix nitrogen into the soil, but what about us individually? What about the value of human life? There’s a little different story and this is a disagreeable looking slide because it’s about slavery and many people think that slavery is a thing of the past but unfortunately we are living right now through a golden age of slavery or 27 million slaves known in the planet right now. It’s more than at any point in recorded human history. So we look in the United States what a healthy male sold for, adjusted for inflation it was a $40,000 in 1850. In the year 2000, traffickers sell a healthy male labor for $600. So clearly our human stock is dropping which is surprising because we’re big brained animals, we’re the most innovative, we’re capable of great flexibility and yet a rack server a good one goes for a about a $1000. So who was using this network more, this internet? Machines are humans. Ah, sad news there too, by 2010 Ziff Davis projects that machine to machine transactions will out number all humans use over the internet and partly this is because of EDI Electronic Data Interchange multi-companies out there leveling inventories, automatically ordering things from each other. There are also sensors on pipe lines and smart phones and location tracking devices, but I put forward that we’re hooking and injecting into the internet hundreds of millions of bots without any clear considerations about the consequences to humanity and also there is no broad examination going on about how these bots might interact with each other and that’s without advanced intelligence just simple reactions. A good example of that was Wall Street in 1987, October 19th, 22.6% of the value of Wall Street disappeared in one day and was because of program trading bots reacting to each other. It’s half a trillion dollars in value. Remember also that bots don’t observe borders. They don’t share our priorities. They don’t need clean air or water and they don’t need liberty either. So, I would suggest that bots are potentially a vector for human despotism as well. Because very few people are needed to run systems with bots and as fewer people make decisions fewer people are needed to make decisions and they don’t learn to make decisions and that makes a smaller number of people, knowledgeable enough to sustain civilization and we might lose our ability to control society. So, who’s building all this bots any way? Usual cast of character of course corporations build tons of them to encapsulate business knowledge Governments create mountains of data that corporations and governments have to mine. You might be surprised to see religions up there. Religions do things like protect their Wikipedia page. They might create a bot to keep an eye on it make sure nobody changes it, and the good thing about the bot is it works 24/7 and it doesn’t have to obey the Sabbath. Criminal groups make some of the most sophisticated bots out there because they have to be stealthy in addition to doing what they’re doing, and of course they are just every individuals student researcher or hacker, grey hat, white hat, or black hat creating this big galaxy of software that’s doing who knows. And this zoo of bots I divide in to two families, datavores that is bots that munch on data and predators, and you might be surprised to see game bots on the predator site, but actually they inform the malware and botnet family, because actually game bots are where some of the most sophisticated software is being made right now. The games, online games in particular reveal a lot about the future I think for social interactions and especially about our interactions with bots artificial life. So we have a lot to learn actually from games. I know it's shameless to use kittens, but I’m actually showing this to make a point. All juvenile mammals indulge and play. It’s our way of probing new realities and of learning our place in them and so I would argue that this is fundamentally the same thing as this because people are probing new realities. Now as work becomes more network based and less reality based, this is very useful. I think it’s one of the reasons why people play games now well into adulthood. It’s because the world is changing so fast, they were continually juvenile and at last one aspect of their lives. Games, online games in particular are also where humans interact most visibly with artificial life forms bots because there are bots all over the place in on line games. This is the game Second Life and this is the reporter Wagner James Au conversing with a chat bot. Now, the difference here is that this bot looks like a bot. The truth is in most online games, bots look just like we do in games. That is a 10-foot panther with twin scimitars, but they’re indistinguishable from us visually. The truth is nobody knows how many, yeah I know it cover the children’s eyes (Indiscernible). Nobody knows how many bots there are in on there are in online games. I should have known better to have a scrolling that’s quite interesting. But you take a game like world of Warcraft, that’s an economy of 10 million people and that’s bigger in some real world, small countries actually, I think I might pause it. I should probably pause it before it gets to the really racing part. So, if you're in World of Warcraft and you are competing against bots because people write bots to play World of Warcraft and I don’t mean to pick on Blizzard here because any successful online game where you can sell virtual things like virtual gold or magic items on the open market, efficiency counts and wherever efficiency counts you’ll find bots and so games like World of Warcraft have bots in them and that makes it so that players, human players are competing in this virtual space for virtual gold and for quests against artificial life forms and this is not a science fiction thing. This is really happening everyday and as more bots appeared and players became irritated, publishers like Blizzard created programs like Warden, and what Warden does is it tries to find bots running on your computer with the game client and as Blizzard started trying to find bots, bot developers made them stealthy and it created a classic Darwinian struggle where the predator was chasing the prey and the prey adopted a strategy of camouflage and as a result rootkit technology is now being driven in mostly by games, by games bots because that’s where truly stealthy software is being created on a daily basis and it’s sort of an incubator. Here by the way is an example of a botting system. This is one system playing nine characters in the World of Warcraft. So, somebody who went through the trouble of writing a program that can play the game for you, but in this case, such a person can make hundred of thousands of dollars in the real word by selling the items that these bots gather and once again stealth, I was able to sit down with Dr. Gary McGraw of Cigital Incorporated He is a foremost software security expert. He is also with Greg Hoglund the author of the books Exploiting Online Games and Exploiting Software, and he had this to say on the subject of game bots and it’s implications for security software in general The most interesting thing that’s happening now is you want to build botting systems that’s totally undetectable, so that you can have this game be played automatically and you don’t have to be there, be present. You build a bot or an AI program to play the game for you, but it has to be undetectable. So, a lot of work has gone into stealthyness and hiding processes and things that rootkits traditionally do. The interesting thing that happened is the whole thing shifted and now in my view rootkit technology is actually being driven by gaming exploits. So, as an example of how gaming technology is leading rootkits can think about an undetectable bot system that uses the fact that most PCs today are multi core systems. So, strangely the game companies have decided to monitor the game process in UserLand on the same processor However, if you use hardware interrupt and just stopped that processor, guess what else stops… the monitoring software. Then, you do some other exploit processing on the other core, inject the payload and off you go, and since there’s no Rip Van Winkle monitoring capability yet on these things, you can get away with that and build an insanely undetectable bot these days. So, who would’ve thought that such sophisticated software will be put towards playing games? That’s why I think online games are the larval stage of something much more serious in both our society and in our relationship with bots in general. This is an animation that shows major botnet activity over a seven-day period in the world and once again, botnets are armies of compromised computers. Now in red, you’ll see the Storm botnet and orange is the Kraken botnet. You noticed as the work day passes, the activity booms! It’s like an algal bloom. It’s very much a natural system and this is what I mean about botnets evolving serious stealth because remember, no one can stop these botnets, no one. Their numbers anywhere from half a million and, I love these the upper end 20 million, that’s how wide the spread is. Nobody’s quite sure how many there are out there. The creators of these botnets– once again, nobody really knows who they are but there are very small number of people who have a great deal of power because they can rent time on these botnets to either send spam, or to direct distribute denial of service attacks against any target on the internet and that has real world consequences. I’ll give one example. A company named Blue Security in Herzliya, Israel in May 2006, this is an internet security company, they try to defend its clients from an attack knocking their clients off the internet and the botnet then turned to Blue Security, focused its attack on them, prevented them from connecting to the internet no matter what they did, and they eventually went out of business; and then the botnet turned its attention back on its original targets. That was in Wired magazine in May 2006. The InformationWeek has said that the biggest botnets can knock entire countries off the internet and the Kraken botnet is estimated to have already penetrated 50 of the Fortune 500 companies; and remember, a very few individuals control this and they’re the number one threat to our collective control of the internet, which is worrisome in a way because bots are increasingly passing for human in everyday communications. Now, by that I don’t mean they’re passing the Turing test; this is more of a short form Turing test. You look at the way we used to communicate writing beautiful letters began in 19th century “Dearest Martha, the children and I are fine.†Everybody had different hand writing and we even had difficulty reading each other’s handwriting, but today, very often "omg" and "lol," simple symbolic language, but again that’s not the way we typically communicate. Very typically, we’ll go to a website. We’ll choose from dropdown combo lists, multiple choice items. These are all things bots are capable of doing and it’s because we’re trying to be highly efficient in our communications and accelerate things that is made the bar lower. CAPTCHAs can also be defeated by bots. Those are the completely automated public Turing test to tell humans and computers apart. A little squiggly lines at the bottom of a webpage. Those can now be cracked in 60 seconds by bots. So, that line is blurring. So, when we break tasks into bot-like repetition as we are doing, we removed the value of the unique adaptability of the human brain when we break it down into simple structured systems. Human beings are also programmable. As we’ve seen in certain social setups. Any society that values obedience and devalues independent thought goes into this category. Now, the disciplines of propaganda and advertising have been honed to a razor’s edge to press our buttons and get a predictable response. So, if you can’t tell a human from a bot what’s to prevent a bot from pushing your buttons from getting you to react in certain specific ways? We might think that would require that we’d be visible to bots moment by moment and actually, as bots are exploding in size and influence over us so too is our visibility to them. As a matter of fact, you’re probably wearing this handy tracking device, the cell phone. Every few seconds its position is recorded in three-dimensional space and that data doesn’t go away, it accumulates. Same is true of your financial transactions. All these things build a picture of you, who you are, and what you’re doing Now, Bruce Schneier, the renowned cryptographer and security expert, calls this data pollution. That’s the cost of gathering data, storing it, retrieving it, has essentially dropped to zero which means data never goes away so that over time, you can go back and determine where somebody has been for years every single moment. Remember, bots are uniquely good at mining data and you may not have a problem with this actually. You might trust your government or whomever but what about 10 years from now or what if that data gets sold or maybe even to a foreign government and if you travel that might be a concern as well because it builds a complete profile of you. Probably, more than you know about even yourself. In 2005, MIT research in Nathan Eagle did a study called reality mining where they track a thousand cell phones as they moved around campus and at the end of that study, they wrote an algorithm that could predict with 85% accuracy what any individual was going to do next and that’s the power of simple software. No conspiracy necessary. So, you might wonder what sort of data is being gathered on a routine basis out there in reality. Now, this is a typical urban street and once again, this is today; we zoom in, we can see surveillance cameras. Of course, these are wired in once again to a system, the data’s archive, and software is analyzing the images, keeping it for long periods of time but those aren’t the only surveillance cameras out in the world, there’s private surveillance cameras. Possibly, nowhere near as well secured data for sale to divorce attorneys and then off course, up at the end of the block, some weird guy with a webcam pointed at the street, once again, available to the web, visible to bot. All of these visual data creates a context for the other data points that are being gathered by cell phones, by smart phones, emails, and so forth. Then of course, there’s the almost legacy process of gathering license plate numbers programmatically to track automobiles but that’s less and less necessary because new technologies make it easier. We’ll pauses here. Many of you have Bluetooth devices in your car but you may not know about the TPMS system; this is the tire pressure monitoring system. It was federally mandated by the 2001 TREAD Act. That’s right. You all remember voting for this, right? It says that any car manufactured up to 2007 has to have wireless nozzle pressure measurement devices that communicate with the computer onboard the car to see that your tires are safely inflated. Now, they have to have a unique I.D. so that the computer knows your tires from the car next to you and of course, it is an open standard and makes it very simple to track the unique identity of an automobile; but of course, to do that you would have to have devices scanning. Fortunately, such scanners have started to spring up at choke points throughout modern cities. These are privately owned scanners with the data being gathered and stored again because it’s cheap to store data, vast amounts of data. This data can be piled up along with your financial transactions and anything else and bots can go through it to find persons of interest or they just find patterns or even just to sell you stuff. I’ll give you an example of just a few such devices as a BlueSweep scanner and a BlueSweep scanner is a device that able to identify all bluetooth devices within its radius, identify what there capabilities are, and what exploits they might be vulnerable to. A BlueSniper can do this up to a kilometer away. Let’s go a little further down the wall. There’s the Bluesnarfer you were all expecting. Now Bluesnarfer can use an exploit and given to it by a Bluesweeper to steal your address book, your text messages, your calendar, your pictures of your kitties, and bluetooth car whisperer can push advertising into your car speakers through your car’s bluetooth system. Now more worrisome, it could also be used to hook into your car bluetooth phone system to eavesdrop on conversations in the car. Now, if you combine that with something like the TPMS system or any future open standard device, you could pretty much track a car and listen to it’s occupants as they move throughout the city at any point in the future or at the moment it’s happening. Now, so you’re walking through this gauntlet of scanning activity with all the wireless devices and again, I’m sure we were all aware of this, and then there’s of course financial transactions every time we buy stuff with a debit card or a credit card. Who, what, where, and when? Combine that with visual data and all of the other points that tell us who was there with you, where you were going can be used to tell some very interesting stories. So it’s a great constellation of information being gathered on us at all times and then of course privately owned devices Hoovering up all these information. So this is the world you live in right now. Who knows what it will be like 10 years from now? And this goes into the increasing complexity of the bot ecosystem in general because of all these corporate mergers have created a vastly complex system that no one really understands. You take mergers where they lay off half the IT staff, if half finished projects, maybe bots running that are completely forgotten doggedly pursuing some original purpose that no one knows about and also maybe even competing with each other even though the company is now one and not two. And then of course there’s the occasional disgruntled employee who takes over the network and I know that sounds impossible but…so this could lead to a loss of human authority. Humans no longer fully being in control because it’s really a system where historical decisions rule. Somebody or some group of people created an algorithm that decided something was true a while back and it is put into force and other people obey what it has to say and over time, it becomes a black box. A proprietary mechanism that might in turn be folded into yet another algorithm which becomes a black box and less people understand the proof of the logic that went into that algorithm. Kurt Vonnegut wrote a book called, The Player Piano where society was completely on autopilot. Bots run everything including the factories. But I wonder if bots would be the ones working in the factories because we might get the crap jobs because you don’t need to have a body to manage a structured process and that’s what society has become, a structure process. Humans offloading decision making authority and ask yourself, “Would you notice?†Do you really know that 75% of the decisions being made at your bank aren’t being made by bots right now? Probably not So rather than rising to some great complex golden age, I am concerned that human civilization might take a side track and head towards a boolean age that is a time categorized or described, that’s a constant bombardment of categorical questions that you must answer. You can’t post any questions that aren’t asked directly of you. I think we see elements of that already, so that’s a structured existence in a structures society and it is supremely efficient. And I think such a thing could give rise very easily to an elite group of people who would be the ones who understand how society works and everybody else just answering the questions. Here’s a quote from Carl Sagan’s book Daemon-Haunted World: Science as a Candle in the Darkness. Carl Sagan wrote “Science is more than a body of knowledge. It is a way of thinking. I have a foreboding of an America in my children’s or grandchildren’s time. When awesome technological powers are in the hands of a very few and no one representing the public interest can even grasp the issues when the people have lost the ability to set their own agendas or knowledgeably question those in authority when clutching our crystals and nervously consulting our horoscopes, our critical faculties in decline, unable to distinguish between what feels good and what’s true, we slide almost without noticing back into superstitions and darkness.†And I think that’s the real danger there and having a small group of people understanding how society works. So that bots has control mechanism, economic disenfranchisement at the flip of a bit and you might not even know what went into making that decisions about you because remember we have vast quantities of data about us and bots are making qualitative decisions about us all the time. As a matter of fact it might not be a person who decides who makes that decisions it might be an algorithm. What if control erodes further though? What if the last person who knew how to stop something or the last person with the passwords dies or quits? I think we could easily picture a bot controlled corporation because corporations are already termed unnatural persons in the eyes of the law. They have all the rights of the human being, privacy, freedom of speech, with the big difference being that they don’t ever die and that’s very similar to a bot. So what if steps were made by corporations to give civil rights of some type to bots for instance that you don’t have the right to delete them from system and that sounds a lot like DRM actually come to think it because you got to ask yourself who controls the data on your systems and in these systems. That’s a perfect example of humans ceding some of their rights to bots. So where does that leave us? Actually I don’t think it’s this bad; although, I have painted a very dreary picture. If I count through it I said that bots environment is exploding in size, their power over us is increasing vastly. Humanity’s value was plummeting and we are losing control of civilization. But I do think there’s hope because one thing humans do supremely well is adapt. We’re incredibly flexible and adaptable creatures. Now bots are inflexible without us and even primitive human societies can innovate tremendously when necessary. And in nature flexibility typically wins so if we’re to start over what would we do differently? Well this is The Long Now Foundations so let’s think big. What if we built a new internet just for starters but this time built it to benefit the greatest number of people possible. What if we try to make it support our preferred way of government that is democracy because I think in the past, we built systems as a tool for society. What if we made systems, IT systems the fabric of society itself, a reflection of it? Could we build an operating system for democracy? Could we hard code our values into the DNA of civilization? I think we might be able to. I think a way that we could start this time would be by building an encrypted network, a Darknet to begin with that is not open automatically to all comers. I think we’d have at least a couple of qualifications for entry into this encrypted network. One would be that you are a verifiably human and not to be too exclusive, I think we could let some bots in because they are useful when they behave but of course, we would have to know the source code of these bots. So any bots that we would let in into this proposed system would have to be white listed. That is a critical mass of humans would need to review the code and agree that they are benign and of course, any misbehaving bot must be able to be banished by collective human action, that is a critical mass of humans voting them out and this would have to be baked into this network from the very start and I think we would also have to have a secure cryptographic hash algorithm to make sure that the source code of these bots doesn’t change and once again, if it does, bots are automatically expunged. But that still leaves the question of how to differentiate bots from humans which is what started this after all, because they are a potentially aggressive organism and I think the answer to that is literally all around us. I think it’s three-dimensional space because if we project this new network on top of reality, it’s very difficult for a bot to pretend to be us. This has been called augmented reality and this had been kicked around for a few years but I think self defense against bots us developing a new sensory perception is an ideal reason, a driving reason to implement such a system, because that’s what we need to do, is to develop new senses. Now, in my book Daemon, such a system is implemented. It’s called D-space and that by projecting data on to three-dimensional space and seeing bots and other organisms coming at you when they want to change your data that really simulates what’s happening. Because anytime your data changes, it is potentially a threat to you. But the human mind is wired to see it graphically and if we created an Internet much like this, we might better be able to defend ourselves against these things. Now we would need heads-up glasses or haptic display, but we wouldn’t need to wear them all the time. I think we could get a buzz on our smart phone that tells us some thing’s changed and we could put on our spectacles and look into the next dimension. I think we could do that. But the key is that in this type of network, you would be in control of your data. Anytime it changed, you would know it. Because in the modern world already you are your data, if your data goes bad, you’re in a heap of trouble. So this really is self defense and I think this Darknet will operate in parallel to the existing Internet and I think people would opt into it to get control of their data if nothing else and especially, I think they would be willing to join a network where human action, collective action could banish bots. And I think this is already familiar to us because we’re already doing this in games, carrying callouts over our head, projecting data on to three-dimensional space. When I said that online games are a larval stage of something more serious in our social relationships and with our relationships with bots, this is what I meant. I think we’re practicing for something in online games. Social networks, Wikis, online games, we’re exploring new realities. We’re already working on this. As a matter of fact, we already have the coordinate system worked out. It’s called GPS. So we build big pieces of the infrastructure of this type of network. But how would a new network like this, a Darknet work person to person. I think it would look very much like an online game, walking with a callout to those on the network, carrying your name, your reputation, your experience, your skills so that people can recognize each other. Now people who are privacy advocates might look at this and say, “Oh my God, this is a nightmare!†But I would point out that your data is probably being sold to somebody just this moment and at least in this new network, you would know anytime someone touched your data. You would know it instantly. You would receive your ratings in such a system by your past interactions with other network members. As you dealt with them, they would rate you and you would rate them and you would classify their skills and this would start to develop an open source database on what this civilization can do collectively. It would keep its collective records as part of the system and it’s good to have reputation in context especially if you’re trying to buy a car or something like this, like the under coating. I wonder if it would encourage like-minded people to hang out together to get better ratings and I’m not sure that that’s bad. I haven’t decided yet, but it might make for some interesting changes in society. But what about bots pretending to be human, because of course, that’s one of the reasons we’ve designed this. It would be very difficult for a bot to masquerade as human in such a system because remember, illicit bots, bots that do not have their code approved by a critical mass of people would be banished instantly, which would leave bots masquerading as humans as the only way to get around. But of course, a callout hovering over nothing is a dead giveaway and hiring people to hold your bot for you does not scale very well. It’s very expensive. So this has a sort of built-in natural protections against that, and I do think that this type of social arrangement has even greater implications actually. Because bots, although they are a threat, we’ve dealt with threatening organisms before. We’ve developed symbiotic relationships with threatening organisms before and the very disinterestedness of bots in us is their great usefulness, because in partiality, is a cornerstone of blind justice and blind justice is the cornerstone of democracy. So one could picture that open source bots, that is bots that have their source code reviewed by everybody and available to all could be supremely useful and impartial about human affairs. And this would be a bot-mediated reality. This would be bots examining what’s going on in the real world and moving information and decisions through civic society on our behalf. I’m going to get a laser pointer for this and step you through this imposing diagram. So