The Singularitian movement. The only problem is that academics don't seem interested. Call me when you have running code. Alon Halevy, a faculty member in the University of Washington's computer science department and an editor at the Journal of Artificial Intelligence Research , said he's not worried about friendliness. The challenges we face are so enormous to even get to the point where we can call a system reasonably intelligent, that whether they are friendly or not will be an issue that is relatively easy to solve.
Rules limiting smart computers to human-approved limits, of course, are nothing new.
The most famous example is the Laws of Robotics, written by Isaac Asimov, one of the fathers of science fiction. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws. In 1993, Roger Clarke , a fellow at the Australian National University, wrote an essay wondering how Asimov's laws applied to information technology. Codes generally fail to reflect the potential effects of computer-enhanced machines and the inadequacy of existing managerial, institutional and legal processes for coping with inherent risks. When one researcher booted up a program he hoped would be AI-like, Yudkowsky said he believed there was a 5 percent chance the Singularity was about to happen and human existence would be forever changed.