trentmkelly/LessWrong-43k huggingface dataset:
AGI in our lifetimes is wishful thinking
by niknoble — 2022-10-24T11:53:11.809Z

A few facts:

Solving intelligence is the most important problem we face because intelligence can be used to solve everything else
We know it's possible to solve intelligence because evolution has already done it
We've made enormous progress towards solving intelligence in the last few years

Given our situation, it's surprising that the broader world doesn't share LessWrong's fascination with AGI. If people were giving it the weight it deserved, it would overshadow even the global culture war which has dominated the airwaves for the last 5 years and pulled in so many of us.

And forget, for a moment, about true intelligence. The narrow AIs that we already have are going to shake up our lives in a big way in the immediate future. The days of the art industry have been numbered ever since the announcement of DALLE-1.

Unfortunately, despite my general enthusiasm for machine learning, I have to take issue with the common sentiment on here that AGI is right around the corner. I think it's very unlikely (<10% chance) that we'll see AGI within the next 50 years, and entirely possible (>25% chance) that it will take over 500 years.

To be precise, let's use the Future Fund's definition of AGI:

For any human who can do any job, there is a computer program (not necessarily the same one every time) that can do the same job for $25/hr or less.

Wishful thinking

My core thesis is that the people on this site are collectively falling victim to wishful thinking. Most of them really want to experience transformative AGI for themselves, and this is causing them to make unreasonably aggressive predictions.

The first clue that wishful thinking might be at play is that everyone's forecasts seem to put AGI smack in the middle of their lifetimes. It's very common to see predictions of AGI in 20 years, but I don't think I've ever seen a prediction in the 200-500 year range. This is strange, because in the broader context of human progress, AGI in 2040 and AGI in 2240 are not significantly different. Those years are side-by-side in a zoomed-out view of human history.

I'm reminded of Ray Kurzweil and Aubrey de Grey, who conveniently predict that immortality will be achieved right around the time they reach old age. Both of those men have a long enough history of predictions that we can now say pretty confidently they've been overoptimistic.

More generally, it's hard to find anyone who both:

Fee

...
TOMT: Post from 1-2 years ago talking about a paper on social networks
by Simon Berens — 2022-10-24T01:29:11.453Z

If anyone remembers this post I would greatly appreciate a link!

IIRC the authors posted a link to their paper on lesswrong. The paper was about 2 flaws of current social networks and how to fix them. I think the first flaw was duplicate content.
AI researchers announce NeuroAI agenda
by Cameron Berg — 2022-10-24T00:14:46.574Z
Link: https://arxiv.org/abs/2210.08340

Last week, 27 highly prominent AI researchers and neuroscientists released a preprint entitled Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution. I think this report is definitely worth reading, especially for people interested in understanding and predicting the long-term trajectory of AI research.

Below, I’ll briefly highlight four passages from the paper that seemed particularly relevant to me.

Doubts about the 'prosaic' approach yielding AGI

The authors write:

The seeds of the current AI revolution were planted decades ago, largely by researchers attempting to understand how brains compute (McCulloch and Pitts 1943). Indeed, the earliest efforts to build an “artificial brain” led to the invention of the modern “von Neumann computer architecture,” for which John von Neumann explicitly drew upon the very limited knowledge of the brain available to him in the 1940s (Von Neumann 2012). The deep convolutional networks that catalyzed the recent revolution in modern AI are built upon artificial neural networks (ANNs) directly inspired by the Nobel-prize winning work of David Hubel and Torsten Wiesel on visual processing circuits in the cat (Hubel and Wiesel 1962; LeCun and Bengio 1995). Similarly, the development of reinforcement learning (RL) drew a direct line of inspiration from insights into animal behavior and neural activity during learning (Thorndike and Bruce 2017; Rescorla 1972; Schultz, Dayan, and Montague 1997). Now, decades later, applications of ANNs and RL are coming so quickly that many observers assume that the long-elusive goal of human-level intelligence—sometimes referred to as “artificial general intelligence”—is within our grasp. However, in contrast to the optimism of those outside the field, many front-line AI researchers believe that major new breakthroughs are needed before we can build artificial systems capable of doing all that a human, or even a much simpler animal like a mouse, can do [emphasis added].

To the degree we take this final comment seriously—that many within the field think that major breakthroughs (plural) are needed before we get AGI—we should probably update in the direction of being relatively more skeptical of prosaic AGI safety. If it appears increasingly likely that AGI won't look like a clean extrapolation from current systems, it would therefore make increasingly less s...
Empowerment is (almost) All We Need
by jacob_cannell — 2022-10-23T21:48:55.439Z

Intro

What/who would you like to become in a thousand subjective years? or a million?

Perhaps, like me, you wish to become posthuman: to transcend mortality and biology, to become a substrate independent mind, to wear new bodies like clothes, to grow more intelligent, wise, wealthy, and connected, to explore the multiverse, perhaps eventually to split, merge, and change - to vasten.

Regardless of who you are now or what specific values you endorse today, I suspect you too would at least desire these possibilities as options. Absent some culture specific social stigmas, who would not like more wealth, health, and power? more future optionality?

As biological creatures, our fundamental evolutionary imperative is to be fruitful and multiply, so our core innate high level value should be inclusive genetic fitness. But for intelligent long lived animals like ourselves, reproduction is a terminal goal in the impossibly distant future: on the order of around 1e11 neural clock cycles from birth[1], to be more precise. Explicit optimization of inclusive genetic fitness through simulation and planning over such vast time horizons is simply implausible - especially for a mere 20 watt irreversible computer such as the human brain, no matter how efficient.

Fortunately there exists an accessible common goal which is ultimately instrumentally convergent for nearly all final goals: power-seeking, or simply: empowerment.

Omohundro proposed an early version of the instrumental convergence hypothesis as applied to AI in his 2008 paper the Basic AI Drives, however the same principle was already recognized by Klyubin et al in their 2005 paper "Empowerment: A Universal Agent-Centric Measure of Control"[2]:

Our central hypothesis is that there exist a local and universal utility function which may help individuals survive and hence speed up evolution by making the fitness landscape smoother. The function is local in the sense that it doesn’t rely on infinitely long history of past experience, does not require global knowledge about the world, and that it provides localized feedback to the individual.

. . .

To a sugar-feeding bacterium, high sugar concentration means longer survival
time and hence more possibilities of moving to different places for reproduction, to a chimpanzee higher social status means more mating choice and interaction, to a person more money means more oppo

...
"Originality is nothing but judicious imitation" - Voltaire
by Vestozia — 2022-10-23T19:00:02.732Z

TLDR: Throughout human history pattern recognition, inspiration, and reasoning have been at the core of most innovations - including artificial intelligence. It is theorised that a code-generation model with the ability to demonstrate the aforementioned parameters should be able to build recursively better AI systems. This is the current realised 'holy grail' of AI and typically referred to as the singularity, in which there appears to be a very highly likelihood (>80%) of it being surpassed within the next 14 years. Naturally, the advent of artificial general intelligence (AGI) would occur shortly-after (~15 years), alongside the development of a superintelligence. As organisations compete to be the first to build such a technology, processing power continues to increase, access to training data surges, and new algorithms and approaches to machine learning are being discovered.

Previously, I had written a post titled 'Objects in Mirror Are Closer Than They Appear' in which I briefly speculated on how a recursive self-improving AI system would soon lead to AGI. Following several discussions within the field, I was guided to three publications[1][2][3] that have further solidified my stance on which approaches may lead to AGI, but also greatly shortened the timeline on when I believe such a threshold will be surpassed.

In order to render this a comprehensive argument without necessitating the read of my previous post, I will rehash several passages from it which I continue to deem relevant to my updated answer of why 'AGI will be developed by Jan 1, 2043'. Now with that out of the way, here is why (and how) a novel algorithm discovery, symbolic reasoning, and code-generation model will result in an intelligence explosion.

Defining Artificial General Intelligence

To ensure this post is an accessible read, I would like to define some core vocabulary and concepts that will be used throughout, in order to avoid conflict of interpretation.

The commonly accepted definition for AGI amongst academics states that 'an agent should possess the ability to learn, understand and preform any intellectual task that a human can'[4]. Though this may appear as overly ambitious to many, one must remember that our own cognitive abilities are the result of neural activation patterns which simultaneously permits us sentience and to experience qualia. By applying this computatio...
I am a Memoryless System
by Nicholas / Heather Kross — 2022-10-23T17:34:48.367Z
Link: https://www.thinkingmuchbetter.com/main/i-am-a-memoryless-system/

Author's Note: this is my entry for the Edit Your Source Code Contest. I am an undergrad student at RIT, NY.

commit 85a4c4f37966a739e88c0a4c70946bd4 (HEAD -> master)
Author: demo
Date: Thu Oct 27 09:38:12 2022 -0400
Initial mind state

I'm sitting in Intro Psych on a Thursday, I think, and I'm staring at a black square with my mind on it.

Well, not technically "a black square with my mind on it". It was more of a rectangle, with the little flip-out keyboard coming out the bottom of the shiny screen. And my mind wasn't "on" the device. For most people, their mind is stored in their brain, more or less. Mine was just 7 miles away, on a cluster of cloud servers.

On my phone, a story open on reddit. Stocks down a bit. Emissions.

The people at Brainle, LLC's AI lab made this PDA-laptop thingy, and I got in early on the beta. They also own the servers with my mind on them. I type commands and touch the screen of my PDA, those go to the servers, my mind changes. "My brain" is just a chip implanted in my skull, mirroring what's on those servers. And "I" am sitting, as usual, with myself at my fingertips and zero clue what to do.

commit 564309daf5b1d832367a8cc330a5ba93 (HEAD -> master)
Author: nkross
Date: Thu Oct 27 10:58:37 2022 -0400
test

It's near the end of class, and I still haven't pushed a real change to my source code. 90 minutes have passed since I got it at the nurse's office. I notice I am frustrated myself for wasting 90 minutes, wasting any minutes, wasting a second, wasting there goes another second staring at the PDA. And no ideas for how to change my mind.

That sounds really dumb, wouldn't I jump at the chance to modify my brain's code? But see, I have ADHD. Not the fun kind, where you take cocaine-and-skydiving lessons and grow up to be a billionaire or a crime lord. No, I have the "inattentive" type, where you sit in a pile of laundry and scroll through reddit for 9 hours a day.

But I can think the thoughts of a cocaine-and-skydiving guy, and that's what pisses me off. I can have a cool idea, make a plan, remember some helpful info I read, and then… I waste the day. The mind is willing, but the brain is weak, like all fleshy things.

Now people are getting up and putting shit in their backpacks. I should too. I should also get my life in order, but one thing at a time. Typical Nick, gets every chance and forgets to grab it. For all I sit around th...
Accountability Buddies: Why you might want one (+ Database to find one!)
by Samuel Nellessen — 2022-10-23T16:25:12.568Z

TL;DR: An accountability buddy is someone to check in with from time to time to give you social motivation to achieve your goals. There are many additional benefits from this process such as planning together and getting feedback on your progress. I think especially EAs in remote areas or those doing EA-related work, or upskilling part-time would benefit from having an accountability buddy. If you’d like to try it out, put your details down

in this table

.

This is partly a post about increasing your productivity. For more ideas check

Effective Self-Help’s long list of recommendations

.

Thank you to Evander and Anabel for your feedback.

Epistemic status: We have had first-hand experience with accountability buddies for the past six months + reflected on the process several times. We’ve also had conversations with others about the topic. Overall our views should be taken as a motivation to experiment instead of a laid-out path.

Author’s note: The first-person perspective in this post is taken in by me. Other remarks by Sam are made here. Nevertheless, we wrote most of this article collaboratively. Furthermore, this article is a concrete outcome of our rejection challenge.

Motivation for having an accountability buddy

I think I wouldn’t be where I am today if I hadn’t met Sam, my accountability buddy at EAG London this year. Our regular meetings made me more structured, helped me frequently reflect on my goals and progress, and made me more ambitious than I was before. I think many, if not all people would benefit from some form of accountability partnership and I encourage you to give it a try if you haven’t.

In an abstract sense, an accountability buddy (AB) is someone to help you better reflect and achieve your goals, either through indirect accountability (”I told them I’d get this done this week and it’s already Thursday, so I better get going!”) or direct accountability (“Hey, didn’t you say you wanted to start that project? How is that going?”). The most common way of doing this is by meeting regularly and going through past progress and future goals.

In addition to accountability, an AB can help you to reflect on your goals and the progress you made toward them. If it works out well, an accountability session can feel like a mini-coaching every week.

I think many people would benefit from accountability buddies, especially if they:

Live in a location wher

...
How to get past Haidt's elephant and listen
by Astynax — 2022-10-23T16:06:20.902Z

First, some examples.

In my beginning programming classes, I tell students (and show them): write comments first, before coding. Some won't. I tell them: write the algorithm first. I prove this will save them time. There's a whole chapter on why and how to do this. Many won't. (Except twice, they really got it. IDK why.) As time goes by -- I hope! -- they learn that this is the way to go. How can I help them hear at the start?
I was with my 6-ish nephew, and he wanted to show me a computer game. Computer wouldn't start. He called in his mom for help. She said, Sure, I'll try, but I think you should know your uncle knows everything I do about computers and more. I could see on his face: he was waiting for her to stop talking nonsense and fix the damn thing. Why couldn't he hear her?
I was at an informal lecture; the presenter used the story of Cain and Abel to comment on a supposed conflict between ranchers and farmers in ancient times. I went to speak to him. He said, "I'm sorry if you were offended." I said, "No, it was great! I really liked [blah blah]." He said, "OK. I'm sorry if you were offended." I said, "I wasn't offended at all. [More details]." He said, "OK. I have to go. I'm sorry you were offended." Why couldn't he hear?

And then there's the kid with the terrible twos. "It's time to eat. We have pizza, your favorite!" Response: "NO!" (Didn't he hear you say "pizza"?)
Haidt (The Righteous Mind) and probably others think it's the elephant, the part of the brain that's emotion and intuition and gut, that's charging on, heedless of the babblings of the rider, the intellectual part we think it in charge but actually functions to justify what the elephant already decided to do. OK. But it sure would be helpful if we could get to that elephant before it's made up its stubborn mind. Haidt suggests: convince the elephant, not the rider (however that would work). I'll add: if it's me, I can decide not to care so much, so the elephant will be more cooperative.

In the case of the beginning programmers, I already use self-interest (think of the time you'll save!) and availability bias (every program they see starts with commenting). It worked better this semester. All I can think I did different was show them the grading scheme for a program, that 60% of it was commenting and algorithm. :) Maybe that was it; IDK.

This being LessWrong, of course we s...
Writing Russian and Ukrainian words in Latin script
by Viliam — 2022-10-23T15:25:41.855Z

I wanted to figure out how to properly write Russian and Ukrainian words using the Latin alphabet, but it turns out there are dozen different standards for Russian and half a dozen different standards for Ukrainian.

My original plan was to post two short algorithms in pseudocode, along with some commentary. I guess it will be only the commentary instead. I hope some of you may find some information here interesting anyway.

*

Disclaimer: I am not a native speaker of either of these languages. This article may contain mistakes, and I would be thankful if you point them out in comments. I have learned Russian at school, decades ago; only a rusty passive knowledge has remained. My knowledge of Ukrainian is limited to the first two lessons at Duolingo. (My native language is Slovak, which is also a Slavic language, but written in Latin script.)

*

The basic strategic choice for conversion between different scripts is the following: would you rather preserve the minutiae of the written version (and sacrifice some information about pronunciation, if necessary), or would you rather preserve the pronunciation (and mostly throw away the original written form)?

As a specific example, consider the Russian word for milk: "молоко". In written form, all its three vowels are the same. When pronounced, the stress is on the last syllable, and a stressed "о" is generally pronounced differently from an unstressed "о". The entire word sounds kinda like "mah luck core"; the first two vowel sounds are the same, and the third one is different.

A system trying to preserve the written form might write this word as "moloko", while a system trying to preserve pronunciation might write it as "malako" instead.

(This is not unique for Cyrillic, by the way. You face a similar dilemma e.g. when trying to write Japanese syllables "た ち つ て と". Would you rather preserve the sound, and write them as "ta, chi, tsu, te, to", or follow the internal logic of hiragana which insists that this is the same consonant, and write them as "ta, ti, tu, te, to"?)

Both options have their advantages and disadvantages. It also depends on the audience. People who do not speak the language and do not plan to learn it, have no reason to care about its orthography; they just want to pronounce the weird scribbles. On the other hand, people who are already used to reading that language, may feel very uncomfortable seeing it wr...
Have you noticed any ways that rationalists differ? [Brainstorming session]
by tailcalled — 2022-10-23T11:32:13.368Z

I have a severe psychometrics addiction, so I've regularly been tempted to create some sort of "rationalist type test" or similar, which rationalists could take to classify themselves compared to other rationalists. I particularly like to create such tests using data-driven methods, such as factor analysis. But factor analysis requires a bunch of primitive variables to summarize. And I thought, the most logical way to get such variables is to ask people what they've already come up with.

So, do you have any thoughts on ways that rationalists differ from each other, or ways that rationalists differ from nonrationalists, or perhaps even ways that nonrationalists differ from each other in ways that might be interesting to put into a test for rationalists?

Also, if you don't have a LessWrong account, or want to share this question with rationalists who don't hang out on LessWrong, I've created a Google Form that you can fill out instead. If you've got something spicier that you would prefer doesn't become public, then I have also created a private version of the form here.

I don't know how useful my analysis could become, but at least in theory it seems like it could become more generally useful. For instance, it might be useful for providing targeted advice to certain subgroups of rationalists.

Edited to add: A lot of the comments seem to be bringing up abstract, general traits that have been studied in differential psychology. Certainly those could also be interesting to investigate rationalist on, but I have numerous philosophical objections to many of the traits, and I would instead like to design a new test with traits specific to the rationalist community, in terms of more concrete things that rationalists do and experience.