trentmkelly/LessWrong-43k huggingface dataset:
Book Review: On Intelligence by Jeff Hawkins (and Sandra Blakeslee)
by Rafael Harth — 2020-12-29T19:48:04.435Z

On Intelligence is a book I've read as part of my quest to understand neuroscience. It attempts to develop a unified theory of the neocortex meant to serve as a blueprint for Artificial Intelligence. I think of the book as being structured into three parts.

Part one: Artificial Intelligence and Neural Networks OR skip ahead to part two if you want to read about the cool neuroscience rather than about me lamenting the author's lack of epistemic rigor

This part is primarily about a single claim: building AI requires understanding the human brain. Depending on how you count, Jeff says this nine times in just the prologue and first chapter. To justify it, he tells us the story of how he came into contact with the field of artificial intelligence. Then and now, he laments that people in the field talk about intelligence without trying to understand the brain, whereas neuroscientists talk about the brain without trying to develop a high-level theory of intelligence. Neural networks are a small step in the right direction, but he quickly got disillusioned with them as they don't go nearly far enough; their connection to the brain is quite loose and high-level. The conclusion is apparent: someone has to bring neuroscience into AI, and only then will the field succeed. And since no-one else is doing it, Jeff steps up; that's what the book is for.

The picture he lays out makes a lot of sense if you take the claim as a given. The flaw is that he neglects to argue why it is true.

I think it's pretty hard to make excuses here. This isn't a dinner conversation; it's a 250-page book that explicitly sets out to reform an entire field. It's a context where we should expect the highest level of epistemic rigor that the author is capable of, especially given how much emphasis he puts on this point. However, after rereading this part of the book, the only evidence I can find that supports AI requiring an understanding of the brain is the following:

The observation that current AI architectures are not like the brain, which I think is uncontroversial but doesn't prove anything

A claim that you can't do AI without understanding intelligence, and you can't understand intelligence without understanding the brain, which isn't an argument unless you already believe that intelligence is inherently tied to the neocortex.

A claim that current AI approaches have failed. This one may have

...
Dario Amodei leaves OpenAI
by Daniel Kokotajlo — 2020-12-29T19:31:04.161Z

This is a linkpost for https://openai.com/blog/organizational-update/

“We are incredibly thankful to Dario for his contributions over the past four and a half years. We wish him and his co-founders all the best in their new project, and we look forward to a collaborative relationship with them for years to come,” said OpenAI chief executive Sam Altman.

Anyone know what the new project is?
Against GDP as a metric for timelines and takeoff speeds
by Daniel Kokotajlo — 2020-12-29T17:42:24.788Z

Or: Why AI Takeover Might Happen Before GDP Accelerates, and Other Thoughts On What Matters for Timelines and Takeoff Speeds

[Epistemic status: Strong opinion, lightly held]

I think world GDP (and economic growth more generally) is overrated as a metric for AI timelines and takeoff speeds.

Here are some uses of GDP that I disagree with, or at least think should be accompanied by cautionary notes:

Timelines:

Ajeya Cotra thinks of transformative AI

as “software which causes a tenfold acceleration in the rate of growth of the world economy (assuming that it is used everywhere that it would be economically profitable to use it).” I don’t mean to single her out in particular; this seems like the standard definition now. And I think it's much better than one prominent alternative, which is to date your AI timelines to the first time world GDP (GWP) doubles in a year!
Takeoff Speeds: Paul Christiano

argues for Slow Takeoff

. He thinks we can use GDP growth rates as a proxy for takeoff speeds. In particular, he thinks Slow Takeoff ~= GWP doubles in 4 years before the start of the first 1-year GWP doubling. This proxy/definition has received a lot of uptake.
Timelines:

David Roodman’s excellent model

projects GWP hitting infinity in median 2047, which

I calculate

means TAI in median 2037. To be clear, he would probably agree that we shouldn’t use these projections to forecast TAI, but I wish to add additional reasons for caution.
Timelines: I’ve sometimes heard things like this: “GWP growth is stagnating over the past century or so; hyperbolic progress has ended; therefore TAI is very unlikely.”
Takeoff Speeds: Various people have said things like this to me: “If you think there’s a 50% chance of TAI by 2032, then surely you must think there’s close to a 50% chance of GWP growing by 8% per year by 2025, since TAI is going to make growth rates go much higher than that, and progress is typically continuous.”
Both: Relatedly, I sometimes hear that TAI can’t be less than 5 years away, because we would have seen massive economic applications of AI by now—AI should be growing GWP at least a little already, if it is to grow it by a lot in a few years.

First, I’ll argue that GWP is only tenuously and noisily connected to what we care about when forecasting AI timelines. Specifically, the point of no return is what we care about, and there’s a good chance it’ll come years before GWP sta...
Why Neural Networks Generalise, and Why They Are (Kind of) Bayesian
by Joar Skalse — 2020-12-29T13:33:53.202Z

Currently, we do not have a good theoretical understanding of how or why neural networks actually work. For example, we know that large neural networks are sufficiently expressive to compute almost any kind of function. Moreover, most functions that fit a given set of training data will not generalise well to new data. And yet, if we train a neural network we will usually obtain a function that gives good generalisation. What is the mechanism behind this phenomenon?

There has been some recent research which (I believe) sheds some light on this issue. I would like to call attention to this blog post:

Neural Networks Are Fundamentally Bayesian

This post provides a summary of the research in these three papers, which provide a candidate for a theory of generalisation:

https://arxiv.org/abs/2006.15191
https://arxiv.org/abs/1909.11522
https://arxiv.org/abs/1805.08522
(You may notice that I had some involvement with this research, but the main credit should go to Chris Mingard and Guillermo Valle-Perez!)
I believe that research of this type is very relevant for AI alignment. It seems quite plausible that neural networks, or something similar to them, will be used as a component of AGI. If that is the case, then we want to be able to reliably predict and reason about how neural networks behave in new situations, and how they interact with other systems, and it is hard to imagine how that would be possible without a deep understanding of the dynamics at play when neural networks learn from data. Understanding their inductive bias seems particularly important, since this is the key to understanding everything from why they work in the first place, to phenomena such as adversarial examples, to the risk of mesa-optimisation. I hence believe that it makes sense for alignment researchers to keep an eye on what is happening in this space.
If you want some more stuff to read in this genre, I can also recommend these two posts:
Recent Progress in the Theory of Neural Networks
Understanding "Deep Double Descent"
EDIT: Here is a second post, which talks more about the "prior" of neural networks:
Deep Neural Networks are biased, at initialisation, towards simple functions
The map and territory of NFT art
by frcassarino — 2020-12-29T11:45:34.960Z
Link: https://federicorcassarino.substack.com/p/the-map-and-territory-of-nft-art

I’ve recently become aware of the world of non-fungible tokens.

Wikipedia puts it as:

A non-fungible token (NFT) is a special type of cryptographic token which represents something unique; non-fungible tokens are thus not mutually interchangeable.

Non-fungible tokens are used to create verifiable digital scarcity, as well as digital ownership, and the possibility of asset interoperability across multiple platforms.

The application of NFTs that I found the most thought-provoking is in digital art. There’s websites, such as

opensea.io, that have listings of NFT-based digital paintings.

In that website, you can buy any of the listed NFT-based paintings, and become the proud owner of the “original” version of a digital painting. People are paying outrageous sums of money for this.

An NFT-portrait of Ethereum co-founder Vitalik Buterin dressed like a medieval harlequin recently sold for $141,536.20.

The NFT-art market is absolutely booming.

What’s the punchline? That the “original” version of a digital painting is pixel-by-pixel identical with any of its copies. On opensea.io you can go ahead and download a perfect .jpeg copy of any of the listed paintings for free. You can download the portrait of Vitalik Buterin right

here

, and set it as your wallpaper. It’ll be the exact same portrait that the buyer paid 141k for. There is absolutely zero material difference between the original and the copy, except for the fact that, in some technical fuzzy way, one is the original and one isn’t.
Most people are either perplexed when hearing this, or react with scorn. After all, it’s intuitive to think that digital paintings being infinitely and perfectly replicable defeats the point of paying for an “original” version.

My reaction is that NFT-paintings are a wonderful reduction of the idea of “originality”, and it’s a great exercise to analyze the phenomenon from the lens of map and territory.
Let’s pick a more traditional example: what makes DaVinci’s Mona Lisa worth more than a replica of the Mona Lisa?

One can point out a bunch of different factors at play here. DaVinci’s Mona Lisa is more valuable because replicas were made by lesser artists and result in inferior paintings that don’t captivate the viewers as much. DaVinci’s Mona Lisa is more valuable because it’s the only one that has the exact strokes of paint that DaVinci plastered on the canvas, while the replicas looks subtly...
Measuring up to incredible potential
by KatjaGrace — 2020-12-29T07:50:17.583Z

Yesterday I wrote that people often talk as if events are basically determined by people’s values and capabilities, ignoring the difficulty of figuring out which opportunities to take, or even noticing opportunities.

I think one reason to have a better model is that this one doesn’t account for a substantial category of felt difficulty in being a human, possibly encouraging a general sense that one is ubiquitously failing, what with not seeming to be demonstrably grabbing the best of a vast multitude of possible options at each moment.

My own experience for instance involves often not remembering even the options that I noticed before, and these not seeming that multitudinous, except relative to my pitiful capacity to hold things in my mind. Also, regularly noticing new aspects of the world suggestive of many options whose existence hadn’t even dawned on me previously, even though they would be pretty obvious to a creature for whom logic was free. And mostly being extremely uncertain about what actions will lead to what outcomes. If I habitually model myself as roughly homo economicus, deftly directing my resources toward my goals, at worst according to my pristine , it seems like I am in danger of continually find my real self incredibly disappointing. Plus, abstracting away parts of the situation that might be rich with potential for improvement.
Give it a google
by Adam Zerner — 2020-12-29T05:30:39.133Z

I'm a programmer. In programming, people talk about how the ability to Google is such an important skill. No one knows everything. In practice, people are always looking things up and running into weird situations that they have to figure out.

That's not what this post is about. Not quite. This post is about the decision to give it a google in the first place.

Poker

Here's an example. I play poker. It's hard to get good at poker, but not sucking at poker isn't very hard. Don't play junky hands preflop. Don't open-limp. Learn some basic poker math such as pot odds. If you can follow basic guidelines like that, you won't suck.

Yet if you sit down at a $1-2 game in Vegas, where hundreds of dollars are exchanging hands, people who have been playing poker longer than I have been alive often don't understand these basic ideas. Ideas that you would learn if you spent ten minutes googling "basics of poker strategy". You'd come across articles like How Not to Suck at Poker and you wouldn't be making such mistakes at the table.

As an even more extreme example, I've played in home games where despite playing poker for many years, people don't know even more basic things such as sizing your bets according to the size of the pot. Betting $20 into a $200 pot is a very small bet even if $20 feels like a lot of money. Your opponent only needs to put in $20 to have the chance at a $220 pot. Great risk-reward. In general, bet sizing is an incredibly complex topic, but all you have to know is to size your bet between 1/2 and the full size of the pot. The 80/20 principle really, really applies here.

Health

Recently my girlfriend and I have been waking up with scratchy throats. Intuitively we thought it might be because it's getting colder out. Especially as we sleep. So we tried sleeping with the heat on higher.

Turns out that was the exact wrong thing to do. I gave it a google and the issue seems to be that using the heat dries out the air, and dry air causes the sore throat symptoms we were experiencing.

Shopping

I bought a humidifier last night. I could have just surfed around on Amazon a bit and picked something out. Instead, I gave it a google first.

It worked out well, I learned some really interesting things. A humidifier is something that seems simple where it doesn't really matter which one you choose. Turns out that intuition was also wrong.

It's something that you'll pro

...
Vaccination with the EMH
by DirectedEvolution — 2020-12-29T03:05:22.466Z

Part 1. Is "1,000 True Fans" reasonable?

To be a successful creator you don’t need millions. You don’t need millions of dollars or millions of customers, millions of clients or millions of fans. To make a living as a craftsperson, photographer, musician, designer, author, animator, app maker, entrepreneur, or inventor you need only thousands of true fans.

The "1,000 True Fans" concept is that a content creator can make a living of $100,000 per year if they make a profit of $100 per true fan, and have 1,000 true fans.

I notice that the concept makes my brain think "that seems do-able!" And that makes me think it sounds like a get rich quick scheme. So I want to spend some time thinking about it carefully.

We should appreciate creative thinkers. Kevin Kelly, who appears to have written the original post on the idea, is giving us a free, possibly very helpful idea. This is going to be a work of criticism, but I hope that it will be constructive. Kelly seems to be a pretty successful writer, and his idea has plenty of high-profile proponents. So I don't think I need to treat it with kid gloves.

Why does my brain think "sounds easy!" when it reads this idea? First of all, 1,000 is a much smaller number than millions and millions. I can imagine my family members and a few friends being true fans. That's, like, 10 true fans already and I haven't even started! Plus, $100 doesn't seem like very much money to ask for. That's like $8.50 per month.

Let's get some perspective on these numbers. A Netflix subscription is $9/month. So you need to be providing your true fans with the kind of value that Netflix provides to the average customer.

Although 1,000 true fans seems like a do-able number, we have to ask about the relationship between "true fans," "regular fans," and "occasional consumers." Go to any concert, and true fans most likely are only a small fraction of the total audience. Likewise for books and blogs and Youtube channels. I think I'm pretty normal in reading the complete works of a very few authors, and one or two books by a much larger number.

At one point in his post, Kevin Kelly estimates that for every 1 true fan, you should expect to have 2-3 regular fans:

... for every single true fan, you might have two or three regular fans.

And later on, he estimates a 1:7 ratio:

Yet if even only one out of million people were interested, that’s potentially 7,000 people on th

...
Morality as "Coordination", vs "Do-Gooding"
by Raemon — 2020-12-29T02:37:04.195Z

Reworked version of a shortform comment. This is still not the optimal version of this post but it's the one I had time to re-publish.

I've spent the past few years trying to get a handle on what it means to be moral. In particular, to be moral in a robust way, that holds up in different circumstances.

A year-ish ago, while arguing about what standards scientists should hold themselves to, I casually noted that I wasn't sure whether, if I were a scientist, and if the field of science were rife with dishonesty, whether it would be better for me to focus on becoming more honest than the average scientist, or focus on Some Other Cause, such as avoiding eating meat.

A bunch of arguments ensued, and elucidating my current position on the entire discourse would take a lot of time. But, I do think there was something important I was missing when I first wondered about that. I think a lot of Effective Altruism types miss this, and it's important.

The folk morality I was raised with, generally would rank the following crimes in ascending order of badness:

Lying
Stealing
Killing
Torturing people to death (I'm not sure if torture-without-death is generally considered better/worse/about-the-same-as killing)

But this is the conflation of a few different things. One axis I was ignoring was "morality as coordination tool" vs "morality as 'doing the right thing because I think it's right'." And these are actually quite different. And, importantly, you don't get to spend many resources on morality-as-doing-the-right-thing unless you have a solid foundation of the morality-as-coordination-tool. (This seems true whether "doing the right thing" looks like helping the needy, or "doing God's work", or whatever)

There's actually a 4x3 matrix you can plot lying/stealing/killing/torture-killing into which are:

Harming the ingroup
Harming the outgroup (who you may benefit from trading with)
Harming powerless people who can't trade or collaborate with you

And I think you need to tackle these mostly in this order. If you live in a world where even people in your tribe backstab each other all the time, you won't have spare resources to spend on the outgroup or the powerless until your tribe has gotten it's basic shit together and figured out that lying/stealing/killing each other sucks.

If your tribe has it's basic shit together, then maybe you have the slack to ask the question: "hey, that outg...
Perfectionism as depth-first search
by MSRayne — 2020-12-29T01:49:22.945Z

Perfectionism could be generally defined as an approach to solving problems and completing tasks fixated primarily on minimizing perceived imperfection.

Depth-first search, meanwhile, is an algorithm for traversing tree structures, which starts at the root, then goes down a single branch until it hits a leaf. Upon reaching a leaf, it backtracks to the last node whose descendants aren't fully searched, and goes down a new branch. This process repeats until the whole tree is explored.

Recently, I realized, in a flash of insight while struggling to stop tweaking tiny flaws long enough to complete a project, that there is a possibly fruitful analogy between these two things.

Every task can be thought of as a tree of sub-tasks. In fact, while writing this very essay, I broke the main idea down into a tree of questions and answers, to help me think about it (essentially an outline, but with each statement having only questions as sub-elements, and each question only having answer statements beneath it). Similarly, when writing a program, one usually splits it into subroutines, themselves split into sub-subroutines.

Perfectionists tend to focus more on sub-tasks lower on the tree. For instance, when writing, I tend to fixate on word choice and comparatively neglect overall structure, and some people trying to tidy up might heavily clean each object, one by one, and burn out before touching most of what's in the room, instead of the standard approach - getting the whole room mostly clean prior to obsessing over specifics. The reason an aversion to imperfection motivates such behavior might be that smaller flaws are easier to perceive, and easier to fix - thus drawing more attention to themselves.

This interestingly parallels the way that depth-first search preferentially moves down the tree while searching - where searching the tree in this analogy represents working on tasks or sub-tasks, with the current node being visited representing the sub-task currently being worked on. A perfectionist will often be tempted to drill right down to a single detail, get it "right", then go to a related detail, and so on, wandering somewhat aimlessly through the leaves of the tree while barely touching the branches.

The reason this matters is that in most task trees, most value comes from working on nodes closer to the root - as higher-level sub-tasks usually lay necessary foundations ...
Chain Breaking
by Mark Xu — 2020-12-29T01:06:04.122Z
Link: https://markxu.com/chain-breaking

This is a rationality technique I’ve been experimenting with. Thank you to Jack Ryan, Thomas Kwa, Sydney Von Arx, Noa Nabeshima, and Kyle Scott for helping me refine the method.

Algorithm

Pick something that has happened before that you would prefer not to happen again.

Examples include: not exercising, not flossing, not sleeping on time, eating too many chocolate almonds, spending too much time on Facebook, being late to meetings, forgetting to schedule things, etc.

In theory, you could pick something that hasn’t happened yet that you want to prevent. This is a more advanced version of the technique. Try it at your own peril.

Why do you think it might happen again? Determine the specific chain of events that will lead to this undesirable outcome.

A chain is only as strong as its weakest link. Locate the link in the chain that seems most fragile.

Intervene to break that link (and thus the entire chain). If it isn’t easy to think of an intervention, that might mean you haven’t gotten specific enough. Recurse.

Examples

Many of these examples do not involve me. I wrote them all in the first person for easier comprehensibility.

Example: messenger

I use messenger in a way that disrupts my work routine.

When doing pomodoros, when I check messenger during breaks. There are often many messages to respond to, some of which require thought. Even worse, sometimes people respond to my responses in almost real-time. Since I do not like not responding to messages, I respond to all messages, nearly always taking more than the five minutes allotted for my break.

More compactly: break -> messenger -> many unread messages -> respond

Once I look at my messages, the compulsion is very strong to respond to all of them. In particular, if I read them, but don’t respond, I might forget to respond. I could make this easier by keeping track of which messages to respond to, but that seems too tedious. Instead, I think violating break -> messenger is easier.

Intervention: don’t check messenger during breaks.

Example: distracted while exercising

I often don’t finish my exercise routine in the time allotted.

When exercising, I have time in-between sets to rest. I often perform miscellaneous activities during these times, such as read articles on my phone. This causes me to lose track of time, so the next set is delayed. This both causes my exercise routine to take

...