trentmkelly/LessWrong-43k huggingface dataset:
Debugging Writer's Block
by matto — 2021-10-09T19:04:31.074Z

The hardest part of writing for me is dealing with writer's block. It appears almost every time I sit down to write–I'll have everything ready: the time, the space, and the super exciting idea I want to share. But right then, the words won't come! And when they eventually do, they come out in awkward strings that beg to be deleted, but I keep at it.

Soon, though, the excitement surrenders to frustration accompanied by a steely determination to get to the finish line. Predictably, this produces writing that feels dry and forced, leaving me disappointed and exhausted. Mistaking the symptoms for causes, I tried style guides: "Style: Lessons in Clarity and Grace" by J. William, "The Art of Plain Talk" by R. Flesch, and, of course, Strunk & White's "The Elements of Style."

While my writing became somewhat better, the process remained frustrating and slow, yielding an essay once every 6 to 8 weeks. But I kept going at it, hoping that sooner or later I'll discover some advice or feedback that will remedy my problem. And then, one day, it happened–I stumbled on an interview with Jerry Seinfeld and because I love his comedy, I gave it a listen. Unexpectedly, at one point he shared this advice about writing:

Set a hard time limit on your writing sessions.

Reward yourself after each session.

When you finish a piece, don't show it to anyone for at least a day.

It seemed simple, and having little to lose, I decided to give it a try. Long story short, it worked pretty well–I wrote about 1.5x to 2x faster. What's more, the process became fun, which made the output less dry and forced. It was great! But after enjoying this state for some time, I got to thinking: how does this advice work? And can it be applied to other problems?

Admittedly, my understanding of how the mind works is pretty shallow. It's mostly informed by sources like The Elephant in the Brain, Kaj Sotala's posts about IFS, and van der Kolk's book about Trauma. There's also quite a bit of bias that comes from writing software, eg. "everything is a system!" But anyway, here's my working theory about why this advice worked for me:

Writing is hard (see footnote 1)! It takes a lot of time and effort to get better. And even if you do produce outstanding work, there's the uncertainty that, at the end, nobody will read any of it.

System 1 takes notice of this and decides it doesn't like the upfront investment a

...
Book Review: Churchill and Orwell
by Sammy Martin — 2021-10-09T19:01:17.615Z

You may have heard of George Orwell as the man who presaged some of the sequences’ insights on rationality, especially as applied to the English language. You might have heard of Winston Churchill as a futurist who had surprisingly longtermist inclinations for someone of his time and background. Did you know they were also both pivotal figures in saving human civilisation from the lock-in of some highly undesirable values?

‘Churchill and Orwell: The fight for freedom’ is a morality tale about the virtue of self-correction. It’s a story of two anti-heroes who recognised an unprecedented danger that threatened all of humanity when no-one else would, and an exploration of why they were capable of this. It’s also fun, engagingly written, and something I originally read for pure entertainment before realizing it explores important lessons for anyone trying to be more rational.

What makes this hagiography different from most others is that the author, Thomas E Ricks, emphasises the unrelenting desire to face reality that kept these two men on the right side of history, not conventional personal virtue nor any uniquely special or deeply inspiring moral commitments. It was the ability to self-correct, to not drift off into fanciful narratives, that let an avowed socialist and a staunch imperialist become the chief opponents of the worst forms of socialism and imperialism in the twentieth century. That’s the core claim of the book - their respect for truth and ability to change their minds when it was called for led them both to recognise the totalitarian threat, and also led them to adopt the right principles. This review won’t dwell on the details of the two men’s lives, but rather tease out this particular claim.

The book is structured like an interleaved biography, and covers all the main beats in the lives of the two men. It opens with two anecdotes about how Churchill and Orwell each came close to death sometime in the 1930s - Orwell was shot in the neck in Spain whilst fighting the Francoists, while Churchill was hit by a car and nearly killed.

The point of these stories is to emphasise the ‘great man’ view of history and the idea of contingency. First, understand how easily your chosen figures could have been removed from the picture, then go on to understand how important they were.

Ricks is completely open about his belief in the great man theory, and also...
Intelligence or Evolution?
by Ramana Kumar — 2021-10-09T17:14:40.951Z

Here are two kinds of super high-level explanation for how things turned out in the future:

Someone wanted things to turn out that way.
Selective forces favoured that outcome.

Which one is a better explanation?

This question is one I’m highly uncertain about, and often shows up as a crux when thinking about where to focus one’s efforts when aiming for a good future.

One way of labelling the alternatives is intelligence vs evolution. There’s a cluster of stuff on each side. I’ll try to point to them better by listing points in the clusters:

Intelligence: an agent, someone capably trying to do something, intentional deliberate action, the will of the most powerful entity, consequentialists, unipolar/singleton or few-agent scenarios
Evolution: Moloch (multipolar traps), natural selection, competitive pressures, the incentive landscape, game theoretic solution concepts, highly multipolar or multi-agent scenarios

Under some view, both intelligence and evolution are good explanations for how things turned out. They just amount to taking a different perspective or looking at the situation at a different scale. I agree with this but want to avoid this view here so we can focus on the distinction. So let’s try to make the alternatives mutually exclusive:

Intelligence: someone was trying to produce the outcome
Evolution: no one was trying to produce the outcome

This question is very similar to unipolar vs multipolar, but maybe not the same. My focus is on whether the main determinant of the outcome is “capable trying” vs anything else. This can be for a few agents, it doesn’t require exactly one.

If you think our future will be better explained by intelligence, you might prefer to work on understanding intelligence and related things like:

Decision theory, anthropics, probability, epistemology, goal-directedness, embedded agency, intent alignment – understanding what agents are, how they work, how to build them
Moral philosophy and value learning
{single, multi}/single alignment, and approaches to building AGI that focus on a single agent

If you think our future will be better explained by evolution, you might prefer to work on understanding evolution and related things like:

Economics and sociology, especially areas like social choice theory, game theory, and topics like bargaining and cooperation
Biological and memetic evolution, and the underlying theory, perhaps something lik

...
Apprenticeship Online
by Henrik Karlsson — 2021-10-09T13:50:53.893Z
Link: https://escapingflatland.substack.com/p/apprenticeship-online

If humans stopped imitating, civilization would vanish.

It is 6:37 in the morning as I’m sitting down to write this, late in August, and I and the three-year-old are down in the old stable we use as study. As I look over my notes, she observes me. Mist is leaking through a crack in the window. Then she gets up, finds a pocket calculator, rummages around with the notes on the table a bit, and starts typing.

“The unicorn ate the human beings”, she says aloud. “Like wheat.”

Human beings cannot help picking up skills if they live immersed in communities where the skills are practiced.

If that statement is correct, or at least a good first approximation of the truth, as I believe it to be, the aim of education is to make sure as many people as possible can immerse themselves in skillful communities. We should, as far as possible, extract learners out of schools – where the incompetent live surrounded by, and therefore imitate, others at the same level of incompetence – and into productive environments.

Historically, this approach to learning has been the norm. Our grandparents lived in direct contact with the environments they would have to navigate as adults, and they could observe and imitate those who had already mastered it. But as modernity, working like a giant centrifuge, started separating everything that in traditional societies co-mingled, it became impossible for children to learn through pure imitation. They were segregated by age groups and denied access to the offices, factories, workshops, stores, whose skills they would have to master to become successful adults.

But maybe we can return to our premodern condition? Maybe we can build systems that allow novices to reintegrate themselves into communities of practice - but at scale?

It is a question of architecture, and can be phrased as such: How do we design workspaces where novices can enter at scale, and where they can easily observe the knowledge that guides experienced practitioners? And how can we do that without obstructing the work being done?

Historically, the rooms we’ve been able to build have been too limited for this. They put a low upper bound on how many novices could be absorbed by productive communities. Peter Paul Rubens, while developing his baroque masterpieces in the early 1600s, could only allow a handful of apprentices in his atelier. How would it have looked with h...
How to think about and deal with OpenAI
by Rafael Harth — 2021-10-09T13:10:56.091Z

Eliezer Yudkowsky writes on twitter:

Nothing else Elon Musk has done can possibly make up for how hard the "OpenAI" launch trashed humanity's chances of survival; previously there was a nascent spirit of cooperation, which Elon completely blew up to try to make it all be about who, which monkey, got the poison banana, and by spreading and advocating the frame that everybody needed their own "demon" (Musk's old term) in their house, and anybody who talked about reducing proliferation of demons must be a bad anti-openness person who wanted to keep all the demons for themselves.

Nobody involved with OpenAI's launch can reasonably have been said to have done anything else of relative importance in their lives. The net impact of their lives is their contribution to the huge negative impact of OpenAI's launch, plus a rounding error.

Previously all the AGI people were at the same conference talking about how humanity was going to handle this together. Elon Musk didn't like Demis Hassabis, so he blew that up. That's the impact of his life. The end.

I've found myself repeatedly uncertain about what to make of OpenAI and their impact. The most recent LessWrong discussion that I'm aware of has happened on the post Will OpenAI-s work unintentionally increase existential risk, but the arguments for negative impact are different from the thing Eliezer named.

I'm also not entirely sure whether publicly debating sensitive questions like whether a person or organization accidentally increased existential risk is a good idea in the first place. However, spontaneously bringing up the issue on a twitter thread is unlikely to be optimal. At the very least, it should be beneficial to discuss the meta question, i.e., how we should or shouldn't talk about this. With that in mind, here are three things I would like to understand better:

Concretely speaking, should we be hesitant to talk about this? If so, what kind of discussions are okay?

And -- conditional on discussing them being a good idea:

What is the more detailed story of how the "nascent spirit of cooperation" has degraded or changed since the inception of OpenAI?

What interventions are possible here, if any? (Is it really that difficult to organize some kind of outreach to Elon to try and reverse some of the effects? Naively speaking, my impression has been that our community is sufficiently well connected t

...
Do you like excessive sugar?
by momom2 — 2021-10-09T10:40:29.942Z

In "Adaptation-Executers, not Fitness-Maximisers", Eliezer Yudkowsky writes:

"Fifty thousand years ago, the taste buds of Homo sapiens directed their bearers to the scarcest, most critical food resources—sugar and fat. Calories, in a word. Today, the context of a taste bud's function has changed, but the taste buds themselves have not. Calories, far from being scarce (in First World countries), are actively harmful. Micronutrients that were reliably abundant in leaves and nuts are absent from bread, but our taste buds don't complain. A scoop of ice cream is a superstimulus, containing more sugar, fat, and salt than anything in the ancestral environment."

This contradicts with my personal experience, since last week I followed a diet consisting almost exclusively of pasta and sugar, and then came to long for vegetables while sugar became distasteful. Perhaps I am committing the typical mind fallacy, but I should at least try to verify Yudkowsky's claim.

So, what about you? This site mentions that women should take no more than 24 grams a day of sugar, and men no more than 36 grams. The average American takes 88.
Can you please:

-state how much sugar you usually take daily
-state how you usually feel about eating sugar
-change your habits today
-report on how that influenced your feelings?

(I don't have the means to organize a real study, am not aware of any done on the subject, and I hope anecdotal evidence might still be overwhelming enough to give a result.)
Bayeswatch 11: Parabellum
by lsusr — 2021-10-09T07:08:06.272Z

Vi wore a suit. She held an empty briefcase in her hand.

"It's not often Bayeswatch agents visit us mere humans on Hive Hill," said Trinity.

"I'm here in no formal capacity. I'm on mandatory medical leave. I collect a regular salary for the rest of my life. In exchange, all I have to do is never speak about my nominal employer," said Vi.

Trinity let her in.

"You must know some dangerous secrets," said Trinity.

Vi rotated the briefcase so Trinity could see the orthonormal cube etched onto it. Trinity eyed the Mark of Pandora.

"What are you doing here?" said Trinity.

"Bayeswatch suppresses AI through a monopoly of the use of force. That works as long as little people couldn't coordinate. Technically-speaking, Bayeswatch still exists. But the announcement a few months ago that it was disbanded―well, let's just say that Bayeswatch's threat of overwhelming force is looking much less credible," said Vi.

"What do you plan to do about it?" said Trinity.

"Rumors say you've been buying tanks," said Vi.

"An armed and well-regulated militia is our constitutionally protected right," said Trinity.

"I know. I want in," said Vi.

"Shall we sign you up as an officer, a scientist or an engineer? Name your terms," said Trinity.

"You heard me," said Vi.

There was a pause.

"I see. Revealing information about Bayeswatch in writing or in speech is illegal but they never specifically forbid mind melds," said Trinity.

Vi waited.

"You know, the most elite collective in the world doesn't accept just anybody. At the absolute minimum you need a working memory at least three point five standard deviations above average just to keep from cluttering up the collective's global workspace," said Trinity.

"I'm forwarding you the psychometric data included on my application to the Bayeswatch internship," said Vi.

Trinity received it via her cornea. "You could have done whatever you wanted. Why be a cop?"

"Someone has to save the world," said Vi.

"Is that even in the cards?" said Trinity.

"We do the best we can with the resources available," said Vi. She shrugged.

"And what resources are available?" said Trinity.

"Whatever you can beg, buy or borrow," said Vi, "Whatever you can seize, salvage or steal".

Vi finished examining the synchronizing mainframe.

"You took a long time scrutinizing that machine," said Trinity.

"According to the darkweb some hiveminds have been hacking these things to produce one-directional transfe...
Steelman arguments against the idea that AGI is inevitable and will arrive soon
by RomanS — 2021-10-09T06:22:25.851Z

I’m pretty sure that AGI is almost inevitable, and will arrive in the next decade or two.
But what if I’m wrong and the "overpopulation on Mars" folks are right?

Let’s try some steelmanning.

Technological progress is not inevitable

By default, there is no progress. There is a progress in a given field only when dedicated people with enough free time are trying to solve a problem in the field. And the more such people are in the field, the faster is the progress.

Thus, progress in AGI is limited by societal factors. And a change in the societal factors could greatly slow down or even halt the progress.

Most of the past progress towards AGI can be attributed to only two countries: the US and the UK. And both appear to be in a societal decline.

The decline seems to be caused by deep and hard-to-solve problems (e.g. elite overproduction amplified by Chinese memetic warfare). Thus, the decline is likely to continue.

The societal decline could reduce the number of dedicated-people-with-enough-free-time working towards AGI, thus greatly slowing down the progress in the field.

If progress in AI is a function of societal conditions, a small change in the function’s coefficients could cause a massive increase of the time until AGI. For example, halving the total AI funding could move the ETA from 2030 to 2060.

AGI is hard to solve

Thousands of dedicated-people-with-enough-free-time have been working on AI for decades. Yet there is still no AGI. This fact indicates that there is likely no easy path towards AGI. The easiest available path might be extremely hard / expensive.

The recent progress in AI is impressive. The top AIs demonstrate superhuman abilities on large arrays of diverse tasks. They also require so much compute, only large companies can afford it.

Like AIXI, the first AGI could require enormous computational resources to produce useful results. For example, for a few million bucks worth of compute, it could master all Sega games. But to master cancer research, it might need thousands of years running on everything we have.

The human brain is one known device that can run a (kinda) general intelligence. Although the device itself is rather cheap, it was extremely expensive to develop. It took billions of years running a genetic algo on a planet-size population. The biological evolution is rather inefficient, but it is the only method known to produce a (kinda) general i...
Welcome to ACX Lisbon [Edit With Your Details]
by daniel.sousa.me — 2021-10-09T05:55:17.209Z

(The following are our suggestions for what kind of information is best to include in the welcome post of your group, feel free to replace them with whatever you think is best)

What kind of events does your group usually run? What does it usually do?

How frequently does your group organize events or meet?

Who would be a good fit for you group?

Should they have any particular skills or have done some specific background reading?
Shoulder Advisors 101
by Duncan Sabien (Inactive) — 2021-10-09T05:30:57.372Z

Motivation for post: As a former CFAR instructor, longtime teacher, and rationality pundit, I find myself giving lots of advice in lots of different contexts. I also try to check in from time to time to find out which bits of advice actually proved helpful to people. Over the years, I've heard from a genuinely surprising number of people that my (offhand, very basic, not especially insightful) thoughts on "shoulder advisors" were quite useful to them, and remained useful over time. So: a primer.

"There's a copy of me inside your head?" Hermione asked.

"Of course there is!" Harry said. The boy suddenly looked a bit more vulnerable. "You mean there isn't a copy of me living in your head?"

There was, she realized; and not only that, it talked in Harry's exact voice.

"It's rather unnerving now that I think about it," said Hermione. "I do have a copy of you living in my head. It's talking to me right now using your voice, arguing how this is perfectly normal."

"Good," Harry said seriously. "I mean, I don't see how people could be friends without that."

The term "shoulder advisor" comes from the cartoon trope of a character attempting to make a decision while a tiny angel whispers in one ear and a tiny devil whispers in the other.

Many people have multiple shoulder advisors. Some, no doubt, carry a literal metaphorical angel and devil around with them. Others may sometimes hear the whispers of some of their favorite beloved fictional characters. It's quite common in my experience for people to have shoulder copies of their parents, or their best friends, or their romantic partners, or particularly impactful teachers or bosses or mentors.

This is not schizophrenia (though for all I know it may use some of the same hardware, or may be a low-key, non-pathological version of schizophrenia in the same way that a healthy self-preservation instinct could be thought of as a low-key, non-pathological version of a phobia or an anxiety disorder).

Rather, there is simply some kind of subroutine in the brain of most humans that is capable of taking in training data and learning what a given person (or character, or archetype) would say, in a given situation. It's predictive software, likely evolved in response to the need to model other chimps in the ancestral environment, and strongly selected for due to the fact that being able to model those...