trentmkelly/LessWrong-43k huggingface dataset:
Podcast with Oli Habryka on LessWrong / Lightcone Infrastructure
by DanielFilan — 2023-02-05T02:52:06.632Z
Link: https://thefilancabinet.com/episodes/2023/02/05/6-oliver-habryka.html

OK sorry to over-advertise but it seemed like this one would be of interest to the LessWrong and EA communities. Episode description below, audio is here, or search for "The Filan Cabinet Habryka" wherever you listen to podcasts.

In this episode I speak with Oliver Habryka, head of Lightcone Infrastructure, the organization that runs the internet forum LessWrong, about his projects in the rationality and existential risk spaces. Topics we talk about include:

How did LessWrong get revived?

How good is LessWrong?

Is there anything that beats essays for making intellectual contributions on the internet?

Why did the team behind LessWrong pivot to property development?

What does the FTX situation tell us about the wider LessWrong and Effective Altruism communities?

What projects could help improve the world's rationality?
Misleading Fast Charging Specs
by jefftk — 2023-02-05T02:50:01.170Z

I was looking at the specs for the

Kia EV6 after someone
brought it up in a

discussion:

DC Fast Charge Time (10-80% @ 350 kW via Electric Vehicle Supply
Equipment) Level 3 Charger: Approx. 18 min.

If you're not familiar with EVs or other similar equipment you might
think that this draws a constant 350kW, but charging a 77.4 kWh
battery from 10-80% at 350kW would take only 9min so it can't be that.
Instead, EVs are smart: they communicate with the charger to draw
varying amounts of current depending on how quickly the battery can
accept charge.

So then you might think that 350kW reflects the peak current the car
draws. But no: when P3
Group measured it they found it peaks at 235kW, before throttling
back when the battery gets to 50%.

This isn't unique to Kia: I started going through P3's report looking
up advertised rates, and drawing much less than advertised was the
norm. Even starting from a low state of charge and preconditioning
the battery for fast charging:

Make
Model
Advertised
Measured
Ratio

Kia
EV 6
350 kW
235 kW
0.67x

Hundai
Kona
100 kW
~75 kW
0.75x

VW
ID.3
[EDIT: 120]
kW
103 kW
0.86x

Hundai
IONIQ 5
>250 kW
~215 kW
0.86x

The two exceptions I found were very small cars that didn't make large
claims: the Fiat 500e (85kW)
and Mini Cooper SE (50kW)
did test as advertised. [EDIT: Tesla is also fine; I originally
misread the report and thought it peaked at 146 kW, but it does hit
it's 250 kW claim.]

On the other hand, peak current doesn't actually matter very much:
these cars have a range of charging curves and what actually matters
is how quickly a charger extends your range. P3 also ran that test,
looking at how many miles you can go with 20min of charging from 10%,
and on this metric the EV6 came in first, hitting 192mi. So while I
don't want to be too hard on Kia—they do seem to have made a car
that can charge very fast—listing 350kW when they only peaked at
235kW is weird.

Comment via: facebook, mastodon
I hired 5 people to sit behind me and make me productive for a month
by Simon Berens — 2023-02-05T01:19:39.182Z
Link: https://www.simonberens.com/p/i-hired-5-people-to-sit-behind-me

Warning: this is not in typical LessWrong "style", but nevertheless I think it is of interest to people here.

Most people approach productivity from the bottom up. They notice something about a process that feels inefficient, so they set out to fix that specific problem. They use a website blocker and a habit tracker, but none of these tools address the root problem. Personally, I even went as far as making my own tools, but they yielded only marginally more productive time. I craved more, and I was willing to go as far as it takes. I wanted to solve productivity top down—with a system that would enforce non stop productivity with zero effort on my part.

I had tried less intense “watch you work” solutions before. Sharing a screen with someone through

FocusMate

coworking was great, but I had problems scheduling and keeping consistent sessions because of my chaotic calendar.

StudyTogether

’s leaderboard was a great way to push myself to spend hours in the server, but I found myself eating dinner or napping instead of being productive with nobody the wiser.

I decided it was time to try the nuclear option: having people physically sit behind me to keep me on task. And if I was going to do that I was going to do it right: they’d be there 16 hours a day and only leave for me to sleep. (I have an endlessly growing list of projects I want to make, books I want to read, and skills I want to learn, so productivity means a lot to me!)

It fit my chaotic schedule well, because if I had a call or appointment I would step out, and then go right back to work when I would get back. There was also no way to game the system because they could see everything I was doing.

Hiring

I made the following Craigslist post and eagerly refreshed my inbox:

At first, I interviewed applicants about their data entry and cooking skills, but realized it was far more important to get a feel for how comfortable we were working around each other. I moved all but one of the interview candidates who actually showed up (which was only ⅓!) to the trial stage and, in the end, chose three people, with two others as backups.

This is what the shift schedule looked like (not their real names):

(I didn’t mean to only hire women; it just turned out that way. One guy actually canceled at the last minute. For reference, ~70% of my applicants were women.)

The Setup

The Experience

Sunday night, before the ...
Modal Fixpoint Cooperation without Löb's Theorem
by Andrew_Critch — 2023-02-05T00:58:40.975Z

Followed By: Payor's Lemma in Natural Language
TL;DR: This post introduces a novel logical approach to achieving group-scale cooperation, based on modal fixpoint theory. This approach is both easier to understand and roughly 3x more efficient than previous approaches that factored through Löb's Theorem, measured in terms of the length / complexity of the proofs involved.

The following theorem was inspired by Scott Garrabrant, and uses a lemma of James Payor in place of Löb's Theorem to prove cooperation between a group of agents. I'll state the theorem for three agents because that's most illustrative of what's going on:

Theorem: Suppose A,B, and C are agents that return "true" to signify cooperation and "false" to signify defection. Let E=A∧B∧C, so E is the statement that "everyone cooperates". Let □A, □B, and □C denote proof systems that extend Peano Arithmetic, let □EX stand for □AX∧□BX∧□CX, and suppose the agents behave in a manner satisfying the following conditions:

⊢□A(□EE→E)→A
⊢□B(□EE→E)→B
⊢□C(□EE→E)→C
Then it follows that ⊢E, i.e., everyone cooperates (provably!).
(Intuitively, the strategy of the agents in this theorem is to check that the group is trustworthy in a certain way before joining (cooperating with) the group. The theorem shows that the collective check on trustworthiness nests inside itself in a way that self-validates and yields cooperation.)
Proof:
⊢□A(□EE→E)∧□B(□EE→E)∧□C(□EE→E)→A∧B∧C, by combining 1, 2, and 3 with ∧.
⊢□E(□EE→E)→E, from 4 by the definition of E and □E.
⊢E, by the lemma below with x=E.
[end proof]

The key to the theorem is the following lemma, due to James Payor:

Lemma: If ⊢□(□x→x)→x then ⊢x.
Historical note: Originally this lemma assumed ⊢□(□x→x)↔x, but then I realized only the forward implication is needed, so I rewrote it.

Proof: The proof uses the same modal rules of inference for □ as Löb's theorem, namely, necessitation and distributivity:

⊢x→(□x→x), by tautology (A→(B→A)).
⊢□x→□(□x→x), from 1 by □ necessitation and distributivity.
⊢□(□x→x)→x, by assumption.
⊢□x→x, from 2 and 3 by modus ponens.
⊢□(□x→x), from 4 by □ necessitation.
⊢x, from 5 and 3 by modus ponens.
[end proof]

Sweet! In comparison to Löb'...
Who invented knitting? The plot thickens 
by eukaryote — 2023-02-05T00:24:39.706Z
Link: https://eukaryotewritesblog.com/2023/02/03/who-invented-knitting-the-plot-thickens/

Last time on Eukaryote Writes Blog: You learned about knitting history.

You thought you were done learning about knitting history? You fool. You buffoon. I wanted to double check some things in the last post and found out that the origins of knitting are even weirder than I guessed.

Humans have been wearing clothes to hide our sinful sinful bodies from each other for maybe about 20,000 years. To make clothes, you need cloth. One way to make cloth is animal skin or membrane, that is, leather. If you want to use it in any complicated or efficient way, you also need some way to sew that – very thin strips of leather, or taking sinew or plant fiber and spinning it into thread. Also popular since very early on is taking that thread, and turning it into cloth. There are a few ways to do this.

By the way, I’m going to be referring to “thread” and “yarn” interchangeably from here on out. Don’t worry about it.

(Can you just sort of smush the fiber into cloth without making it into thread? Yes. This is called felting. How well it works depends on the material properties of the fiber. A lot of traditional Pacific Island cloth was felted from tree bark.)

Now with all of these, you could probably make some kind of cloth by taking threads and, by hand, shaping them into these different structures. But that sounds exhausting and nobody did that. Let’s get tools involved. These different structures correspond to some different kind of manufacturing technique.

By far, the most popular way of making cloth is weaving. Everyone has been weaving for tens of thousands of years. It’s not quite a cultural universal but it’s damn close. To weave, you need a loom.[1] There are ten million kinds of loom. Most primitive looms can make a piece of cloth that is, at most, the size of the loom. So if you want to make a tunic that’s three feet wide and four feet long, you need cloth that’s at least three feet wide and four feet long, and thus, a loom that’s at least three feet wide and four feet long. You can see how weaving was often a stationary affair.

Recap

Here’s what I said in the last post: Knitting is interesting because the manufacturing process is pretty simple, needs simple tools, and is portable. The final result is also warm and stretchy, and can be made in various shapes (not just flat sheets). And yet, it was invented fairly recently in human history.

I mostly stand by what I sai...
Some miscellaneous thoughts on ChatGPT, stories, and mechanical interpretability
by Bill Benzon — 2023-02-04T19:35:27.853Z

I started playing with ChatGPT on December 1, 2023 and made my first post on December 3. Including this post, I’ve made 54 posts about ChatGPT. Nine of those posts report the work of others (45), perhaps with a comment or two from me, and the other report my own experiments and observations. Some of those posts can be quite long, over 2000 words, in part because I’m including examples of output from ChatGPT. I suspect that’s the longest run of work on a single subject since I’ve been posting to New Savanna, which started in April 2010.

It's time to ramble through some things just to get them on the table

Where do things stand with the tantalizing working paper?

I’m not sure. I uploaded the first version on Jan. 23, the second version on Jan. 28, and then, the next day, decided I needed to revise it yet again. I uploaded a long post on Thursday (Feb 2) in which I thought things through, ChatGPT: Tantalizing afterthoughts in search of story trajectories [induction heads]. I’m still thinking them through.

Do I make a minimal revision for Version 3 and upload it ASAP? I do I try to do a bit better than that? At this moment it’s not clear to me just what a minimal revision would be. Perhaps I could add a paragraph or two up front saying that I’ve decided to drop the idea of a story grammar in favor of the idea of a story trajectory, but that a proper accounting for the change is more than is sensible for this paper.

Meanwhile I’m working on a relatively short paper in which I publish a number of before-and-after-tables like I did in A note about story grammars in ChatGPT. That note had four tables. The short paper I’m working on would have ten or more tables, plus light commentary on them.

We’ll see.

Mechanistic interpretability

The need to change things was brought on by looking at a very interesting paper by some researchers at Anthropic, In-Context Learning and Induction Heads, which I discussed in Thursday’s tantalizing post (link above). They are working on mechanistic interpretability. What is going on in the neural net to produce the behavior we observe?

By working with ‘toy model’ transformers (with only two layers) these researchers were able to identify a mechanism that they call induction heads. Judging from what they say about that mechanism, it seems that induction heads are responsible for performing the story generation task I’ve been exploring. The idea is...
O(“AGI Safety”)>O(“Stop Tyrants”)
by AnthonyRepetto — 2023-02-04T18:38:55.133Z

~ AGI safety is at LEAST as hard as a protocol which prevents tyranny ~

When we want to keep ourselves safe from AGI, then one of the key criteria is “can we turn it off if it tries to go berserk?” That is the same requirement whenever we put some person in charge of an entire government: “can we depose this person, without them using the military to stop us?”

AGI has extra risks and problems, being super-intelligent, while most dictators are morons. Yet, if someone told me “we solved AGI safety!” then I would happily announce “then you’ve also solved protecting-governments-from-dictators!” You might be able to avoid all dictators in a way which does NOT cure us of AGI risks… though, if you reliably prevent AGI-pocalypse, then you’ve definitely handled dictators, using the same method.

So, does that mean we’ll soon find panacea to AGI-death threats? Considering that we haven’t stopped dictators in the last few… centuries? Millenia? Yeah, we might be screwed. Considering that dictators have nukes, and can engineer super-viruses… Oh, and that would imply: “Dictators are the existential risk of ‘a berserk machine we can’t turn-off’… meaning that we need to fight those AGI overlords today.”
Monthly Doom Argument Threads?  Doom Argument Wiki?
by LVSN — 2023-02-04T16:59:31.537Z

Disclaimer: I might have no idea what I'm talking about and my proposal might be an infohazard but it also might save us all; I don't know

LessWrong has monthly AGI safety basic questions threads where people typically ask in the comments why their imagined solution wouldn't work, without understanding the problem. We are essentially prompting people to act like Anti-Eliezers in this way.

I can't read Eliezer's mind super well. I don't know what solutions he will approve but he'll probably disapprove 99+% of them. Maybe he will feel better-assured if I am training myself in seeing the difficulty of the task instead of the ease.

And so, I think what would be more productive is a monthly AGI Doom Argument thread, where people make theorem-style multi-step logical arguments about what is necessary or sufficient for aligned AGI, and why attempting to acquire those desiderata is a terrible idea. That will make people more like the kind of people who would spontaneously write the List of Lethalities from scratch without being prompted to, so goes my hypothesis.

Each argument should be associated with its creator and numerically indexed by its recency of creation, and then maybe we could find some way to store, categorically sort, and rate on multiple metrics, those arguments on a wiki or something.

Here's an example:

LVSN's Doom Argument #1:

1. In order for an AI to become aligned, it must live with humans and evolve bottom-up thinking styles analogously to humans through reproduction

2. Living and evolving analogously to humans takes several generations

3. We don't have several generations' time before AGI

4. Therefor when AGI comes it will not be aligned

Maybe this is a dumb argument. Maybe it's a bad argument. Maybe the premises are definitely wrong or maybe we don't know if they're wrong. Maybe it's ignoring a lot of desiderata. Maybe it's right. Whatever you have to say about the argument, the argument can be labeled as such. Eventually the strongest arguments for our doom will be distilled, and copy/pasting those distillations will be a lot easier than explaining to each alignment-wannabe on a case-by-case basis why their solution doesn't work.

This would create progress on simply understanding the problem. The more would-be alignment savants we can get to actually understand the problem, the better chances we have of dying with dignity.
The Future of Structured Self Improvement
by  — 2023-02-04T16:02:37.348Z
Link: https://guildoftherose.org/articles/the-future-of-structured-self-improvement

Two years ago, we founded the Guild of the ROSE to provide structure and community for people interested in self improvement. Since then, we've grown from a three-month Alpha project to a year-round workshop host.

If you've been in our Discord server, you've probably seen regular announcements about people leveling up on the Practitioner's Path, our RPG-inspired self improvement framework. You might even have heard rumors that we've been working on a new version of the Path.

Practitioner's Path 2.0

The Practitioner's Path 2.0 overhauls every aspect of the Path system, from Bugs to the Character Sheet to levels. The new system is split into three parts: Attributes, Tasks, and Skills. Together, these make up your Character Sheet.

Attributes

Attributes measure long-term metrics such as:

Net Worth (how much money you have)

Reach (how many people you can influence)

Stamina (the efficiency of your cardiovascular system)

The Attributes section of the Character Sheet gives a sense of long-term progress on tangible metrics. Of course, not everything can be tracked as a metric, which is where Skills and Tasks come in.

Tasks

A Task is something that you want to do or achieve. Tasks can be nested inside each other, representing the way that a complex goal can be split into smaller goals.

For example, the Task of "Learn how to program" could be broken into the subtasks of "Decide what language I want to learn" and "Find a tutorial on the language". This nesting system allow you to break down complex goals into easily-achievable chunks.

Skills

The core of the new Path system is the Skilltree. Skills provide a structured approach to self-improvement and a sense of progression as you acquire them. Each skill has a certain amount of XP, allowing you to track your progress on a granular and broad level.

Skills in the Path are organized into three main categories: Pragmatist, Meditative, and Empiricist. Acquiring a Skill will only level up its category — so you can be a high-level Pragmatist but a low-level Empiricist. This division allows you to specialize in a given tree and pursue the Skills that you find most interesting and rewarding.

Pragmatist (red): Take direct action, make money, and expand your social network.

Meditative (green): Make art, improve your mental and physical health, and build a community.

Empiricist (blue): Learn new things, teach others, run experiment

...
Empathy as a natural consequence of learnt reward models
by beren — 2023-02-04T15:35:23.894Z

Epistemic Status: Pretty speculative but built on scientific literature. This post builds off my previous post on learnt reward models. Crossposted from my personal blog.

Empathy, the ability to feel another's pain or to 'put yourself in their shoes' is often considered to be a fundamental human cognitive ability, and one that undergirds our social abilities and moral intuitions. As so much of human's success and dominance as a species comes down to our superior social organization, empathy has played a vital role in our history. Whether we can build artificial empathy into AI systems also has clear relevance to AI alignment. If we can create empathic AIs, then it may become easier to make an AI be receptive to human values, even if humans can no longer completely control it. Such an AI seems unlikely to just callously wipe out all humans to make a few more paperclips. Empathy is not a silver bullet however. Although (most) humans have empathy, human history is still in large part a history of us waging war against each other, and there are plenty of examples of humans and other animals perpetuating terrible cruelty on enemies and outgroups.

A reasonable literature has grown up in psychology, cognitive science, and neuroscience studying the neural bases of empathy and its associated cognitive processes. We now know a fair amount about the brain regions involved in empathy, what kind of tasks can reliably elicit it, how individual differences in empathy work, as well as the neuroscience underlying disorders such as psychopathy, autism, and alexithmia which result in impaired empathic processing. However, much of this research does not grapple with the fundamental question of why we possess empathy at all. Typically, it seems to be tacitly assumed that, due to its apparent complexity, empathy must be some special cognitive module which has evolved separately and deliberately due to its fitness benefits. From an evolutionary theory perspective, empathy is often assumed to have evolved because of its adaptive function in promoting reciprocal altruism. The story goes that animals that are altruistic, at least in certain cases, tend to get their altruism reciprocated and may thus tend to out-reproduce other animals that are purely selfish. This would be of especial importance in social species where being able to form coalitions of likeminded and reciprocating...
Mech Interp Project Advising Call: Memorisation in GPT-2 Small
by Neel Nanda — 2023-02-04T14:17:03.929Z

I've recently been having advising calls with REMIX teams (Redwood's interpretability sprint) trying to give advice & feedback on projects. As an experiment, I've published a recording of one advising call (with Tessa Barton & Kushal Jain on memorisation in GPT-2 Small), I'm curious whether this is useful to anyone! IMO getting detailed feedback from a more experienced research is one of the best ways to improve at research, but have no idea whether someone else's feedback is comparatively useful, or whether my advice is good enough lol. Thanks to the team for being down to publish this, and the work!

https://youtu.be/39hDx25qsS8
Do IQ tests measure intelligence? - A prediction market on my future beliefs about the topic
by tailcalled — 2023-02-04T11:19:29.163Z
Link: https://manifold.markets/tailcalled/do-iq-tests-measure-intelligence?r=dGFpbGNhbGxlZA

I created a market on the meaning of IQ tests. Would be curious to hear arguments.

The text of the market also contains my initial position on the question.