trentmkelly/LessWrong-43k huggingface dataset:
My side of an argument with Jacob Cannell about chip interconnect losses
by Steven Byrnes — 2023-06-21T13:33:49.543Z

Context / How I came to write this

Jacob Cannell (@jacob_cannell) made some claims about fundamental limits of interconnect losses on chips at

Brain Efficiency: Much More Than You Wanted To Know

, and in particular claimed that current chips have interconnect losses close to as low as they will ever get.

When I read that claim I thought it was wrong, as was Jacob’s discussion of interconnect losses more generally, but

I didn’t (and still don’t) think the disagreement directly mattered for AI x-risk

, so I mostly didn’t want to spend time arguing about it.

But then later Alexander Gietelink Oldenziel wrote

$250 prize for checking Jake Cannell's Brain Efficiency

, and I wound up in a

15-comment back-and-forth with Jacob about it

, before ducking out. (Other people continued that thread afterwards).

Unfortunately, I quit the discussion while still being confused about where Jacob was coming from. So this post will not be maximally good and useful, sorry. Nevertheless, here’s a summary of my current perspective and understanding, in case anyone cares.

Also see comments section at the bottom for a response by Jacob.

(Jargon level: medium-low maybe? There is still some unexplained physics & EE jargon, but hopefully I made the centrally important parts accessible to non-experts. DM or email me if something is confusing, and I will try to fix it.)

(All numbers in this post should be treated as Fermi estimates.)

(Thanks very much to Jacob for his extraordinary patience in trying to explain to me his perspective on this topic. And also his perspective on many other topics!)

Background to the technical disagreement

“Interconnects” send information from one point to another on a chip.

The fundamental thermodynamic limit for the power required to send a bit of information from point A to point B is 0.[1] As a stupid example, there is a lot of digital information on Earth, and it all travels 1012 meters in orbit around the sun each year for roughly zero energy cost.

Chip interconnect losses are obviously much much higher than the thermodynamic limit of “zero”—they might even constitute a majority of chip power consumption these days. Everyone knows that, and so does Jacob. So what is he saying?

I think Jacob divides the world of interconnects into two categories, “reversible” and “irreversible” interconnects, with the former including optical interco...
Short timelines and slow, continuous takeoff as the safest path to AGI
by rosehadshar — 2023-06-21T08:56:30.675Z

There are various different timelines/takeoff dynamics scenarios:

Figure 1

In this post, we’re going to set out our understanding of the case for short timelines and slow, continuous takeoff as the safest path to AGI.

A few reasons why laying out this case seems worth doing:

OpenAI’s strategy assumes that short timelines and slow, continuous takeoff are the safest path to AGI.[1] Whether or not this is the safest path, the fact that OpenAI thinks it’s true and is one of the leading AI labs makes it a path we’re likely to take. Humanity successfully navigating the transition to extremely powerful AI might therefore require successfully navigating a scenario with short timelines and slow, continuous takeoff.
Short timelines and slow, continuous takeoff might indeed be the safest path to AGI. After spending some time trying to state the strongest version of the case, we think it’s stronger than we initially expected (and than many in the AI safety community seem to assume).
If it’s not true that short timelines and slow, continuous takeoff are the safest path to AGI, it might be really important to convince people at OpenAI that this is the case. It seems unlikely that it’s possible to do this without deeply understanding their current view.

The arguments we’re going to lay out are:

Slow, continuous takeoff is safer than fast, discontinuous takeoff
Slow, continuous takeoff is more likely given short timelines than long ones
Time during takeoff is more valuable than time before, so it’s worth trading time now for time later
It’s possible to safely navigate short timelines and slow, continuous takeoff

Note that Sam Altman usually refers just to slow takeoff rather than slow continuous takeoff, but in this piece we discuss slow continuous takeoff, because we think that it’s the strongest way of making this case and that it’s likely that this is what Altman means.

Disclaimers:

Neither of us work at OpenAI or are in touch with OpenAI leadership. This post represents our best guess charitable interpretation of the case. We’ve done a lot of filling in the blanks on our side in order to make the arguments as coherent as possible. Our ideal in writing this post is to pass an

ideological Turing Test

, where Sam Altman could read this post and think ‘yes, exactly, that’s why I think that’. Probably we fall short of this as we haven’t engaged with Sam directly.
Neith

...
A way to make solving alignment 10.000 times easier. The shorter case for a massive open source simbox project.
by AlexFromSafeTransition — 2023-06-21T08:08:10.218Z

When you have the code that might be an AGI and probably misaligned, how do you test it in a way that is safe? The world has not yet converged on a way to do this. This post attempts to provide an outline of a possible solution. In one sentence: perform all learning and testing in a virtual world that contains no information on the real world, it is crucial that the AI must believe it is not being watched/judged from the outside. A simulation sandbox or in short a simbox. If done well, this would lead to having unlimited retries to test alignment techniques. If this idea survives scrutiny, it should lead to an ambitious, well funded large scale open source effort to build this world.
EDIT: After writing this post, through a comment here, I was pointed to the post LOVE in a simbox is all you need, due to this, I renamed the concept here to simbox and made a lot of updates. This IMO well written lengthy post from 2022 describes very similar ideas among other topics. Why read my post? I give you a shorter case for why we need a simbox. I also give an argument for why it is crucial to threat the AI horribly and why we need one or a few massive projects to build this now, while the simbox post proposes to have many different simboxes. I also have created a short new religious text for the AI's that could also be true for our world. Among other additions.

Treacherous turn talk: The problem

In the book Superintelligence by Nick Bostrom, he has a chapter on the treacherous turn it is the same thing as deceptive alignment. I explain the concept here. When you have an AI with some goal, with a good understanding of humans and the world, then it will understand that when the humans do not like its actions (for example disobeying its creators) that they will shut down the AI or modify its goals. The instrumental convergence theorem says that for most goals, the AI will pursue self-preservation as a sub-goal (among other things). Therefore, we can expect the AI to "play nice" and to perform the actions it predicts the humans would like to see. The treacherous turn is then the moment when the AI has gained enough power, more power than the humans, where it turns: To stop playing nice and start pursuing its own goals without apology.

I want to clarify that this problem occurs for an autonomous agent that can act for a longer time horizon. G...
My tentative best guess on how EAs and Rationalists sometimes turn crazy
by habryka — 2023-06-21T04:11:28.518Z

Epistemic status: This is a pretty detailed hypothesis that I think overall doesn’t add up to more than 50% of my probability mass on explaining datapoints like FTX, Leverage Research, the LaSota crew etc., but is still my leading guess for what is going on. I might also be really confused about the whole topic.

Since the FTX explosion, I’ve been thinking a lot about what caused FTX and, relatedly, what caused other similarly crazy- or immoral-seeming groups of people in connection with the EA/Rationality/X-risk communities.

I think there is a common thread between a lot of the people behaving in crazy or reckless ways, that it can be explained, and that understanding what is going on there might be of enormous importance in modeling the future impact of the extended LW/EA social network.

The central thesis: "People want to fit in"

I think the vast majority of the variance in whether people turn crazy (and ironically also whether people end up aggressively “normal”) is dependent on their desire to fit into their social environment. The forces of conformity are enormous and strong, and most people are willing to quite drastically change how they relate to themselves, and what they are willing to do, based on relatively weak social forces, especially in the context of a bunch of social hyperstimulus (lovebombing is one central example of social hyperstimulus, but also twitter-mobs and social-justice cancelling behaviors seem similar to me in that they evoke extraordinarily strong reactions in people).

My current model of this kind of motivation in people is quite path-dependent and myopic. Even if someone could leave a social context that seems kind of crazy or abusive to them and find a different social context that is better, with often only a few weeks of effort, they rarely do this (they won't necessarily find a great social context, since social relationships do take quite a while to form, but at least when I've observed abusive dynamics, it wouldn't take them very long to find one that is better than the bad situation in which they are currently in). Instead people are very attached, much more than I think rational choice theory would generally predict, to the social context that they end up in, with people very rarely even considering the option of leaving and joining another one.

This means that I currently think that the vast m...
The Importance of Judging: A Reflection on Rational Thought
by CrimsonChin — 2023-06-20T22:49:42.182Z

Epistemic status: Mostly sorting out my own thoughts here. Feel free to try to convince me I am wrong.

In earlier drafts of this post, I held the belief that "judging people is bad." I even went to great lengths to avoid passing judgment and occasionally adjusted my moral framework accordingly. However, I have recently come to realize that my previous stance was mistaken. In this post, I will explain how I reached this conclusion, although I won't provide extensive evidence. While not a groundbreaking idea, this shift in my long-held belief holds personal significance to me.

Is Judging Beneficial?

Defining Judgment

For the purpose of this discussion, I will define judgment as follows:

Judgment is the act of forming an opinion regarding the extent of positive and negative impact something will have on oneself or the world.

Although this definition may appear complex, it captures the essence of how the term is typically used in literature and personal experiences.

Surprisingly, merely clarifying the definition of judgment was sufficient to completely reverse my moral intuition.

Separating Judgment from Credence

Judgment represents an opinion about the inherent "goodness" or "badness" of something. It is essentially the outcome of a utility calculation—whether the overall impact is positive or negative. However, it is important to differentiate judgment from credence, which denotes the level of belief in a particular judgment.

For instance, I have judged that alcohol is detrimental to my well-being, and I assign 80% credence to that judgment. Consequently, it would require significant life changes and a shift in perspective to convince me otherwise.

On the other hand, I have judged that Justin Timberlake is the best singer from NSYNC, but my credence in this judgment is less than 1%. If a stranger were to challenge my opinion, it would easily change since I hold this view rather lightly.

Both judgment and credence are necessary to motivate action. I would never act upon a judgment in which I have near-zero credence. Even if someone were to inquire, "What is your opinion on X, crimson_chin?" I would respond with something like, "Honestly, I don't know much about it. Do you?" While this response may be misleading, as my gut can form a judgment on anything, without supporting reasoning, I assign zero credence.

Judgment and Action

The combination of judgment and credence ...
"Natural is better" is a valuable heuristic
by Neil  — 2023-06-20T22:25:49.841Z

tl;dr Until we understand how complex and chaotic systems[1] in nature work, "natural is better" is a valuable heuristic. It applies wherever science does not understand things well enough, especially when the stakes are high.

If you throw a stick into a forest it will biodegrade quickly and not affect the forest much; this is because forests are used to sticks.[2] Throw a piece of plastic in the forest and you will affect the forest a lot more. Not because plastic is inherently bad; but because the forest has not evolved to deal with plastic and will thus have a harder time breaking it down. There's nothing inherently unhealthy or destructive about plastic, in the same way that there is nothing inherently unhealthy or destructive about sticks: natural systems in the forest are just not used to coping with the former.

Some other things natural biological systems tend to not be used to: nuclear radiation, the vacuum of space, some pesticides, some GMOs, some fertilizers, a global rising of temperatures, asteroid impacts, supernovae, acidification of the oceans, sudden changes in atmospheric composition, etc. The status quo of life on Earth has faced all these things at various moments throughout history. They are examples of perturbations in a very complex system, but, like plastic, they are not inherently bad; they are just things biology has to adapt to.

I feel like this is a fact that both "natural is always better!" types and "I'll roll my eyes at that" types forget. Here's how I see it:

Science is a cool new trick the monkeys figured out which lets them make maps of reality that are so accurate that they know where the levers are, and can pull them in a way that makes the universe behave a certain way. Structure atoms right and the universe will, among many other things, do math for you, generate photons, detect an atom, establish probabilities, store information, land you on the moon, change states of matter, ...

Sometimes, however, science doesn't know where all the levers are: sometimes systems, like forests, are so complex and chaotic that pressing one lever might set off a cascade and bring about a whole slew of unintended consequences. In principle, give science enough time and it will figure out how to press the right levers safely. But science is a rather new thing and the amount of unknowns in nature is vast.

Scie...
№.6 For Those About To Dress...
by party girl — 2023-06-20T21:14:35.656Z
Link: https://affale.substack.com/p/6-for-those-about-dress

Why don't we consider great personal style to be high art?

The dressing room in Karl Lagerfeld’s apartment in Louveciennes. You bet I’ve sunk many hours into browsing

Pamono

for a replica of that gorgeous Bruno Paul dressing table for AFFALÉ H.Q.

Personal style is a dangerous thing for a fashion designer to possess. Never trust a skinny cook, but never, ever trust a chic designer. No one is more susceptible to getting high on their own supply, save for, like, a ski lift operator with a cannabis-flavored side hustle. It’s almost exactly like that Picasso quote about turpentine…

“When art critics get together they talk about Form and Structure and Meaning. When artists get together they talk about where you can buy cheap turpentine.”

…except, invert it!

“When fashion designers get together they talk about Form and Structure and Meaning. When consumers get together they talk about where you can buy cheap basics.”

As with boxing, and jazz—“

The better it is, the less people appreciate it!

”—exquisite sartorial taste is inherently self-effacing. But not in the trendy,

much-discoursed

“quiet luxury” sense that pop culture is currently enamored with. (Thanks, Succession

costume department

; thanks, ski accident victim-victim

Gwyneth Paltrow

.)

No, truly great personal style is self-effacing precisely because it’s so over-the-top. It doesn’t cover its own tracks, so much as it stomps around on those stilts with giant molds of different kinds of animals’ paws strapped to the bottoms. People inevitably grow accustomed to filtering it out,

chunking

it into your broader personality, assigning you a kind of emeritusy GOAT status: “Oh, that’s just so-and-so; they have great taste.”

So you become obsessed with finding new, heretofore inconceivable ways to cry, “Wolf!”. Because when you’ve already committed to going all-in, each and every hand, upping the ante is no longer a viable get-your-kicks strategy. Congratulations, you’ve

shot the moon

. Hitting upon dramatic new ways to splay all of your chips out,

Casino Royale-

style

, is all you can do to stave off death-by-boredom.

Like, concrete recent example: over-engineering feathers:

“…Everyone was still abuzz over [stylist Law Roach’s] client Hunter Schaefer attending the Vanity Fair Oscar party with only an Ann Demeulemeester feather as a top. That she avoided a wardrobe malfunction is a styling feat at least worthy of an engineering Ph.

...
Frame Bridging v0.8 - an inquiry and a technique
by Unreal — 2023-06-20T19:46:39.502Z

Hello dear friends,
Question.
What is the nature of the fracture found in unbridgeable frames?
In one meeting with my teacher, I encountered what seemed to be an Unbridgeability. No way to communicate. An impasse.

I couldn't find a single sentence to say that would help me reach him where he was, from where I was. Our frames were too different. There seemed to be a chasm between us.
Later on, he would ask me about the nature of this thing. What made it so? What is the origin? What is the cause? What is its nature? ((Not in these words, but here, I try to capture the essence of the deeper question.))
(When we know the origin and the cause, we may also know the escape.)
He was asking me a real question. Real questions have the power to capture our minds. My teacher pointed directly at the mystery, and now I have this question, and it has me.
If I were expansive and large enough to hold multiple frames simultaneously without losing anything valuable from either of them, I think I could find a way to bridge. It would take a lot of computation, perhaps? A lot of pedagogical skill—since pedagogy is basically bridge-building. And I don't know how to do distant-frame-bridging pedagogy without a fair amount of computation.
But!
Is it really the case that the individuals need to be the ones doing the work? That this work is done at the nodes, rather than in the line?
Is there a way to trust a third, larger process or force? Something that holds the in-between? I imagine a mysterious alternative process. Circling tries to touch on this.
Using our bodies, minds, egos, and beings, these frames can find voice and manifestation in reality... enough for two different frames to try to meet from across the room. What is it like to see? What is it like to feel? What happens in the meeting?
In order for this to work as cleanly as possible, our 'small selves' are best kept in check. Otherwise their insecurities try to wrangle the communication away from the main thread.
The pure meeting... clear of hindrances...
I yearn for this moment. And opportunities to create this moment. My Eros is drawn to this, but also I personally feel helpless in trying to create the circumstance...
This is the True "Double Crux." The technique the rationalists tried to invent through a step-by-step process. But it was like teenagers trying to invent t...
Public Transit is not Infinitely Safe
by jefftk — 2023-06-20T18:40:02.011Z

I recently came across

this tweet, screenshotted into a

popular Facebook group:

Here's a truth bomb:

Take the U.S. city you're most afraid of, one with a very high murder
rate or property crime rate.

If it has any sort of public transit, it is still statistically safer
to use public transit in that city at ANY time of day than to drive
where you live.

—Matthew Chapman, 2023-06-14

This got ~1M views, doesn't cite anything, was given without any
research, and, I'm pretty sure, is wrong. While I'm a major fan of public transit, they've stacked
this comparison in a way that's really favorable to cars, and it's not
surprising that public transit doesn't make it.

Safety is a complicated concept, and risks are situational: in a car
you're much more likely to be hurt in a collision, while on public
transit you're much more likely to be hurt by another passenger. To
get a clear comparison I looked just at deaths, which is also an area
where we can get good statistics.

I can't find a listing of public transit agencies by homicide rate,
but Chicago is a large city with a lot of homicides and they make
their data available so let's look there. In 2022 there were 244M CTA
rides. Downloading the Chicago
Police Data and filtering to 2022 homicides on public transit, I
see nine. This is 3.7 homicides per 100M trips.

(Note that the original claim was for any city, and there are dozens
of US cities with homicide rates higher
than Chicago's. I think it's pretty likely that at least one of
these cities has a public transit system with more homicides than the
CTA.)

To fairly compare this to the risk from driving, we need homicides per
distance. How long is a trip? I can't find 2022 data, but the CTA's
President's
2020 Budget Recommendations gives 4.1mi (1359M + 613M passenger miles
divided by 230M + 249M trips). This means 0.9 homicides per 100M miles
travelled.

I live in MA, and while 2022 FARS
data isn't out yet, in 2021 there were 0.71
driving deaths per 100M miles travelled. While MA does happen to be the safest
state for driving, MN, NH, and RI also have lower driving fatality rates than the CTA.

I'm not sure how to get to a place where we see fewer bold false
claims, but a culture of giving sources, and supporting requests for
them, seems like it would help?

Comment via: facebook, mastodon