Harley-ml/lesswrong huggingface dataset:
# Feature request: voting buttons at the bottom?

This is a brief question/request regarding LessWrong and AlignmentForum UI and voting buttons.

## Uncontroversial (I expect) factual statements

When I read a post or comment which fills more than one screen, I want to consume the whole content before providing any reinforcement or agreement signals.

But the vote buttons are rendered at the top of posts and comments, so I have to scroll back up again[^top-to-bottom], and (especially if I'm reading through a thread of comments) it can therefore be tricky to keep track of my place.

[^top-to-bottom]: I almost always read things from top to bottom, at least on a first reading, which I also expect to be relatively uncontroversial.

## A basic request

Could we have additional voting buttons rendered (perhaps optionally configurably?) at the bottom of posts and comments? This would make the above, presumably common, workflow that little bit more straightforward.

Or (more fiddly but maybe better), could the buttons float and follow me down the page, or some other fancy UI trick?

## A more controversial normative claim

It seems like the factual statement of my reading/digesting/voting workflow above _should_ be the default workflow. After all, how can you provide a reliable voting signal for something you haven't consumed?[^consumed]

Would it make sense to _always_ have the buttons exclusively at the bottom, to nudge[^nudge] the 'correct' workflow to be marginally easier, and 'aberrant' workflows (like voting based on the author's name or the title or introduction)[^gasp] to be marginally more difficult? What might the downsides be (beside the obvious developer opportunity cost)?

[^consumed]: There are of course degrees of what constitutes 'consumed' and 'reliable'!

[^nudge]: cf [Recommendation Features on LessWrong](https://www.lesswrong.com/posts/PfceWjEqdRDvGkKZ8#LessWrong_as_a_Nudge)

[^gasp]: Gasp! Surely not!

### Even more controversial speculation

What if the karma/agreement tallies were also at the bottom? (i.e. unseen until after deliberate scroll)

It wouldn't in principle change my options as a reader, but it would make it easier to avoid anchoring and instead evaluate content on its merit (but I could still readily see the community consensus signal and adopt that information into my all-things-considered opinion).

This seems a lot easier to backfire, because often we filter what to even start reading based on more quickly-digested summaries (probably quite rightly), which include karma and the like, as well as recommendations. It might work better, for that reason, if at all, on comments than on top-level posts.
# Do alignment concerns extend to powerful non-AI agents?

Right now the main focus of alignment seems to be on how to align powerful AGI agents, a.k.a AI Safety. I think the field can benefit from a small reframing: we should think not about aligning AI, but about alignment of systems in general, if we are not already.  
 

It seems to me that the biggest problem in AI Safety comes not from the fact that the system will have unaligned goals, but from the fact that it is superhuman.

As in, it has nearly godlike power in understanding of the world and in turn manipulation of both the world and other humans in it. Does it really matter if an artificial agent gets godlike processing and self-improvement powers or a human will, or government/business?  
 

I propose a little thought experiment – feel free to answer in the comments.

If you, the reader, or, say, Paul Christiano or Eliezer gets uploaded and obtains self-improvement, self-modification and processing speed/power capabilities, will your goals converge to damaging humanity as well?  
  
If not, what makes it different? How can we transfer this secret sauce to an AI agent?  
 

If yes, maybe we can see how big, superhuman systems get aligned right now and take some inspiration from that?
# [Link] Adversarially trained neural representations may already be as robust as
corresponding biological neural representations

Abstract:

> Visual systems of primates are the gold standard of robust perception. There is thus a general belief that mimicking the neural representations that underlie those systems will yield artificial visual systems that are adversarially robust. In this work, we develop a method for performing adversarial visual attacks directly on primate brain activity. We then leverage this method to demonstrate that the above-mentioned belief might not be well founded. Specifically, we report that **the biological neurons that make up visual systems of primates exhibit susceptibility to adversarial perturbations that is comparable in magnitude to existing (robustly trained) artificial neural networks.**

[https://arxiv.org/pdf/2206.11228.pdf](https://arxiv.org/pdf/2206.11228.pdf)
# [Link] Childcare : what the science says

This is a pretty good summary of the research on daycare effectiveness. It is a Much More Than You Wanted To Know-style post from a science blogger [criticalscience](https://criticalscience.medium.com/) who has some interesting posts on Medium.

TL;DR:

*   Adverse effects on children younger than three years old.
*   The effect is worse the younger the children are.
*   The effect is stronger for longer hours.
*   The effect is ~~weaker~~ [CORRECTION](https://www.lesswrong.com/posts/kivCEEpfJXHnJJBAT/link-childcare-what-the-science-says?commentId=vytKuvbBRmZwiJkXp): worse for children from socio-economically better-off families.
*   Nannies are OK.

[Childcare : what the science says](https://criticalscience.medium.com/on-the-science-of-daycare-4d1ab4c2efb4)
# Do you consider your current, non-superhuman self aligned with “humanity” already?

Any actual “hard” superhumans who may be listening are excluded from the audience of this question. I am currently refraining from describing my own ideas to avoid biasing the results; I'm not sure whether I'll reveal something later or not.

I will probably be posting two comments to use the agreement axis of as a sort of poll for the yes/no version of the question, but other more detailed answers and comments are welcome as well. (The reason for using two is so that the per-comment aggregation will make the full counts visible.)
# Child Contracting

The kids came to me with a contract dispute: they disagreed about what the deal had been. After some talking, and hearing what they each thought they'd agreed to, it turned out that the contract wasn't just metaphorical:

[![](https://www.jefftk.com/lily-anna-contract-1.jpg)](https://www.jefftk.com/lily-anna-contract-1-big.jpg)

*"Anna rents the living room from Lily in exchange for Lily's use of bedroom. And Lily will do free cleaning."*

It turned out that Anna signed it after hearing a rough paraphrase. I told Lily that she would need to read any future contracts aloud, word for word, before Anna signed. I also explained about [interpretation against the drafter](https://en.wikipedia.org/wiki/Contra_proferentem), and how Lily should be careful to avoid ambiguity in writing down agreements.

I'm not sure what "free cleaning" means here? Generally they both do very little cleaning, free or otherwise.

Cleaning up this evening I came across a second contract:

[![](https://www.jefftk.com/lily-anna-contract-2.jpg)](https://www.jefftk.com/lily-anna-contract-2-big.jpg)

*"Anna will let Lily use bedroom and bathroom. In exchange Lily will let Anna use the living room and Lily does free cleaning. If furniture is moved or added please put away".*

This one wasn't signed, so perhaps they couldn't agree?
# Robin Hanson asks "Why Not Wait On AI Risk?"

**UPDATE: This is an accidental duplicate. Please go to the first posting** [**here**](https://www.lesswrong.com/posts/H2N6eWw8JgxwKHPM3/linkpost-robin-hanson-why-not-wait-on-ai-risk.)**.**

Since the [AI-Foom Debate](https://www.lesswrong.com/tag/the-hanson-yudkowsky-ai-foom-debate) Robin Hanson has been quiet for a while but now: 

> I feel I should periodically try again to explain my reasoning, and ask others to please help show me what I’m missing.

After having analyzed multiple widely expressed AI concerns from an economical angle and comes to the conclusion:

> Similar concerns have often been expressed about allowing too many foreigners to immigrate into a society, or allowing the next youthful generation too much freedom to question and change inherited traditions. Or \[many other comparable concerns\].
> 
> Many have deep and reasonable fears regarding big long-term changes. And some seek to design AI so that it won’t allow excessive change. But this issue seems to me much more about change in general than about AI in particular.

There is much more in it but take this as a TL;DR.
# Contest: An Alien Message

### Scenario

You're not certain that aliens are real. Even after receiving that email from your friend who works at NASA, with the subject line "What do you make of this?" and a **[single binary file attached](https://github.com/DaemonicSigil/an-alien-message)**, you're still not certain. It's rather unlikely that aliens would be near enough to Earth to make contact with humanity, and unlikelier still that of all the humans on Earth, *you* would end up being the one tasked with deciphering their message. You *are* a professional cryptanalyst, but there are many of those in the world. On the other hand, it's already been months since April Fools, and your friend isn't really the prankster type.

You sigh and download the file.

### Rules

No need to use spoiler text in the comments: This is a collaborative contest, where working together is encouraged. (The contest is between all of you and the difficulty of the problem.)

Success criteria: The people of LessWrong will be victorious if they can fully describe the process that generated the message, ideally by presenting a short program that generates the message, but a sufficiently precise verbal description is fine too.

The timeframe of the contest is about 2 weeks. The code that generated the message will be revealed on Tuesday July 12.

### Why is this interesting?

This contest is, of course, based on [That Alien Message](https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien-message). Recently there was some discussion under [I No Longer Believe Intelligence to be "Magical"](https://www.lesswrong.com/posts/rqAvheoRHwSDsXrLw/i-no-longer-believe-intelligence-to-be-magical) about how realistic that scenario actually was. Not the part where the entire universe was a simulation by aliens, but the part where humanity was able to figure out the physics of the universe 1 layer up by just looking at a few frames of video. Sure it may be child's play for Solomonoff Induction, but do bounded agents really stand a chance? This contest should provide some experimental evidence.
# Are long-form dating profiles productive?

There's been a noticeable trend over the last few years of in this community of writing long-form dating profiles. They have the benefit of being slightly costly signals, requiring significantly more time and attention and a much greater degree of disclosure than can be devoted to a traditional online dating profile. But they are (at present) publicized and distributed manually, sharply limiting the possible audience.

Has anyone who's written one found a date (and subsequently, relationship) by doing so?

Separately, if you've written one, regardless of whether you met anyone because of it, do you believe it was worth the effort?
# [Yann Lecun] A Path Towards Autonomous Machine Intelligence 

# Abstract
> How could machines learn as efficiently as humans and animals?  How could machines learn to reason and plan?  How could machines learn representations of percepts and action plans at multiple levels of abstraction, enabling them to reason, predict, and plan at multiple time horizons?  This position paper proposes an architecture and training paradigms with which to construct autonomous intelligent agents. It combines concepts such as configurable predictive world model, behavior driven through intrinsic motivation, and hierarchical joint embedding architectures trained with self-supervised learning.

___

Meta's Chief AI Scientist Yann Lecun lays out his vision for what an architecture for generally intelligent agents might look like.
# Limits of Bodily Autonomy

Bodily autonomy is a strong consideration in favor of both abortion and optional vaccination. Because it is not the only consideration, however, many of us are in favor of abortion for anyone who wants one while also seeing some cases in which we favor mandatory vaccination. I've recently seen posts from friends supporting the right to abortion, such as shares of screenshots of this tweet:

> If you don't accept that bodily autonomy is an essential unconditional liberty, it's a waste of time talking to you at all. No other liberties survive without that one, more fundamental than property rights: if you don't own yourself absolutely, you own nothing.
> 
> —[@ScavengerEthic](https://twitter.com/ScavengerEthic/status/1521834653954428929)

In some ways this is similar to the use of the slogan '[my body, my choice](https://en.wikipedia.org/wiki/My_body,_my_choice)', but I don't think people generally interpret short phrases as a complete argument; a slogan often draws attention to a major consideration without claiming it's decisive. In this case, however, people are sharing statements that do claim to be a full argument, and make the case for abortion in a way that also makes the case against vaccination requirements.

*Q: A right to abortion isn't anything like a right to refuse vaccination! Their impact is very different, and there are lots of other considerations: vaccination prevents something contagious, pregnancy and childbirth can be deeply difficult, unpleasant, and dangerous, etc.*

A: I agree. Which is why we should make the case for these policies in a way that depends on those considerations, instead of resting the entire case on autonomy.

*Q: I do think bodily autonomy is an essential unconditional liberty, and I'm opposed to both abortion bans and vaccination requirements.*

A: That's a consistent position, but you're not my audience here.

*Q: A vaccine mandate doesn't mean you'll have one forced on you, it just means you can't go certain places. That's very different from threatening jail time for abortions.*

A: If the court had ruled that states could exclude people who had ever had an abortion from restaurants, gyms, universities, and jobs in healthcare, government, or schooling, we would see this as very nearly as bad as a blanket prohibition an abortion. But I would also go farther: a future pandemic could be much worse than covid, and I could see one in which we would need to choose between literally mandatory vaccination and the lives of immunocompromised people.
# Is there any way someone could post about public policy relating to abortion access (or another sensitive subject) on LessWrong without getting super downvoted?

I'm aware there are some sensitive subjects that have been debated to death among rationalists, if not on LessWrong, than elsewhere online, such that it would in general be fruitless to start the conversation over instead of following what has been debated thus far. I'm curious if this is considered to apply to potentially literally any single socially, culturally or politically sensitive subject. It has been several years since I've checked but my impression from before is that the prevailing norm on LessWrong was based on a particular interpretation of "[Politics is the Mind-Killer](https://www.lesswrong.com/posts/9weLK2AJ9JEt2Tt8f/politics-is-the-mind-killer)" that amounts to "politics=anti-rational=downvote on site." This seems contradicted by the original post: 

> I’m not saying that I think we should be apolitical, or even that we should adopt Wikipedia’s ideal of the Neutral Point of View. But try to resist getting in those good, solid digs if you can possibly avoid it. If your topic legitimately relates to attempts to ban evolution in school curricula, then go ahead and talk about it—but don’t blame it explicitly on the whole Republican Party; some of your readers may be Republicans, and they may feel that the problem is a few rogues, not the entire party. 

It seems like the alternative interpretation, though, was adopted because nobody stuck to that and so it was more convenient for reinforcing norms of civility in discourse to go with the "politics=anti-rational=downvote" interpretation.  
  
Is that still a norm? Even if someone like myself posts about it in a way that doesn't suck, is the concern still that others will lower the quality of discourse with their comments, and downvote anyway? (I'm aware there are not formal rules but I'm in essence asking if according to social norms what categories of topics are considered forbidden by default.)
# What success looks like

**TL;DR:** We wrote a post on possible success stories of a transition to TAI to better understand which factors causally reduce the risk of AI risk. Furthermore, we separately explain these catalysts for success in more detail and this post can thus be thought of as a high-level overview of different AI governance strategies.  

Summary
=======

Thinking through scenarios where TAI goes well informs our goals regarding AI safety and leads to concrete action plans. Thus, in this post,

*   We sketch stories where the development and deployment of transformative AI go well. We broadly cluster them like
    1.  Alignment won’t be a problem, …
        *   Because alignment is easy: Scenario 1
        *   We get lucky with the first AI: Scenario 4
    2.  Alignment is hard, but …
        *   We can solve it together, because …
            *   We can effectively deploy governance and technical strategies in combination together: Scenario 2
            *   Humanity will wake up due to an accident: Scenario 3
            *   The US and China will realize their shared interests: Scenario 5
        *   One player can win the race, by …
            *   Launching an Apollo Project for AI: Scenario 6
*   We categorize central points of influence that seem relevant for causing the success of our sketches. The categories with some examples are:
    1.  **Governance:** domestic laws, international treaties, safety regulations, whistleblower protection, auditing firms, compute governance and contingency plans
    2.  **Technical:** Red teaming, benchmarks, fire alarms, forecasting and information security
    3.  **Societal:** Norms in AI, publicity and field-building
*   We lay out some central causal variables for our stories in the third chapter. They include the level of cooperation, AI timelines, take-off speeds, size of the alignment tax, type of actors and number of actors
# Scott Aaronson and Steven Pinker Debate AI Scaling

> Before June 2022 was the month of the possible start of the Second American Civil War, it was the month of [a](https://astralcodexten.substack.com/p/my-bet-ai-size-solves-flubs?s=w) [lively](https://garymarcus.substack.com/p/what-does-it-mean-when-an-ai-fails) [debate](https://astralcodexten.substack.com/p/somewhat-contra-marcus-on-ai-scaling) [between](https://garymarcus.substack.com/p/does-ai-really-need-a-paradigm-shift?s=w) Scott Alexander and Gary Marcus about the scaling of large language models, such as [GPT-3](https://en.wikipedia.org/wiki/GPT-3).  Will GPT-*n* be able to do all the intellectual work that humans do, in the limit of large *n*?  If so, should we be impressed?  Terrified?  Should we dismiss these language models as mere “stochastic parrots”?
> 
> I was privileged to be part of various email exchanges about those same questions with Steven Pinker, Ernest Davis, Gary Marcus, Douglas Hofstadter, and Scott Alexander.  It’s fair to say that, overall, Pinker, Davis, Marcus, and Hofstadter were more impressed by GPT-3’s blunders, while we Scotts were more impressed by its abilities.  (On the other hand, Hofstadter, more so than Pinker, Davis, or Marcus, said that he’s terrified about how powerful GPT-like systems will become in the future.)
> 
> Anyway, at some point Pinker produced an essay setting out his thoughts, and asked whether “either of the Scotts” wanted to share it on our blogs.  Knowing an intellectual scoop when I see one, I answered that I’d be honored to host Steve’s essay—along with my response, along with Steve’s response to *that*.  To my delight, Steve immediately agreed.  Enjoy!  –SA
# CFAR Handbook: Introduction

The [Center for Applied Rationality](rationality.org) is a Bay Area non-profit that, among other things, ran lots of workshops to offer people tools and techniques for solving problems and improving their thinking. Those workshops were accompanied by a reference handbook, which has been available as a [PDF](https://rationality.org/resources/handbook/) since 2020.

The handbook hasn't been substantially updated since it was written in 2016, but it remains a fairly straightforward primer for a lot of core rationality content.  The LW team, working with the handbook's author [Duncan Sabien](https://www.lesswrong.com/users/duncan_sabien), have decided to republish it as a lightly-edited sequence, so that each section can be linked on its own.

In the workshop context, the handbook was a *supplement* to lectures, activities, and conversations taking place between participants and staff. Care was taken to emphasize the fact that each tool or technique or perspective was only as good as it was *effectively applied* to one's problems, plans, and goals. The workshop was intentionally structured to cause participants to actually try things (including iterating on or developing their own versions of what they were being shown), rather than simply passively absorb content. Keep this in mind as you read—mere knowledge of how to exercise does not confer the benefits of exercise!

Discussion is strongly encouraged, and disagreement and debate are explicitly welcomed. Many LWers (including the staff of CFAR itself) have been tinkering with these concepts for years, and will have developed new perspectives on them, or interesting objections to them, or thoughts about how they work or break in practice. What follows is a historical artifact—the rough state-of-the-art at the time the handbook was written, circa 2017. That's an excellent jumping-off point, especially for newcomers, but there's been a lot of scattered progress since then, and we hope some of it will make its way into the comments.
# Challenge: A Much More Alien Message

Since [this challenge](https://www.lesswrong.com/posts/dFFdAdwnoKmHGGksW/contest-an-alien-message) was solved pretty quickly, I thought I'd try to make a substantially more difficult program induction challenge.

### **The Challenge**

Find a short program which generates [this binary file](https://github.com/qu4rt3rb4k3d/a-more-alien-message).

### **Hints (because I expect this to be extremely hard)**

*   For a sense of how simple the program is, I generated the file with a python script of a similar length and complexity to [the solution](https://www.lesswrong.com/posts/dFFdAdwnoKmHGGksW/contest-an-alien-message?commentId=tEKeFo2a9fMEbHkJ5) to the prior challenge.
    *   Other than writing out to the file at the end, the generator uses only basic primitives of the python language (integer arithmetic, logic and flow control, arrays)
*   The only numeric constants used in the generator script are $0, 1, 2, 2^{2^{2^2}}$.
*   The generator script takes nearly 10 minutes to run on my laptop!
    *   So if you have any good hypotheses, you might want to test them with a compiled language for a 100-fold speedup over python.
# Kurzgesagt – The Last Human (Youtube)

TLDW: Big Youtube channel Kurzgesagt released a video on the potential of humanity's future that's I think my favorite videos of theirs yet.

I have a pretty high bar for posting video content to LessWrong, but this video seemed better than most other videos in terms of capturing some of the things that make me interested in a lot of things on LessWrong.
# Quick survey on AI alignment resources

TLDR: In collaboration with CEA, an external collaborator is trying to determine the value of various AI alignment resources. If you are interested in or currently working on AI alignment, please fill out [**this survey**](https://www.guidedtrack.com/programs/qd2mn4k/run). We expect it will likely take 5-15 minutes. Thanks!

\_\_\_

There are a lot of resources available to learn about AI Alignment. For field builders and those interested in the field, it can be difficult to know when to prioritize which readings. To address this, an external collaborator has created a [**survey**](https://www.guidedtrack.com/programs/qd2mn4k/run) to collect feedback on the resources you’ve engaged with, in collaboration with the Centre for Effective Altruism (CEA). 

We hope this survey will:

1.  Inform which materials have previously helped people get into alignment. We can then promote those materials broadly or target them to particularly relevant groups (e.g. students).
2.  Identify gaps where more materials are needed.
3.  Provide feedback to those who made the materials, encouraging them to create more or try new stuff!

**If you are interested in or currently working on AI Alignment, we would be so grateful for your input. We expect this survey to take around 5-10 minutes. **

Once you’ve filled it out, please consider sharing this survey with others in your network.

We appreciate your time. Thanks!
# Cryonics-adjacent question

(This question involves my somewhat bizarre "[mind backup](https://www.lesswrong.com/posts/gZzLaKGuaMqYwmHeJ/what-i-m-about)" project, that was inspired by Rudy Rucker's "Lifebox" idea.)

Unfortunately, I made some HUGE blunders last year involving my life savings and the market (bonds, though I almost wish it was crypto). It's painful to admit this involved laziness on my part as well as bad judgment.  
As a result, my finances are such I can't see a way to pay for both cryonics AND retire.

My question is: could I arrange to mail a hard drive (or future hyper-DVDs) to a company like Alcor after my death, containing all of my "mind backup" data. It's basically hundreds of thousands of disjointed diary entries and diagrams and flowcharts and sketches and lists and preserved records and pics and receipts and instructions and whatnot.  
They would then store this media instead of my frozen corpse (which may be disposed of in a pauper's grave or ideally used for medical research).

Obviously, they should charge less to store a small box of disks for five hundred years at room temperature, than an upside-down frozen head in a canister of liquid nitrogen that needs constant topping up.

Once future software has become advanced enough, this data could then be used to make [a crude "copy"](https://www.lesswrong.com/posts/EKnDXLxqLAtkkix4w/personal-imitation-software) of my mind; and hey presto, back from the dead.  
It's not much, but better than the absolute certainty of oblivion, which is the alternative of doing nothing.

I'm not the only individual this question applies to.  
Many users of this site already have huge collections of digital data, which will become MUCH larger before they die.   
Some might not want it all to disappear after they're gone.
# One is (almost) normal in base π

See previous posts on my blog if you’re confused:

**Part 1:** [Normal numbers in base ten](https://adam.scherlis.com/2022/06/30/non-unique-decimals-0-999-and-normal-numbers-in-base-ten/)  
**Part 2:** [Numbers in base φ and π](https://adam.scherlis.com/2022/06/30/numbers-in-base-%cf%86-%cf%80-and-so-on/)  
**Part 3: One is normal in base π**

I'm only posting Part 3 (the punchline) here on LW; Parts 1 and 2 explain things in much more detail.

One is a [normal number](https://en.wikipedia.org/wiki/Normal_number).

“No it’s not.”

In base π, I mean.

“No, it’s still just **1.**“

Don’t be so greedy. Start with **0.** and go from there.

“So it’s **0.222…** or **0.333…** or something?”

Nope, it’s **0.3011021110…**

“That’s… bad. Does it repeat?”

I think it’s normal, but that seems hard to prove.

“So each digit occurs equally often?”

No, normal numbers in base π are about a 37%-30%-29%-4% split of 0, 1, 2, 3.

“Where did *that* distribution come from?”

Integrals of this weird function:

[![](https://ascherlis.files.wordpress.com/2022/06/image-2.png?w=595)](https://ascherlis.files.wordpress.com/2022/06/image-2.png)

(Specifically, integrals over \[0, 1/π\], \[1/π, 2/π\], \[2/π, 3/π\], and \[3/π, 1\].)

“And where did *that* come from?”

It’s the distribution of remainders when computing a random number in base π.

“What are the x coordinates of those discontinuities?”

Sequence of remainders when computing **0.3011021110…**

“um”

which are dense in \[0, 1\], by the way.

“Can you prove that?”

No, and neither can you.

“…And the y coordinates?”

The discontinuities get smaller by a factor of π each time.

“And almost all numbers have this distribution of remainders?”

Yup.

“Including 1, if you use the **0.3011021110…** representation.”

Yup.

“So the histogram of x-coordinates of the N largest discontinuities in this function approaches… this function itself, as N goes to infinity.”

Yup.

“Which has derivative zero almost everywhere, but has a dense set of discontinuities.”

Yup!

“Any other neat facts about it?”

It’s a fractal.

“How?”

Stretch out the colored regions by a factor of π horizontally, shrink by π vertically, and add them to each other. You get the original function back:

[![](https://ascherlis.files.wordpress.com/2022/06/image-3.png?w=595)](https://ascherlis.files.wordpress.com/2022/06/image-3.png)

(Some imagination required.)

"And this is all because π is normal, or transcendental, or something?"

Nope, I think analogous statements are all true for base 3.5.
# How to Navigate Evaluating Politicized Research?

I want to look into the research literature on an issue that has unfortunately become extremely politicized and controversial. (It isn't gun ownership, but let's say for the sake of argument that it is.) If I do a search for, say, "effects of gun ownership on crime", it is quite predictable that I will find a bunch of studies that say gun ownership decreases crime and a bunch of studies that say gun ownership increases crime. Some of those studies may be legitimate efforts at truthseeking, while others may be cherrypicked or manipulated, studies where the authors' bottom line has already been written before they do any work (to one degree or another), etc.  
  
This seems like a potentially difficult environment to evaluate things in compared to one with less of a charged atmosphere, and I'm curious how best to navigate it -- any recommendations for how to best do that?
# Open & Welcome Thread - July 2022

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend [reading the Library,](https://www.lesswrong.com/library) [checking recent Curated posts](https://www.lesswrong.com/?view=curated), [seeing if there are any meetups in your area](https://www.lesswrong.com/community), and checking out the [Getting Started](https://www.lesswrong.com/faq#Getting_Started) section of the [LessWrong FAQ](https://www.lesswrong.com/faq). If you want to orient to the content on the site, you can also check out the new [Concepts section](https://www.lesswrong.com/tags/all).

The Open Thread tag is [here](https://www.lesswrong.com/tag/open-threads?sortedBy=new). The Open Thread sequence is [here](https://www.lesswrong.com/s/yai5mppkuCHPQmzpN).
# Safetywashing

In southern California there’s a two-acre butterfly preserve owned by the oil company Chevron. They spend little to maintain it, but many millions on television advertisements featuring it as evidence of their environmental stewardship.[^9m6ozmbv1h]

Environmentalists have a word for behavior like this: *greenwashing*. Greenwashing is when companies misleadingly portray themselves, or their products, as more environmentally-friendly than they are.

Greenwashing often does cause real environmental benefit. Take the signs in hotels discouraging you from washing your towels:

![](https://lh4.googleusercontent.com/BlM3A53dzz5gNnV17V5InZoqxEq7YYvzwZjNMFzwN90T1EK_4J4x0ViGMRUqkC_B2Ub0tzxbdYowQzURTcpKqFbos0fYhMFhsgtt16P7mgc29UYMwYfReWHVfBpwwMG1Nqh3f8EPOVCNQE1qBw)

My guess is that the net environmental effect of these signs is in fact mildly positive. And while the most central examples of greenwashing involve deception, I’m sure some of these signs are put up by people who earnestly care. But I suspect hotels might tend to care less about water waste if utilities were less expensive, and that Chevron might care less about El Segundo Blue butterflies if environmental regulations were less expensive.

The field of AI alignment is growing rapidly. Each year it attracts more resources, more mindshare, more people trying to help. The more it grows, the more people will be incentivized to misleadingly portray themselves or their projects as more alignment-friendly than they are.

I think some of this is happening already. For example, a capabilities company launched recently with the aim of training transformers to use every API in the world, which they described as the “safest path to general intelligence.” As I understand it, their argument is that this helps with alignment because it involves collecting feedback about people’s preferences, and because humans often wish AI systems could more easily take actions in the physical world, which is easier once you know how to use all the APIs.[^19yz1jq2g5p]

It’s easier to avoid things that are easier to notice, and easier to notice things with good handles. So I propose adopting the handle “safetywashing.”

[^9m6ozmbv1h]: From what I can tell, the original source for this claim is the book “The Corporate Planet: Ecology and Politics in the Age of Globalization,” which from my samples seems about as pro-Chevron as you’d expect from the title. So I wouldn’t be stunned if the claim were misleading, though the numbers passed my sanity check, and I did confirm the preserve and advertisements exist. 

[^19yz1jq2g5p]: I haven’t talked with anyone who works at this company, and all I know about their plans is from the copy on their website. My guess is that their project harms, rather than helps, our ability to ensure AGI remains safe, but I might be missing something.
# Trends in GPU price-performance

Executive Summary
=================

Using a dataset of 470 models of graphics processing units (GPUs) released between 2006 and 2021, we find that the amount of floating-point operations/second per $ (hereafter FLOP/s per $) **doubles every ~2.5 years**. For top GPUs, we find a slower rate of improvement (FLOP/s per $ doubles every 2.95 years), while for models of GPU typically used in ML research, we find a faster rate of improvement (FLOP/s per $ doubles every 2.07 years). GPU price-performance improvements have generally been slightly slower than the 2-year doubling time associated with Moore’s law, much slower than what is implied by Huang’s law, yet considerably faster than was generally found in prior work on trends in GPU price-performance. Our work aims to provide a more precise characterization of GPU price-performance trends based on more or higher-quality data, that is more robust to justifiable changes in the analysis than previous investigations.

![](https://lh3.googleusercontent.com/OIJZGKNzuxviRPJF4ipQoNeIuWwWwiMxgwVPkC7Wl88saooRqwiVUNYQgKpjlLE9Fh7eHCQDinp3JT-yhKtZda2GfFq1nGyLc29rJfpWhmWVD8nOgeVUPR6HKzA8nFoa6_3qzFXnGD9ysC99tMo)

**Figure 1.** Plots of FLOP/s and FLOP/s per dollar for our dataset and relevant trends from the existing literature

| **Trend** | **2x time** | **10x time** | **Metric** |
| --- | --- | --- | --- |
| Our dataset (n=470) | 2.46 years \\[2.24, 2.72\\] | 8.17 years \\[7.45, 9.04\\] | FLOP/s per dollar |
| ML GPUs (n=26) | 2.07 years \\[1.54, 3.13\\] | 6.86 years \\[5.12, 10.39\\] | FLOP/s per dollar |
| Top GPUs (n=57) | 2.95 years \\[2.54, 3.52\\] | 9.81 years \\[8.45, 11.71\\] | FLOP/s per dollar |
| Our data FP16 (n=91) | 2.30 years \\[1.69, 3.62\\] | 7.64 years \\[5.60, 12.03\\] | FLOP/s per dollar |
| Moore’s law | 2 years | 6.64 years | FLOP/s |
| Huang’s law | 1.08 years | 3.58 years | FLOP/s |
| CPU historical ([AI Impacts, 2019](https://aiimpacts.org/trends-in-the-cost-of-computing/)) | 2.32 years | 7.7 years | FLOP/s per dollar |
| [Bergal, 2019](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/) | 4.4 years | 14.7 years | FLOPs/dollar |

**Table 1.** Summary of our findings on GPU price-performance trends and relevant trends in the existing literature with the 95% confidence intervals in square brackets.

In future work, we intend to build on this work to produce projections of GPU price-performance, and investigate how our findings inform us about the growth in dollar-spending on computing hardware in Machine Learning.

*We would like to thank Alyssa Vance, Ashwin Acharya, Jessica Taylor and the Epoch team for helpful feedback and comments.*
# Examples of practical implications of Judea Pearl's Causality work

I was recently talking with a friend about the practical usefulness of modern academic epistemology. Intuitively, I would have wanted to say that Judea Pearl's work causality has a lot of practical implications, however, I couldn't think of any examples.

Do you have examples of conclusions you have drawn because you learned about Pearl's causality that you wouldn't have drawn otherwise, or can point to other people making practical use of the concepts?
# [Linkpost] Existential Risk Analysis in Empirical Research Papers

I’ve noted before that it can be difficult to separate progress in safety from progress in capabilities. However, doing so is important, as we want to ensure that we are making differential progress on safety, rather than just advancing safety as a consequence of advancing capabilities. In particular, I think that research should rigorously evaluate trade-offs between improving safety and advancing capabilities.

This paper introduces the concept of “capabilities externalities,” and provides concrete steps for how to analyze the risk of different research papers. It also introduces “x-risk sheets,” which can be added to papers and help researchers discuss their impact on AI x-risk. Since discussion of AGI is becoming increasingly normalized in mainstream ML, I think careful discussions of x-risk are increasingly feasible within mainstream ML. The paper is largely aimed at mainstream ML researchers, and it is one step towards normalizing the discussion of existential risks.
# Five views of Bayes' Theorem

I think rearranging Bayes' Theorem sheds light on it in interesting ways.

P(A|B) = P(B|A) P(A)/P(B)
-------------------------

This is the most common form: you want to know the conditional probability of A given B, but what you actually know is the probability of B given A (and your priors on A and B). Bayes lets you swap the two events.

One way to think about this one: to swap around the conditional, multiply by a correction factor of P(A)/P(B) to change the "units" from "some sort of probability of B" to "some sort of probability of A". (This is literally a unit conversion if P(A) and P(B) are measures over two different spaces!)

P(A|B) P(B) = P(B|A) P(A)
-------------------------

These are two ways of writing the joint probability P(A, B), corresponding to two ways of sampling (A, B) sequentially: B and then A, or A and then B. The first thing you sample comes from your prior, and the second thing comes from a conditional distribution that depends on the first thing.

P(A|B)/P(B|A) = P(A)/P(B)
-------------------------

A lot of the time, people say things like "A is pretty likely given B, but B isn't quite as likely given A." Or: "many Bs are As, but not that many As are Bs." These are equivalent to saying "A is more likely than B, a priori" or "there are more As than Bs".

(I actually wrote this post because people kept saying the first thing around me and I had to think for a moment each time to realize it was the same as the second thing.)

P(A|B)/P(A) = P(B|A)/P(B)
-------------------------

Let's say both sides are equal to three. That means:

A is three times more likely than priors if B.

But also:

B is three times more likely than priors if A.

So if people in San Francisco are three times as likely to own crypto as the general population, then people who own crypto are three times as likely to live in SF.

This *almost* implies that "the relative risk of A given B is the relative risk of B given A", but "relative risk" is apparently defined as P(A|B)/P(A|¬B) rather than P(A|B)/P(A). Well, at least it's approximately true when A and B are both rare.

P(B) = P(A) P(B|A)/P(A|B)
-------------------------

This one's weird. I guess it's useful if you sampled B after A, but then realized you want to know the prior probability of B? And for some reason P(A|B) is what you have lying around?

Leave a comment if you know a cute intuition for this one!
# Literature recommendations July 2022

It happens frequently, that I'm researching a topic and don't really know which sources to read to get a good grasp of the topic. I expect that many fellow aspiring rationalists feel the same. 

This thread is a place to list topics for which you are looking for reading recommendations.

Please open one comment per topic that you want recommendations about. 

If you want to discuss the merits of this thread, please do it as a reply to the [meta-comment](https://www.lesswrong.com/posts/kPETbhurC2ynstaq6/literature-recommendations-july-2022?commentId=8piwef89CtNbYnzJa).
# Sexual self-acceptance

I have noticed very often that I can't predict how my brain reacts towards women. I often expect that nothing special will happen, compared to talking to males. However very often sexual thoughts come to mind that seems out of place to me. Normally they are very short (1-5s) and visual, but with very low detail.

This would happen most often when meeting women for the first time. Normally within the first minute of meeting them. Actually, this does not only happen in meetings but even when I watch a video with a woman (I mean a normal video, not pornography/erotica). Also, the frequency and strength of this are proportional to the time to last orgasm. Although that correlation is much weaker for me than one would expect. Though I am trying to maximize the time between orgasms. If I would not do this, there might be a stronger correlation. Certainly, if there are many orgasms not far apart (e.g. on the same day), there would be no sexual thoughts at all.

My normal reaction now for years was to feel embarrassed about my own thoughts. Interestingly, I always fail to anticipate my thoughts. Thus far I have seen the arising of sexual thoughts as a bad thing. A failure of self-control. Maybe when I manage to not have sexual thoughts for some time, my brain seems to think that now this problem is gone, and then it gets disappointed when thoughts come up again.

However recently I have managed to actually sort of just let it happen. Not only the imagination but also the suppression of it, without judging it at all. I sort of just let it happen and observed it. Right as I was starting to think sexual thoughts, I was thinking in a flash that this is a part of myself and that fighting it with negative emotions does not make sense. And then I managed to just observe it over the next 3 seconds before it disappeared, without any strong negative feelings. And it still disappeared. The negative feelings were not required to make it disappear!

Learning to accept/love yourself is old wisdom. I think I was applying it to various degrees before. But never to this explicitly. This might help me, to avoid unproductive negative feelings in the future. I wonder if this will continue to work.

I could imagine that this actually increases the frequency and intensity of sexual thoughts as it might make me not suppress them as hard. But even if this is the case, the psychological relief might still be worth it.
# SOMA - A story about Consciousness

SOMA has a great story. I like it so much that I made a story video compilation about it, containing only the best things. [The video](https://www.youtube.com/watch?v=Kp_Eroq5cOo) actually presents a coherent story, that somebody can enjoy without any prior knowledge, even though ~92.5% of the content has been cut.

I think it has one of the best "stories" of all video games I know. However, I think the gameplay is not that great. So I think watching the video that I made is actually a superior experience to playing the game yourself.

I recommend the long video, but if you don't want to spend much time on this I recommend just watching the [suit switch scene](https://www.youtube.com/watch?v=wQsKM1T3ugU), which is only 6 minutes, and then seeing if you like it. This is one of the best scenes if not the best. It is part of the longer video. There are more scenes of similar quality in the longer video.
# New US Senate Bill on X-Risk Mitigation [Linkpost]

Two US Senators have introduced a bipartisan bill specifically focused on x-risk mitigation, including from AI. From the [post on Senate.gov](https://www.hsgac.senate.gov/media/majority-media/peters-introduces-bipartisan-bill-to-ensure-federal-government-is-prepared-for-catastrophic-risks-) (bold mine):

> WASHINGTON, D.C.  – U.S. Senator Gary Peters (MI), Chairman of the Homeland Security and Governmental Affairs Committee, introduced a bipartisan bill **to ensure our nation is better prepared for high-consequence events, regardless of the low probability, such as new strains of disease, biotechnology accidents, or naturally occurring risks such as super volcanoes or solar flares that though unlikely, would be exceptionally lethal if they occurred.**
> 
> “Making sure our country is able to function during catastrophic events will improve national security, and help make sure people in Michigan and across the country who are affected by these incidents get the help they need from the federal government,” said Senator Peters. “Though these threats may be unlikely, they are also hard to foresee, and this bipartisan bill will help ensure our nation is prepared to address cataclysmic incidents before it’s too late.”
> 
> \[The legislation\] will establish an interagency committee for risk assessment that would report on the adequacy of continuity of operations (COOP) and continuity of government (COG) plans for the risks identified. **The bipartisan legislation would also help counter the risk of artificial intelligence (AI), and other emerging technologies from being abused in ways that may pose a catastrophic risk.**
> 
> \[...\]

It's interesting the term 'abused' was used with respect to AI. It makes me wonder if the authors have misalignment risks in mind at all or only misuse risks.

I haven't been able to locate the text of the bill yet. If someone finds it, please share in the comments.

[*Cross-posted to EA Forum*](https://forum.effectivealtruism.org/posts/fuGCxfmSYLJjbkodD/new-us-senate-bill-on-catastrophic-risk-mitigation-linkpost)*. Credit to Jacques Thibodeau for posting a link on Slack that made me aware of this.*