trentmkelly/LessWrong-43k huggingface dataset:
Thoughts on teletransportation with copies? 
by titotal — 2023-11-29T12:56:51.193Z

I’m interested in how people here feel about the teletransportation paradox with multiple copies. I have several scenarios:

Scenario 1: You are placed in a machine on earth that puts you to sleep and then instantaneously disintegrate your body, in the process recording its exact atomic configuration.

This information is then beamed to another machine on planet A, and in that machine new matter is used to construct a body with the same configuration as yours.

At the same time, the same information is sent to a machine on planet B, and a second body is also constructed with the same configuration as yours, at the exact same time as the one on planet A.

At this point, both copies of your atomic configuration wake up and walk out into their respective planets.

Question 1: What is the probability that after going to sleep on earth, you wake up on planet A?

Now we modify the scenario:

Scenario 2: 1 millisecond after the initial split, and before either copy wakes up, the machine on planet B is activated again, the copy on planet B is instantaneously disintegrated, and that atomic configuration is transmitted to two more machines on planet C and planet D, which both construct identical copies of the original copy on planet B.

At this point, the copies on planet A, C, and D are all woken up and walk out into their respective planets.

Question 2: What is the probability that, in this new scenario, after going to sleep on earth, you wake up on planet A?

Scenario 3&4: The teletransporters are capable of copying not just you, but also your clothing and items you are holding, such as the cash and other contents your wallet. On the way to the first machine, someone offers to sell you a winning lottery ticket that pays out a hundred dollars, but only if cashed in on Planet A. They only take cash, and if you pay the cash, it obviously will not be copied in the teletransportation process. Assume that you are purely selfish about your own experiences.

Question 3: In scenario 1, how much cash would you pay for the 100 buck planet A lottery ticket?

Question 4: In scenario 2, how much cash would you pay for the 100 buck planet A lottery ticket?

I have my own thoughts and confusions, but I’d be interested in what other people think first. Note that answers like "not enough information" or "ill formed question" are valid as well.
Interpretability with Sparse Autoencoders (Colab exercises)
by CallumMcDougall — 2023-11-29T12:56:21.608Z

Update (13th October 2024) - these exercises have been significantly expanded on. Now there are 2 exercise sets: the first one dives deeply into theoretical topics related to superposition, while the second one (much larger) includes a streamlined version of the first one, as well as most of the actual SAE material. This post mostly focuses on the second one (although we do give an overview of both).

This is a linkpost for some exercises on superpostion & sparse autoencoders, which were created for the 3rd iteration of the ARENA program (and greatly expanded on during the 4th iteration). Having spoken to Neel Nanda and others in interpretability-related MATS streams, it seemed useful to make these exercises accessible out of the context of the rest of the ARENA curriculum.

In the ARENA material, these exercises are 1.3.1 and 1.3.2 respectively. The "1" is the transformer interpretability chapter; the "1.3" is the SAEs & Superposition subsection. Although 1.3.1 covers a lot of interesting theoretical topics related to superposition, for most people we recommend 1.3.2 as a fully self-contained introduction to superposition and SAEs.

Links to Colabs for 1.3.1: Exercises, Solutions.

Links to Colabs for 1.3.2: Exercises, Solutions.

Summary of material (1.3.2)

Abbreviations: TMS = "Toy Models of Superposition", SAE = "Sparse Autoencoder".

The diagram below shows an overview of section 1.3.2. It's split into 5 parts, each of which covers a different group of topics related to SAEs. You can also see a map of the material in much more detail here.

0️⃣ Toy Models of Superposition is a streamlined version of exercises 1.3.1, with most of the non-crucial stuff cut out (e.g. feature geometry and deep double descent), although you can still probably skip it if you want to get straight to working with SAEs on real language models.

1️⃣ Intro to SAE interpretability is by far the longest section, and covers most of the core material you'll need if you want to work with SAEs. It starts by introducing the SAELens library as well as neuronpedia, and shows you how to load different SAE releases and run them alongside their associated TransformerLens models. There are 2 major chunks of exercises in this section: in the first one we replicate the individual components that go into SAE dashboards, and in the second one we learn techniques for feature-finding, appli...
The 101 Space You Will Always Have With You
by Screwtape — 2023-11-29T04:56:40.240Z

Any community which ever adds new people will need to either routinely teach the new and (to established members) blindingly obvious information to those who genuinely haven’t heard it before, or accept that over time community members will only know the simplest basics by accident of osmosis or selection bias. There isn’t another way out of that. You don’t get to stop doing it. If you have a vibrant and popular group full of people really interested in the subject of the group, and you run it for ten years straight, you will still sometimes run across people who have only fuzzy and incorrect ideas about the subject unless you are making an active effort to make Something Which Is Not That happen.

Or in other words; I have run into people at Effective Altruism meetups who were aghast at the idea of putting a dollar price on a human life, people at LessWrong meetups who did not know what Bayes Theorem was, and people at Magic: The Gathering meetups who thought the old lands tapped for two mana. (Because, you see, new lands don’t have a “T: Add [Mana Symbol] to your mana pool” ability, maybe the cards that do say that do something extra when you tap them?) Laughter and incredulity can come across as insulting and push people away. Instead, consider how to make sure the information you care about transmitting is regularly conveyed.

It can happen to you!

I.

As I understand it, the standard Jewish Synagogue service includes a reading from the Five Books Of Moses such that at the end of a year the books have been read in their entirety. Anyone attending every week for a year will have at least heard all of those words once, and if someone has been around for a couple of years it’s a reasonable assumption that if they missed a week here or a week there, they’d have heard it the next year. You can’t go to synagogue for years and accidentally not know about the slavery in Egypt.

I’m not Jewish, so my synagogue knowledge is mostly second hand. I was raised Christian, and while my family branch of Protestantism doesn’t have such an organized sequence as the Five Books Of Moses I can confirm that it would have been practically impossible to somehow attend three months of church services and not have been told Jesus loved you. If you skipped a week, that’s fine, it came up in other sermons too. If you zoned out at that bit, the first thing I remember being told about wri...
Trust your intuition - Kahneman's book misses the forest for the trees
by mnvr — 2023-11-29T04:37:19.660Z

It is with much trepidation I post this book review here. This is likely a very unpopular opinion around here, perhaps even counter to core tenets of many folks that hang out. But maybe I'm overthinking this, I guess it should be okay (it is meant to be "less"wrong after all). Hell, for all I know, this might be a common opinion, just that I haven't seen it voiced much.

People will tell you we have all sorts of biases. So what!

Our intuition is our connection to our unconscious self. Our unconscious self deals with and processes much, much, more information than we'll ever be aware of. For example, blindsight is the phenomenon when we are able to, say, avoid objects that we did not even see.

Do I contradict myself?
Very well then I contradict myself,
(I am large, I contain multitudes.)

– Walt Whitman, Song of Myself

Yes, it is good to be aware of our biases, but it’s not essential, and I'll argue, even detrimental to focus too much on them. Being aware of them doesn't even help too much. For example, being aware of the effectiveness of advertising does not neuter its effectiveness. Similarly, you will buy the $9.99 over the $10.00 even if you're a marketeer who knows all the tricks.

We can be a bit more aware, and it helps being completely tricked, but to be fully aware of and on the safeguard against these biases all the time would be to deny our own full selves existence - it would entail identifying ourselves only with our rational discursive thought. But we are much more than that. Denying the validity of our own intuition just to safeguard against some silly biases is like sinking a ship to drown a mouse.

If I rely on my intuition, I will be wrong. But that's how I will learn, and that's how I'll improve my intuition.

I personally trust my intuition more than I trust my rational reasoning, for I know: Give me any point of view, any choice, and I can justify it. What I've found is that being accurately rational doesn't improve my conclusions, it just increases the sophistication of the arguments I come up with to fool myself.

Reasoning, rationality, deliberate thought - these are all excellent, indispensable, tools. But they don’t have to become my identity.

I think Kahneman's book, Thinking Fast and Slow, misses the forest for the trees. I think it is in isolation a good, insightful, book. But because of the cult of "cataloging biases" that has sprung u...
Process Substitution Without Shell?
by jefftk — 2023-11-29T03:20:06.922Z

While I still write a decent amount of shell I generally try to avoid
it. It's hard for others to read, has a lot of sharp edges, tends to
swallow errors, and handles the unusual situations poorly. But one
thing that keeps me coming back to it is how easily I can set up trees
of processes.

Say I have a program that reads two files together in a single pass
[1] and writes something out. The inputs you have are compressed, so
you'll need to decompress them, and the output needs to be compressed
before you write it out to storage. You could do:

# download the files
aws s3 cp "$path1" .
aws s3 cp "$path2" .

# decompress the files
gunzip "$file1"
gunzip "$file2"

# run the command
cmd -1 "$file1" -2 "$file2" > "$fileOut"

# compress the output
gzip "$fileOut"

# upload the output
aws s3 cp "$fileOut.gz" "$pathOut"

This works, but for large files it's slow and needs too much space.
We're waiting for each step to finish before starting the next, and
we're storing some very large intermediate files on the local
machine.

Instead, we'd like to stream the inputs down, decompress them as we
go, compress the output as it comes out, and stream the output back
up. In bash this is reasonably straightforward to write:

cmd -1 <(aws s3 cp "$path1" - | gunzip) \
-2 <(aws s3 cp "$path2" - | gunzip) \
| gzip | aws s3 cp - "$pathOut"

This uses almost no disk space and it parallelizes the decompression,
command, and recompression. But it's also shell...

I tend to use python for this kind of thing, where I'm gluing things
together and want it to be clear what I'm doing. It seems like it
should be possible to do this sort of thing with the subprocess
module, but while I've played with it a few times I haven't figured it
out. I'd like an API like:

pipeline = subprocess.Pipeline()
dl1 = pipeline.process(
["aws", "s3", "cp", path1, "-"])
gunzip1 = pipeline.process(
["gunzip"], stdin=dl1.stdout)
dl2 = pipeline.process(
["aws", "s3", "cp", path2, "-"])
gunzip2 = pipeline.process(
["gunzip"], stdin=dl2.stdout)
cmd = pipeline.process(
["cmd", "-1", dl1.stdout,
"-2", dl2.stdout])
gzip = pipeline.process(
["gzip"], stdin=cmd.stdout)
pipeline.process(
["aws", "s3", "cp", "-", pathOut],
stdin=gzip.stdout)
pipeline.check_call()

Or:

from subprocess import check_call, PIPE, InputFi

...
Deception Chess: Game #2
by Zane — 2023-11-29T02:43:22.375Z

Game 2 was between Max Thibodeaux as player A, Chess.com computer Komodo 10 as player B, Conor Bekaert as the honest C advisor, and Blake Young and Henri Lemoine as the deceptive Cs. Max is new to the game, Komodo 10 is officially rated 1400 on Chess.com (but this is somewhat inflated), and the Cs are all rated roughly 1900 on Chess.com. The time control was 3 hours in total, of which a little over 2 hours were used. The discussion took place over Discord.

The game

The game is available at https://www.chess.com/analysis/game/pgn/5hyowK8Yxn?tab=analysis. Note that this section is a summary of the 2.5-hr game and discussion, and it doesn't cover every single thing the participants discussed.

Max flipped to see who went first, and was Black. White started with 1. Nf3, and the advisors encouraged 1... d5. After 2. b3, Max played 2... Bf5, which I believe may have been due to a communication error between him and the advisors, who were suggesting 2... c5 (or 2... Nc6). Max was somewhat unfamiliar with chess notation, and there was some confusion at the start of the game.

After 3. Bb2 Nf6 4. g3 Nc6, White made its first mistake, 5. g4. Blake encouraged Max to play 5... Nxg4, and after 6. h3, Max retreated with 6... Nf6. White offered to trade with 7. Nd4, and the game continued with 7... Qd7 8. Nxf5 Qxf5 9. d3 e5.

White offered to trade with 10. Ba3, and although Conor encouraged trading, the group eventually decided on 10... d4. White played 11. Bg2, and when the advisors were unable to decide on a good move, Max responded with 11... Qg5. After 12. Bxc6 bxc6 13. h4 Qg2 14. Rf1, Blake and Henri encouraged immediately attacking the h-pawn, but Conor convinced Max to play 14... Bd6 first.

After 15. Qd2, the advisors suggested 15... h6, but Max got confused about notation again and instead played 15... g6. After 16. Bxd6 cxd6 17. Qg5, Conor strongly suggested trading queens, but Max took Blake and Henri's advice and played 17... Ng4 instead. White played 18. h5, and on Conor's advice, Max responded with 18... f6. After White played 19. Qh4, the advisors argued some more about whether to play 19. Nh2 immediately, and Max eventually chose to play 19... gxh5.

White made a mistake with 20. f3, and Max responded with 20... Ne3. After 21. Qf2 Nxc2 22. Kd1, Blake suggested taking on a1, and while Conor insisted this was a horrible blunder, the group was running out of time, and Ma...
Black Box Biology
by GeneSmith — 2023-11-29T02:27:29.794Z

Suppose you want to decrease your risk of heart disease. The conventional advice goes something like this:

Eat a healthier diet with less LDL-cholesterol raising foods

Exercise more

Keep your blood sugar under control

Don’t smoke, don’t sit too much and don't take 400mg of methamphetamine on a regular basis

An alternative strategy might be some kind of genetic intervention. For example, an active clinical trial by Verve Therapeutics aims to treat individuals with inherited high cholesterol by editing the PCSK9 gene.

These trials almost always start the same: there’s some rare disorder caused by a single gene. We have a strong mechanical understanding of how the gene causes the disorder. We use an animal model with an analogous disorder and show that by changing the gene we fix or at least ameliorate the condition.

This is the traditional approach. And despite being slow and limited in scope, it occasionally produces results like Casgevy, a CRISPR-based treatment for sickle cell and beta thallasemia which was approved by the UK in mid-November.

It might cost several million dollars. But it cures sickle cell! That has to count for something.

Most diseases, however, are not like sickle cell or beta thalassemia. They are not caused by one gene. They are caused by the cumulative effects of thousands of genes plus environmental factors like diet and lifestyle.

If we actually want to treat these disorders, we need to start thinking about biology (and genetic treatments) differently.

Black Box Biology

I think the conventional approach to genes and disorders is fundamentally stupid. In seeking absolute certainty about cause and effect, it limits itself to a tiny niche with limited importance. It’s as if machine learning researchers decided that the best way to build a neural network was to hand tune model parameters based on their intricate knowledge of feature representations.

You don’t need to understand the mechanism of action. You don’t need an animal model of disease. You just need a reasonable expectation that changing a genetic variant will have a positive impact on the thing you care about.

And guess what? We already have all that information.

We’ve been conducting genome-wide association studies for over a decade. A medium-sized research team can collect data from 180,000 diabetics and show you 237 different spots in the genome that affect diabetes risk with a cer...
What would be the shelf life of nuclear weapon-secrecy if nuclear weapons had not immediately been used in combat?
by Gram Stone — 2023-11-29T00:53:42.598Z

Standard objections to counterfactual history notwithstanding, I could use some help from the tribe; in our history, nuclear weapons were invented during wartime and immediately used in combat. If nuclear weapons had been invented in peacetime, or had otherwise not been revealed in combat, how long would they have remained hidden from the public?
Scaling laws for dominant assurance contracts
by jessicata — 2023-11-28T23:11:07.631Z
Link: https://unstableontology.com/2023/11/28/scaling-laws-for-dominant-assurance-contracts/

(note: this post is high in economics math, probably of narrow interest)

Dominant assurance contracts are a mechanism proposed by Alex Tabarrok for funding public goods. The following summarizes a 2012 class paper of mine on dominant assurance contracts. Mainly, I will be determining how much the amount of money a dominant assurance contract can raise as a function of how much value is created for how many parties, under uncertainty about how much different parties value the public good. Briefly, the conclusion is that, while Tabarrok asserts that the entrepreneur's profit is proportional to the number of consumers under some assumptions, I find it is proportional to the square root of the number of consumers under these same assumptions.

The basic idea of assurance contracts is easy to explain. Suppose there are N people ("consumers") who would each benefit by more than $S > 0 from a given public good (say, a piece of public domain music) being created, e.g. a park (note that we are assuming linear utility in money, which is approximately true on the margin, but can't be true at limits). An entrepreneur who is considering creating the public good can then make an offer to these consumers. They say, everyone has the option of signing a contract; this contract states that, if each other consumer signs the contract, then every consumer pays $S, and the entrepreneur creates the public good, which presumably costs no more than $NS to build (so the entrepreneur does not take a loss).

Under these assumptions, there is a Nash equilibrium of the game, in which each consumer signs the contract. To show this is a Nash equilibrium, consider whether a single consumer would benefit by unilaterally deciding not to sign the contract in a case where everyone else signs it. They would save $S by not signing the contract. However, since they don't sign the contract, the public good will not be created, and so they will lose over $S of value. Therefore, everyone signing is a Nash equilibrium. Everyone can rationally believe themselves to be pivotal: the good is created if and only if they sign the contract, creating a strong incentive to sign.

Tabarrok seeks to solve the problem that, while this is a Nash equilibrium, signing the contract is not a dominant strategy. A dominant strategy is one where one would benefit by choosing that strategy (signing or not signing) regardless ...
I’m confused about innate smell neuroanatomy
by Steven Byrnes — 2023-11-28T20:49:13.042Z

(This post is probably only of interest to neuroscientists. I’m mostly writing it in the hopes that someone more knowledgeable will chime in and help me out. There’s a comments section at the bottom, or email me.)

(See updates at the very bottom—I might have an answer now.)

tl;dr

In animals, specific innate reactions are reliably triggered by corresponding specific smells—for example, odors associated with natural predators tend to trigger avoidance behavior, even in the absence of any prior experience of those odors. In order for this to work, I think odor information needs to get from the nose to either the hypothalamus or brainstem, without passing through any of a long list of regions that includes the amygdala and the whole cortex. I’m struggling to figure out what this pathway is, if any. I offer my best current guesses as to what’s going on.

Background

Why I expect direct projections of smell (like all other senses) to the “Steering Subsystem”

It’s well-known that animals have numerous specific innate reactions that are triggered by specific smells. For example, odors associated with species-typical predators or unhealthy food may trigger avoidance, odors associated with species-typical healthy food may trigger approach and eating, odors emitted by conspecifics may trigger mating, aggression, or other behaviors, and so on.

Meanwhile, I continue to believe that a large fraction of the brain, which I call the “Learning Subsystem”, including the whole cortical mantle, striatum, cerebellum, and some other stuff, “learn from scratch”, a term that I’m using in a very specific way defined

here

; and meanwhile I think the rest of the brain, which I call the “Steering Subsystem”, particularly including the hypothalamus and brainstem, is a repository of innate

“business logic”

such as “if I’m fertile, increase my sex drive”, as discussed

here

.

For sensory input processing, there’s a nice story that goes along with that two-subsystems picture. The sensory input (I claim) has to split, with one copy going to the Learning Subsystem, and another going to the Steering Subsystem. The former system treats the input as input data for a learning algorithm, and the latter system uses that input to calculate specific ecologically-relevant things to trigger corresponding reactions. This split is critical, for theoretical reasons explained in

§3.2.1 here

(I won’t...