trentmkelly/LessWrong-43k huggingface dataset:
Parasitic Language Games: maintaining ambiguity to hide conflict while burning the commons
by Hazard — 2023-03-12T05:25:26.496Z

“They are playing a game. They are playing at not playing a game. If I show them I see they are, I shall break the rules and they will punish me. I must play their game, of not seeing I see the game”
- R. D. Laing

"It's not lying if everyone knows it's lying."

I see this sentiment in a lot of places. It pops up in

corporate managerial contexts

. It's been used as a

legal defense

and worked. It's a claim that communication that looks adversarial isn't, it's just high-context communication between people "in the know", there's no deception happening, no conflict, you just don't get how we do things here.

I don't buy it. My claim in a nutshell:

It situations where people insist "it's not lying because everyone knows it's lying" the people in the know aren't deceiving each other, but the reason this game is being played is to fool people not in the know, and insisting that it's just "high context communication" is part of an effort to obscure the fact that a conflict is going on.

If that makes perfect sense to you, dope, you already get my main point. The rest of this post is adding nuance, actually arguing the case, and providing more language for talking about these sorts of dynamics.

Case Study: "Are Founders Allowed to Lie?"

This

essay by Alex Danco talks about how "it's not lying because everybody knows it's lying" works in the Silicon Valley startup scene. It's short enough that it's worth reading now so you can decide for yourself if I'm misrepresenting him. If you don't feel like reading it I still quote enough of it for my post to make sense.

Some snippets.

It's really hard to start a business without lying:

If you are only allowed to tell the literal, complete truth, and you’re compelled to tell that truth at all times, it is very difficult to create something out of nothing. You probably don’t call it “lying”, but founders have to will an unlikely future into existence. To build confidence in everyone around you – investors, customers, employees, partners – sometimes you have to paint a picture of how unstoppable you are, or how your duct tape and Mechanical Turk tech stack is scaling beautifully, or tell a few “pre-truths” about your progress. Hey, it will be true, we’re almost there, let’s just say it’s done, it will be soon enough.

It's not lying because everyone's in on it.

You’re not misleading investors; your investors get it: they’re optimizing for authentic

...
Is "Regularity" another Phlogiston?
by Cole Wyeth — 2023-03-12T03:13:44.646Z
Link: https://colewyeth.com/regularity.html

People have a tendency to say meaningless things, often without realizing it. An extreme example is to attribute a poorly understood phenomena like lightning to "magic" or "God," which is really just a fancy way of saying "I don't know why this thing happened." Unfortunately, it's possible to use non-explanations by accident, while thoroughly convinced that you are offering a real explanation. I fell prey to this for a long time, attempting to explain intelligence/consciousness in terms of "emergence," until Eliezer Yudkowsky changed my mind here. Who knows how much time I might have wasted studying cellular automata or some such interesting but irrelevant (to A.I.) topic if I hadn't read that when I did. In the hopes of not repeating my mistake, I try to stay on the lookout for meaningless explanations.

I believe I have found one: "Regularity." This occurs in a few forms. I will describe them in order of increasing uselessness. Beware that I am using this post to jot down some rough thoughts, so I have neither attempted to make the post understandable to a wide "non-technical" audience, nor worked out the technical details fully for every point.

The first is regularization. Machine learning engineers (and sometimes statisticians/data scientists) discuss regularizing their models to avoid overfitting. This basically means that a very squiggly line can go through any number of points exactly, whereas a relatively smooth line cannot. He who can explain everything can explain nothing (this is essentially a no free lunch theorem from statistical learning theory) so the simplest hypothesis class containing the true "data generating process" (or a reasonable approximation) should be chosen. That is one formalization of the idea. However, it is often used to justify doing things like weight decay, or in its simplest form ridge regression/lasso. In the case of ordinary least squares (essentially the linear regression model most of us learned in high school) this penalizes large weights to prevent the resulting hyperplane from having a very large slope in any direction. Since features can be nonlinear combinations of data entries this does correspond intuitively to making the line through some points "smoother." But ridge regression actually has a rigorous justification without any hand-waving about regularization: it corresponds to the MLE assuming data are generated ...
Minor Life Optimization: Consider Ordering Your Food To-Go
by sudo — 2023-03-12T02:08:54.348Z

At some restaurants, you will be asked after ordering if you’d like to have your meal “for here” or “to go.”

As far as I can tell, the only thing that changes based on your response is how your food is served.

If it is “for here,” it’ll be served on a plate with metal utensils. If it is “to go,” it’ll be wrapped and served in a plastic or paper bag.

Personally, I don’t care much about how my food is presented. However, I have occasionally thrown out food or ate more than I’d like because I couldn’t take my food with me. I could’ve avoided this by asking for my meals “to go,” and my meals would’ve been just as appealing and nutritious.

Here are some pros and cons of ordering your meal “to go.”

Pros:

You can order more food without worrying about waste.

You will have snacks for later, saving money on food delivery.

You won’t have to waste food if you run out of time to eat. Instead, you can save your meal for later.

Cons:

The food isn’t presented as well.

Plastic cutlery might be harder to use than metal cutlery.

It might be healthier to avoid plastic cutlery and food packaging.

I estimate that ordering my meals “to go” buys me 10-20 minutes a week, and also saves me around $20 per week. Consider making it your default!
A bunch of videos for intuition building (2x speed, skip ones that bore you)
by the gears to ascension — 2023-03-12T00:51:39.406Z

did I mention 2x speed? hit play on many, pause on almost as many.

This is a big list of the youtube videos I find myself linking to people most often. These are all from the same playlist: https://www.youtube.com/playlist?list=PLgx5WuezywJMj_JS47QIqcn8_3UiiEwPs - comment here if you'd like edit access, I give it out readily. I'd love to have people moving the most important and insightful videos towards the beginning. I'd also love to see people clone the playlist and just make their own version.

These vary wildly in topic and difficulty level. I generally do not try to avoid watching things above my level, I just use it as inspiration for how to fill in what I'm missing. If something sounds basic to you, it probably is.

Many of these videos are quite short, many are quite long.

1min: neuron clip

23min: neuroscience overview (bio neuron interpretation)

or 10min with 2x speed!

10min: overview of learning techniques (bit clickbaity title but I include because I like it plenty anyhow)

or 5min with 2x speed!

2min: visual intuition - details of how one particular chaotic fluid flow move

11min: research talk on what collective intelligence is. (see also many more cool talks from MITCBMM!)

or 5min with 2x speed!

2min: visualization of a volume of neurons in a (mouse?) amygdala

8min: cognitive biases in practice

33min: absolutely incredible visual intro to physics sims focusing towards fluid simulation

or 15min with 2x speed!

15min: cs101 "ok, but what does it mean to abstract over the matter of a computer"

or 7min with 2x speed!

1min: visualization of particle lenia

20min: overview of Michael Levin's research on the bioelectric communication of cells for morphogenesis and morphogenic editing without genetic change

or 10min with 2x speed

11min: cs101 how a neural network is actually just line segments (with relu, anyway)

12min: nice intro to what chaos theory is actually about

18min: overview of ways visual proofs can mislead

4min: overview of some important additional notes on how to learn efficiently. this playlist does not satisfy them all.

14min: Visual intro to why neural networks work. goes into detail about the geometric interpretation of neural networks.

15min: geometric interpretation of bayes' rule. Useful for intuition building even if you get the math.

See also chris olah's blog post on the same topic from a few years prior.

4min: visualization of atoms that better communicates wh

...
The issue of meaning in large language models (LLMs) 
by Bill Benzon — 2023-03-11T23:00:28.009Z

Cross posted from New Savanna.

Noam Chomsky was trotted out to write a dubious op-ed in The New York Times about large language models and Scott Aaronson registered his displeasure at his blog, Shtetl-Optimized: The False Promise of Chomskyism. A vigorous and sometimes-to-often insightful conversation ensued. I wrote four longish comments (so far). I’m reproducing two of them below, which are about meaning in LLMs.

Meaning in LLMs (there isn’t any)
Comment #120 March 10th, 2023 at 2:26 pm

@Scott #85: Ah, that’s a relief. So:

But I think the importantly questions now shift, to ones like: how, exactly does gradient descent on next-token prediction manage to converge on computational circuits that encode generative grammar, so well that GPT essentially never makes a grammatical error?

It's not clear to me whether or not that’s important to linguistics generally, but it is certainly important for deep learning. My guess – and that’s all it is – is that if more people get working on the question, that we can make good progress on answering it. It’s even possible that in, say five years or so, people will no longer be saying LLMs are inscrutable black boxes. I’m not saying that we’ll fully understand what’s going on; only that we will understand a lot more than we do know and are confident of making continuing progress.

Why do I believe that? I sense a stirring in the Force.

There’s that crazy ass discussion at LessWrong that Eric Saund mentioned in #113. I mean, I wish that place weren’t so darned insular and insisting on doing everything themselves, but it is what it is. I don’t know whether you’ve seen Stephen Wolfram’s long article (and accompanying video) but has some nice visualizations of the trajectory GP-2 takes in completing sentences and is thinking in terms of complex dynamic – he talks of “attractors” and “attractor basins” – and seems to be thinking of getting into it himself. I found a recent dissertation in Spain that’s about the need to interpret ANNs in terms of complex dynamics, which includes a review of an older literature on the subject. I think that’s going to be part of the story.

And a strange story it is. There is a very good reason why some people say that LLMs aren’t dealing with meaning despite that fact that they produce fluent prose on all kinds of subjects. If they aren’t dealing with meaning, then how can they produce the prose?

The fa...
[Linkpost] Scott Alexander reacts to OpenAI's latest post
by Orpheus16 — 2023-03-11T22:24:39.394Z
Link: https://astralcodexten.substack.com/p/openais-planning-for-agi-and-beyond

Scott Alexander recently wrote a post about OpenAI's Planning for AGI and beyond. I found it thoughtful, and I think others here might want to read or discuss it.

Some highlights:

ExxonMobil analogy

Imagine ExxonMobil releases a statement on climate change. It’s a great statement! They talk about how preventing climate change is their core value. They say that they’ve talked to all the world’s top environmental activists at length, listened to what they had to say, and plan to follow exactly the path they recommend. So (they promise) in the future, when climate change starts to be a real threat, they’ll do everything environmentalists want, in the most careful and responsible way possible. They even put in firm commitments that people can hold them to.

An environmentalist, reading this statement, might have thoughts like:

Wow, this is so nice, they didn’t have to do this.
I feel really heard right now!
They clearly did their homework, talked to leading environmentalists, and absorbed a lot of what they had to say. What a nice gesture!
And they used all the right phrases and hit all the right beats!
The commitments seem well thought out, and make this extra trustworthy.
But what’s this part about “in the future, when climate change starts to be a real threat”?
Is there really a single, easily-noticed point where climate change “becomes a threat”?
If so, are we sure that point is still in the future?
Even if it is, shouldn’t we start being careful now?
Are they just going to keep doing normal oil company stuff until that point?
Do they feel bad about having done normal oil company stuff for decades? They don’t seem to be saying anything about that.
What possible world-model leads to not feeling bad about doing normal oil company stuff in the past, not planning to stop doing normal oil company stuff in the present, but also planning to do an amazing job getting everything right at some indefinite point in the future?
Are they maybe just lying?
Even if they’re trying to be honest, will their bottom line bias them towards waiting for some final apocalyptic proof that “now climate change is a crisis”, of a sort that will never happen, so they don’t have to stop pumping oil?

This is how I feel about OpenAI’s new statement, Planning For AGI And Beyond.

Doomer argument: Acceleration burns time

Recent AIs have tried lying to, blackmailing, threatening, and seducing users. AI compani

...
Compositional language for hypotheses about computations
by Vanessa Kosoy — 2023-03-11T19:43:40.064Z

TLDR: I sketch a language for describing (either Bayesian or infra-Bayesian) hypotheses about computations (i.e. computable[1] logical/analytic facts). This is achieved via a novel (AFAIK) branch of automata theory, which generalizes both ordinary and tree automata by recasting them in category-theoretic language. The resulting theory can be equipped with a intriguing self-referential feature: automata that "build" and run new automata in "runtime".

Epistemic status: this is an "extended shortform" i.e. the explanations are terse, the math is a sketch (in particular the proofs are not spelled out) and there might be errors (but probably not fatal errors). I made it a top-level post because it seemed worth reporting and was too long for a shortform.

Motivation

Finite-state automata are a convenient tool for constructing hypotheses classes that admit efficient learning. A classical example is MDPs in reinforcement learning. Ordinary automata accept strings in input. This is convenient when the input is a time sequence. However, in Turing Reinforcement Learning and infra-Bayesian physicalism we want to equip our agent with beliefs about computations, which don't form a time sequence in the ordinary sense.

Computations are often represented as trees: e.g. syntax trees of lambda calculus, or any other functional programming language, or expressions defining recursive functions. Hence, the obvious guess is using tree automata. Specifically, we will focus on bottom-up tree automata which are more powerful and more convenient for other reasons as well.

One limitation of this is that hypotheses described using (infra-)probabilistic tree automata are missing correlations between computations. Even if two programs contain a literally identical subroutine, or a single program uses multiple copies of an identical subroutine, the uncertainty is about the different copies is uncorrelated.

Another limitation is that sophisticated automata might require exponentially many states, meaning that a straightforward (tabular) description of the transition rules is exponentially large. Instead, we would like a convenient compositional language for constructing such automata (analogously[2] to the compositional language I proposed for MDPs).

Therefore, a generalization is called for.

Deterministic Operadic Automata

Bottom-up deterministic tree automata are algebras over the operad of tre...
Understanding and controlling a maze-solving policy network
by TurnTrout — 2023-03-11T18:59:56.223Z

Previously: Predictions for shard theory mechanistic interpretability results

Locally retargeting the search by modifying a single activation. We found a residual channel halfway through a maze-solving network. When we set one of the channel activations to +5.5, the agent often navigates to the maze location (shown above in red) implied by that positive activation. This allows limited on-the-fly redirection of the net's goals.
(The red dot is not part of the image observed by the network, it just represents the modified activation. Also, this GIF is selected to look cool. Our simple technique often works, but it isn't effortless, and some dot locations are harder to steer towards.)

TL;DR: We algebraically modified the net's runtime goals without finetuning. We also found (what we think is) a "motivational API" deep in the network. We used the API to retarget the agent.

Summary of a few of the most interesting results:

Langosco et al. trained a range of maze-solving nets. We decided to analyze one which we thought would be interesting. The network we chose has 3.5M parameters and 15 convolutional layers.

This network can be attracted to a target location nearby in the maze—all this by modifying a single activation, out of tens of thousands. This works reliably when the target location is in the upper-right, and not as reliably when the target is elsewhere.
Considering several channels halfway through the network, we hypothesized that their activations mainly depend on the location of the cheese.

We tested this by resampling these activations with those from another random maze (as in causal scrubbing). We found that as long as the second maze had its cheese located at the same coordinates, the network’s behavior was roughly unchanged. However, if the second maze had cheese at different coordinates, the agent's behavior was significantly affected.
This suggests that these channels are inputs to goal-oriented circuits, and these channels affect those circuits basically by passing messages about where the cheese is.

Our statistical analysis suggests that the network decides whether to acquire cheese not only as a function of path-distance to cheese, but—after controlling for path-distance—also as a function of Euclidean/"perceptual" distance between the mouse and the cheese, even though the agent sees the whole maze at once.
Another simple idea: We define a

...
How to Support Someone Who is Struggling
by David Zeller — 2023-03-11T18:52:25.060Z

[Crossposted from my blog]

There’s no shortage of pain, tragedy and loss in the world.

And if you’re anything like me

, you don’t always know how to be helpful when a loved one is going through the worst of it.

Over the past few years, I’ve been trying to get better at that.

I’ve read a couple dozen therapy textbooks, I’ve done four hundred or so hours of client-centered counselling, and I’ve been in a handful of other official and unofficial helping roles. By no means am I an expert, but I sure know more than I used to.

For my first blog post, I wanted to write something that past-me might have found helpful when he started stumbling through it all. In time, there’s so much more that I want to say on the art of supporting others. But for now...

Here are four fundamentals for helping someone who’s having a rough time:

1 - Simply listen. It helps far more than most of us expect.

When a catastrophe happens, it can change the whole landscape of one’s world. The tectonic plates shift, things break, and everything comes to look bewilderingly different to how it did before.

In the aftermath, we may have no good choice other than to stop, watch the buildings fall, and slowly map out this strange new world we’re in. Perhaps only then we can move forward.

Unfortunately, processing such big changes purely in one’s own head is… hard. Thoughts are ephemeral and it’s easy to think in circles, to get stuck, to have blind spots, to ruminate.

This is where listening comes in. A good listener can be of much help with that working through process. Patiently, the listener can keep track of where a conversation is getting stuck, gently bring up the things that are being avoided or missed, help bring attention towards what is most important, and bring a genuine sense of connection that makes all the bad stuff a little easier to bear.

As simple as it seems, having someone there to just listen may be exactly what the person in front of you needs.

2 - Rather than focusing on the bright side, sit with the other person’s real feelings.

This next point comes straight from Brené Brown. I’ve been shown the

same video

of her so many times in different training courses that I’m starting to get Stockholm syndrome. All the same, what it says is important.

Often when we’re trying to support another person, we try to get them to focus on the bright side. Standing separately from the other’s experience, we a...
Given one AI, why not more?
by Frank Adk — 2023-03-11T18:52:09.292Z

Suppose there is a threshold of capability beyond which an AI may pose a non-negligible existential risk to humans.

What is the argument against this reasoning: If one AI passes or seems likely to pass this threshold, then humans, to lower x-risk, ought to push other AI past this threshold in light of the following.

1) If only one AI passes this threshold and it works to end humanity either directly or indirectly, humanity has zero chance of survival. If there are other AIs, there is a non-zero chance that they support humanity directly or indirectly, and thus humanity's chance of survival is above zero.

2) Even if, at some point, there is only one AI past this threshold and it presents as aligned, the possibilities of change and deception argue for more AIs to be brought over the threshold, see 1).

3) The game board is already played to an advanced state. If one AI passes the threshold, the social and economic costs of preventing other AIs from making the remaining leap seem very unlikely to result in a net positive return. Thus pushing a second, third, hundredth AI over the threshold would have a higher potential benefit/cost ratio.

Less precisely, if all it takes is one AI to kill us, what are the odds that all it takes is one AI to save us?
I can think of all sorts of entropic/microstate (and not hopeful) answers to that last question, and counterarguments for all of what I said, but what is the standard response?
Links appreciated. I'm sure this has been addressed before; I looked; I can't find what I'm looking for.