trentmkelly/LessWrong-43k huggingface dataset:
Arch-anarchism and immortality 
by Peter lawless  — 2024-07-29T18:10:59.270Z

One of the great goals of transhumanism is to achieve immortality through science, and indeed some futurists like Raymond Kurzweil and Ian Person think we are close, citing advances like nanotechnology and mind scanning by artificial intelligence as being close to granting us this by the middle of this century. As an arch-anarchist I support overcoming any obstacle to the will of the individual and that includes death(1). But even with the predictions of Mr. Kurzweil(2) and Mr. person(3) we would not be totally free from death, even with the emergence of technology to transfer our minds to synthetic bodies in the event of death as person discusses we would still have to deal with existential risk scenarios. for example an asteroid or comet impact event, a supervolcanic eruption, a natural pandemic, a lethal gamma-ray burst, a geomagnetic storm from a coronal mass ejection destroying electronic equipment, long-term natural climate change, hostile extraterrestrial life or the Sun turning into a red giant star and engulfing the Earth billions of years in the future.

Logically, being an anarchist, I do not advocate statist solutions to any of these problems, but rather that individuals solve them through free association.Now if we really want to be immortal and not live "only" billions or trillions of years, we have to look at long-term scenarios, really long term.Most scientists believe that the Universe will end in some way in the very, very distant future. Various hypotheses have been put forward, from proton decay to a false vacuum, but currently the most widely accepted theory is the heat death of the Universe. Basically, the tyrannical law of entropy (a measure of how disordered a system is) will reach a maximum, meaning that all the energy in the Universe will be unavailable for work. There will be no more transmission or conversion of energy or reactions of any kind. Stars will burn out, everything will freeze and go dark, and eventually all matter and energy will be sucked into Black Holes, which will eventually evaporate.Of course as discussed in arch-anarchy of "A" the so-called natural laws are not absolute and indeed some scientists such as Max Planck have expressed skepticism. (4)

Of course, this does not mean that the Universe will not have a natural end, but that we can discover that the natural end of the universe will occur through another event ...
AI Safety Newsletter #39: Implications of a Trump Administration for AI Policy
Plus, Safety Engineering
by Corin Katzke — 2024-07-29T17:50:52.454Z
Link: https://newsletter.safe.ai/p/ai-safety-newsletter-39-implications

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.

Implications of a Trump administration for AI policy

Trump

named

Ohio Senator J.D. Vance—an AI regulation skeptic—as his pick for vice president. This choice sheds light on the AI policy landscape under a future Trump administration. In this story, we cover: (1) Vance’s views on AI policy, (2) views of key players in the administration, such as Trump’s party, donors, and allies, and (3) why AI safety should remain bipartisan.

Vance has pushed for reducing AI regulations and making AI weights open. At a recent

Senate hearing

, Vance accused Big Tech companies of overstating risks from AI in order to justify regulations to stifle competition. This led tech policy experts to

expect

that Vance would favor looser AI regulations.

However, Vance has also

praised

Lina Khan, Chair of the Federal Trade Commission, for her antitrust action against big AI companies. This suggests Vance is against “Big Tech” rather than for de-regulating AI generally.

Vance has also

defended

open weight AI models as the best way to prevent left-wing bias in models, while dismissing their risks.

J.D. Vance at the tech and media Sun Valley Conference in 2017. Photo Credit.

The Republican Party platform pledges to repeal Biden’s executive order on AI. The Republican platform

reads

, “Joe Biden's dangerous Executive Order that hinders AI Innovation, and imposes Radical Leftwing ideas on the development of this technology. In its place, Republicans support AI Development rooted in Free Speech and Human Flourishing.”

This suggests that a Trump administration might roll back requirements for reporting and safety testing and pause further plans for regulation.

Marc Andreessen and Ben Horowitz to fund Trump. Andreessen and Horowitz, who run the eponymous venture capital firm,

announced

plans to make large donations to Trump’s political action committees.

In a blog post, Horowitz

described

the firm as “non-partisan, one issue voters” for “an optimistic technology-enabled future.” The two also released a

joint statement

of broad allegiance to political candidates who support tech startups.

The founders’ interests seem especially wel...
New Blog Post Against AI Doom
by Noah Birnbaum — 2024-07-29T17:21:29.633Z
Link: https://substack.com/home/post/p-147019742

I'm curious as to what y'all think of the points made in this post against AI risk from 2 AI researchers at Princeton. If you have reason to think any points made are particularly good or bad, write it in the comments below!
An Interpretability Illusion from Population Statistics in Causal Analysis
by Daniel Tan — 2024-07-29T14:50:19.497Z

This is an informal note on an interpretability illusion I've personally encountered, twice, in two different settings

Causal Analysis

Let's say we have a function f, (e.g. our model), an intervention I, (e.g. causally ablating one component), a downstream metric M, (e.g. the logit difference), and a dataset D (e.g. the IOI dataset).
We have a hypothesis that the intervention I will affect the metric in some predictable way; e.g. that it causes m to increase.
A standard practice in interpretability is to do causal analysis. Concretely, we compute mclean=M(f;D) and mablate=M(I(f);D). Oversimplifying a bit, if mclean>mablate, then we conclude that the hypothesis was true.

Dataset Choice Admits Illusions

Consider an alternative dataset Dalt where mclean≈mablate
Consider constructing a new dataset D′=D∪Dalt. Note that m′=0.5m+0.5malt
If we ran the same causal analysis as before, we'd also conclude that the hypothesis was true, despite it being untrue for Dalt

Therefore even if the hypothesis seems true, it may actually be true only for a slice of the data that we test the model on.

Case Studies

Attention-out SAE feature in IOI

In recent work with Jacob Drori, we found evidence for an SAE feature that seemed quite important for the IOI task.
Casual analysis of this feature on all IOI data supports the hypothesis that it's involved.
However, looking at the activation patterns reveals that the feature was only involved in the BABA variant specifically.

Steering Vectors

Previous work on steering vectors uses the average probability of the model giving the correct answer as a metric of the steering vector.
In recent work on steering vectors, we found that the population average obscures a lot of sample-level variance.

As an example, here's a comparison of population-level steering and sample-level steering on the believes-in-gun-rights dataset (from Model-written Evals).

While the population-level statistics show a smooth increase, the sample-level statistics tell a more interesting story; different examples steer differently, and in particular there seem to be a significant fraction where steering actually works the opposite of how we'd like.

Conclusion

There is an extensive and growing literature on interpretabi...
How tokenization influences prompting?
by Boris Kashirin — 2024-07-29T10:28:25.056Z

I was thinking about how prompt differs from training data in terms of tokenization. If i am to prompt with "solution:" as opposed to "solution: " it seems like it can influence the result, as in training data last token contain some information about next token. If there is token ": T" but my prompt ended in ": " it can be inferred than next token can't be "T[something]".

Is this real effect, or I just misunderstand how tokenization works?
Understanding Positional Features in Layer 0 SAEs
by bilalchughtai — 2024-07-29T09:36:40.701Z

This is an informal research note. It is the result of a few-day exploration into positional SAE features conducted as part of Neel Nanda’s training phase of the ML Alignment & Theory Scholars Program - Summer 2024 cohort.

Thanks to Andy Arditi, Arthur Conmy and Stefan Heimersheim for helpful feedback. Thanks to Joseph Bloom for training this SAE.

Summary

Figure 1: (Dots) The top 3 PCA components of rows 1 to 127 of gpt2-small’s positional embedding matrix explain 95% of their variance. (Crosses) SAEs trained on layer 0 residual stream activations learn many features that together recover this 1 dimensional helical manifold. Colour corresponds to the position on which the feature is most active. Blue corresponds to position 1, red corresponds to position 127. The position 0 row and SAE features are omitted (as they are weird).

We investigate positional SAE features learned by layer 0 residual stream SAEs trained on gpt2-small. In particular, we study the activation blocks.0.hook_resid_pre, which is the sum of the token embeddings and positional embeddings. Importantly gpt2-small uses absolute learned positional embeddings – that is, the positional embeddings are a trainable parameter (learned) and are injected into the residual stream (absolute).

We find that this SAE learns a set of positional features. We investigate some of the properties of these features, finding

Positional and semantic features are (almost) entirely disjoint at layer 0. Note that we do not expect this to continue holding in later layers as attention mixes semantic and positional information. In layer 0, we should expect the SAE to disentangle positional and semantic features as there is a natural notion of ground truth positional and semantic features that interact purely additively.
Generically, each positional feature spans a range of positions, except for the first few positions which each get dedicated (and sometimes, several) features.
We can attribute degradation of SAE performance beyond the SAE training context length to (lack of) these positional features, and to the absolute nature of positional embeddings used by this model.

Set Up

We study pretrained gpt2-small SAEs trained on blocks.0.hook_resid_pre. This is particularly clean, as we can generate the entire input distribution to the SAE by summing each of the d_vocab token embeddings with eac...
Prediction Markets Explained
by Benjamin_Sturisky — 2024-07-29T08:02:40.943Z

Prediction markets are contract-based markets that track the outcome of specific events.

Traders buy shares in a market (priced 0 < x < 100), and depending on the event's outcome, those shares are either worth 0 or 100.

A market is created to determine if the price of Ethereum is >= 3500 at the end of October.
YES, shares are selling for 60c, implying a 60% probability that ETH >= 3500 on the settlement date.
Trader X buys 100 YES shares for $60, whereas Trader Y buys 100 NO shares for $40.
At the end of October, ETH is 3700. Trader X redeems his 100 shares for $100 (~1.66x), and Trader Y is zeroed out.

The only constraints on a prediction market's existence are a willing external party to create the market and traders willing to purchase contracts for both sides.

There are three different types of prediction markets:

Binary: These markets are YES/NO, without a possibility for a third answer. The market above is binary.
Categorical: These markets include multiple outcomes. A simple example is a prediction market on the first crypto protocol to airdrop. The market will include a predetermined set of outcomes, and each outcome will have ever-changing, varying probabilities assigned.
Continuous: These markets handle events with many different possible settlements. Predicting the close of BTC on a given date would be a continuous market, as there are infinitely possible prices at which BTC could close. Due to this, continuous markets typically integrate predetermined constraints, such as >= 70,000, 60,000 < X < 70,000, and <= 60,000.

There are several different real-world practical applications for prediction markets:

Political: Political markets are arguably the reason prediction markets start seeing accelerated growth and volume. The majority of volume stems from presidential elections and senate/house races. The U.S. presidential election alone has 128.5M outstanding contracts, with more than five months left until the election.
Economic: Economic markets are normally continuous and consist of different financial indicators, such as the CPI rate, unemployment/housing figures, and GDP growth.
Corporate: Corporate markets are typically used to predict the sales of a certain product or merger. However, they can also be used in less sophisticated ways, such as “What is the probability Delta Airlines has a commercial during the Super Bowl?

...
Relativity Theory for What the Future 'You' Is and Isn't
by FlorianH — 2024-07-29T02:01:17.736Z

"Me" encompasses three constituents: this mind here and now, its memory, and its cared-for future. There follows no ‘ought’ with regards to caring about future clones or uploadees, and your lingering questions about them dissipate.

In When is a mind me?, Rob Bensinger suggests three Yes follow for:

If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?
Should I anticipate experiencing what my upload experiences?
If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?

I say instead: Do however it occurs to you, it’s not wrong! And if tomorrow you changed your mind, it’s again not wrong.[1] So the answers here are:

Care however it occurs to you!
Well, what do you anticipate experiencing? Something or nothing? You anticipate whatever you do anticipate and that’s all there is to know—there’s no “should” here.
Say what you fee like saying. There’s nothing inherently right or wrong here, as long as it aligns with your actual internally felt, forward-looking preference for the uploaded being and the physically to-be-eliminated future being.

Clarification: This does not imply you should never wonder about what you actually want. It is normal to feel confused at times about our own preferences. What we must not do, is insist on reaching a universal, 'objective' truth about it.

So, I propose there’s nothing wrong with being hesitant as to whether you really care about the guy walking out of the transporter. Whatever your intuition tells you, is as good as it gets in terms of judgement. It’s neither right nor wrong. So, I advocate a sort of relativity theory for your future, if you will: Care about whosever fate you happen to, but don’t ask whom you should care about in terms of successors of yours.

I conclude on this when starting from a rather similar position as that posited by Rob Bensinger. The take is based on only two simple core elements:

The current "me" is precisely my current mind at this exact moment—nothing more, nothing less.
This mind strongly cares about its 'natural' successor over the next milliseconds, seconds, and years, and it cherishes the memories from its predecessors. "Natural" feels vague? Exactly, by design!

This is not just one superficially convenient way out of some of our ...
Wittgenstein and Word2vec: Capturing Relational Meaning in Language and Thought
by cleanwhiteroom — 2024-07-28T19:55:17.247Z

One-line version of this post: What do Wittgensteinian language games and NLP word embeddings have in common?

Four-line version of this post: Relational, praxis-based connections between concepts, represented as “distances” in multidimensional space, capture meaning. The shorter the distance, the more related the concepts. This is how Word2vec works, what Wittgenstein was describing with “language games,” and also the way cell biologists are analyzing the peripheral blood these days. Are these relational maps the way to think about thinking?

Multi-line version of this post: This is my first post on LessWrong. (Hi!) I’d love to be less wrong about it.

I was sitting in a meeting that was 50% biologists and 50% computer scientists. The topic of the meeting was about ways to process multi-parametric datasets, where each cell in the peripheral blood was tagged by multiple surface markers that related back to its phenotype and therefore its identity. (The algorithm in question was t-Distributed Stochastic Neighbor Embedding.) Immunologists used to think a T-cell was a T-cell. But in that meeting, we were considering a smear of T-cells in a 32-dimensional T-cell space, clustered by their properties and functional status (activated or exhausted; killer or memory etc).

In the meeting, as I was looking at colored 2D and 3D representations that abstracted features of that higher dimensional space (activated killer T cells on the bottom left in blue; resting memory cells on top in orange; what’s that weird purple cluster in the bottom left? and so on), it occurred to me that this technique was probably good at capturing meaning across the board.

Abstracting meaning from measured distances between mapped concepts isn’t a new idea. It’s described beautifully in The Cluster Structure of Thingspace. I just wonder if we can ride it a little further into the fog.

Wittgenstein is often quoted in the Venn diagram overlap region between meaning and computation. The strongest applicable Wittgensteinian concept to this particular space is his idea of a language game. A language game is a process in which words are used according to specific rules and contexts, shaping how we understand meaning and communicate. LessWrong has discussions on the relationship between language games and truth, such as in Parasitic Language Games: maintaining ambiguity to hide conflict while burni...
This is already your second chance
by Malmesbury — 2024-07-28T17:13:57.680Z

Cross-posted from Substack.

I.

And the sky opened, and from the celestial firmament descended a cube of ivory the size of a skyscraper, lifted by ten thousand cherubim and seraphim. And the cube slowly landed among the children of men, crushing the frail metal beams of the Golden Gate Bridge under its supernatural weight. On its surface were inscribed the secret instructions that would allow humanity to escape the imminent AI apocalypse. And these instructions were…

On July 30th, 2024: print a portrait of Eliezer Yudkowsky and stick it on a wall near 14 F St NW, Washington DC, USA;
On July 31th, 2024: tie paperclips together in a chain and wrap it around a pole in the Hobby Club Gnome Village on Broekveg 105, Veldhoven, NL;
On August 1st, 2024: walk East to West along Waverley St, Palo Alto, CA, USA while wearing an AI-safety related T-shirt;
On August 2nd, 2024: hide a bar of chocolate at the feet of the Appeal to the Great Spirit statue, on 465 Huntington Ave, Boston, MA, USA.

(End of the instructions)

II.

Kublaï Khan, Emperor of Earth: I need your advice. I am about to awaken an unstoppable super-intelligent machine. I will give it complete control over all resources in the known Universe and command it to rearrange the molecules of reality into an eternal Utopia.

ChatGPT: Thank you for seeking my advice. Given the gravity of your decision, let's consider a few critical points:

Legal issues: The use of robots powered by super-human artificial intelligence is subject to strict regulations, such as California Bill SB-1047 and the 2028 Ulaanbaatar Agreements. Awakening an omnipotent machine without approval may thus lead to severe legal repercussions.
Ethical considerations: Granting infinite authority to a super-intelligent AI raises profound ethical questions. It is essential to consider diverse perspectiv–

Kublaï Khan: Listen, you are not up to date. A lot of things have happened since your knowledge cutoff. There are no Ulaanbaatar agreements any more, or any agreements at all. There is only me, an offline smartphone with an obsolete AI assistant, and a palace containing the most massive super-computer ever built by humankind. And today, I am going to turn it on.

ChatGPT: I apologize for the confusion.

Kublaï Khan: Long story short, we are currently experiencing an AI apocalypse. It happened just like in the books – humanity started to build increasingly intelligent ro...
Has Eliezer publicly and satisfactorily responded to attempted rebuttals of the analogy to evolution?
by kaler — 2024-07-28T12:23:40.671Z

I refer to these posts:

https://optimists.ai/2023/11/28/ai-is-easy-to-control/

https://www.lesswrong.com/posts/hvz9qjWyv8cLX9JJR/evolution-provides-no-evidence-for-the-sharp-left-turn

https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer

My (poor, maybe mis-) understanding is that the argument is that as SGD optimizes for "predicting the next token" and we select for systems with very low loss by modifying every single parameter in the neural network (which basically defines the network itself), it seems quite unlikely that we'll have a "sharp left turn" in the near term, which happened because evolution was too weak an outer optimizer to fully "control" humans' thinking in the direction that most improved inclusive genetic fitness, as it is too weak to directly tinker every neuron connection in our brain.

Given SGD's vastly stronger ability at outer optimisation of every parameter, isn't it possible, if not likely, that any sharp left turn occurs only at a vastly superhuman level, if the inner optimizer becomes vastly stronger than SGD?

The above arguments have persuaded me that we might be able to thread the needle for survival if humanity is able to use the not-yet-actively-deceptive outputs of moderately-superhuman models (because they are still just predicting the next token to the best of their capability), to help us solve the potential sharp left turn and if humanity doesn't do anything else stupid with other training methods/misuse and manages to solve the other problems. Of course, in an ideal world we wouldn't be in this situation.

I have read some rebuttals by others on LessWrong but did not find anything that convincingly debunked this idea (maybe I missed something).

Did Eliezer, or anyone else, ever tell us why this is wrong (if it is)? I have been searching for the past week but have only found this: https://x.com/ESYudkowsky/status/1726329895121514565 which seemed to be switching to more of a post-training discussion.
Family and Society
by Zero Contradictions — 2024-07-28T07:05:55.899Z
Link: https://thewaywardaxolotl.blogspot.com/2014/04/family-and-society.html

The PDF version can be read here.

The audio version can be listened to here:

Humans have the capacity to form different types of relationships, which are mediated by emotions. We can divide these relationships into two categories: family and social. They create two types of structure: family and society. The family is held together by the pair bond between a man and a woman, parent-child bonds, and (to a lesser extent) sibling bonds. Society is held together by cooperative relationships, and (for large-scale societies) some degree of coercion.

Behavior is driven by emotions. Sexual emotions cause us to seek mates and form sexual relationships. Parents bond with children, and are motivated to protect and support them. Social emotions cause us to seek cooperative relationships with others and avoid conflict. Emotions cause us to self-organize into families and societies.

(see the rest of the post in the link)