trentmkelly/LessWrong-43k huggingface dataset:
FTX expects to return all customer money; clawbacks may go away
by Mikhail Samin — 2024-02-14T03:43:13.218Z
Link: https://www.nytimes.com/2024/02/03/business/dealbook/what-happens-to-ftx-clawback-cases-if-the-company-repays-its-creditors.html?unlocked_article_code=1.VU0.QBLc.h36dKm_EgU8L&smid=url-share

New York Times (unpaywalled):

When the cryptocurrency exchange FTX declared bankruptcy about 15 months ago, it seemed few customers would recover much money or crypto from the platform. As John Ray III, who took over as chief executive during the bankruptcy, put it, “At the end of the day, we’re not going to be able to recover all the losses here.” He was countering Sam Bankman-Fried’s repeated claims that he could get every customer their money back.

Well, it turns out, FTX lawyers told a bankruptcy judge this week that they expected to pay creditors in full, though they said it was not a guarantee and had not yet revealed their strategy.

The surprise turn of events is raising serious questions about what happens next. Among them: What does this mean for the lawsuits FTX has filed in an attempt to claw back billions in assets that the company says it’s owed?

Will the possibility that customers could be made whole be raised at Bankman-Fried’s sentencing? Will potential relief for customers help his appeal?

[...]

Some of the clawback cases involve allegations of fraud, but not all do. Before fraud claims are argued, there is typically a legal fight over whether a company was insolvent at the time of the investment or that the investment led to insolvency. If every FTX creditor stands to get 100 cents on the dollar, the clawback cases that don’t involve fraud wouldn’t serve much of a financial purpose and may be more difficult to argue, some lawyers say. “In theory, clawbacks may go away there,” said Eric Monzo, a partner at Morris James who focuses on bankruptcy claims.

Court proceedings:

we can now cautiously predict some measure of success. Based on our results to date and current projections we anticipate filing a disclosure statement in February describing how customers and general unsecured creditors, customers and general unsecured creditors with allowed claims, will eventually be paid in full. I would like the Court and stakeholders to understand this not as a guarantee, but as an objective. There is still a great amount of work and risk between us and that result, but we believe the objective is within reach and we have a strategy to achieve it.
Scale Was All We Needed, At First
by Gabe M — 2024-02-14T01:49:16.184Z
Link: https://aiacumen.substack.com/p/scale-was-all-we-needed-at-first

This is a hasty speculative fiction vignette of one way I expect we might get AGI by January 2025 (within about one year of writing this). Like similar works by others, I expect most of the guesses herein to turn out incorrect. However, this was still useful for expanding my imagination about what could happen to enable very short timelines, and I hope it’s also useful to you.

The assistant opened the door, and I walked into Director Yarden’s austere office. For the Director of a major new federal institute, her working space was surprisingly devoid of possessions. But I suppose the DHS’s Superintelligence Defense Institute was only created last week.

“You’re Doctor Browning?” Yarden asked from her desk.

“Yes, Director,” I replied.

“Take a seat,” she said, gesturing. I complied as the lights flickered ominously. “Happy New Year, thanks for coming,” she said. “I called you in today to brief me on how the hell we got here, and to help me figure out what we should do next.”

“Happy New Year. Have you read my team’s Report?” I questioned.

“Yes,” she said, “and I found all 118 pages absolutely riveting. But I want to hear it from you straight, all together.”

“Well, okay,” I said. The Report was all I’d been thinking about lately, but it was quite a lot to go over all at once. “Where should I start?”

“Start at the beginning, last year in June, when this all started to get weird.”

“All right, Director,” I began, recalling the events of the past year. “June 2024 was when it really started to sink in, but the actual changes began a year ago in January. And the groundwork for all that had been paved for a few years before then. You see, with generative AI systems, which are a type of AI that—”

“Spare the lay explanations, doctor,” Yarden interrupted. “I have a PhD in machine learning from MIT.”

“Right. Anyway, it turned out that transformers were even more compute-efficient architectures than we originally thought they were. They were nearly the perfect model for representing and manipulating information; it’s just that we didn’t have the right learning algorithms yet. Last January, that changed when QStar-2 began to work. Causal language model pretraining was already plenty successful for imbuing a lot of general world knowledge in models, a lot of raw cognitive power. But that power lacked a focus to truly steer it, and we had been toying around with a bunch of trillion-paramet...
CFAR Takeaways: Andrew Critch
by Raemon — 2024-02-14T01:37:03.931Z

I'm trying to build my own art of rationality training, and I've started talking to various CFAR instructors about their experiences – things that might be important for me to know but which hadn't been written up nicely before.

This is a quick write up of a conversation with Andrew Critch about his takeaways. (I took rough notes, and then roughly cleaned them up for this. Some of my phrasings might not exactly match his intended meaning, although I've tried to separate out places where I'm guessing what he meant from places where I'm repeating his words as best I can)

Note that Andrew Critch is one particular member of the early CFAR team. My sense is each CFAR founder had their own taste on "what's most important". Treat this as one stone in a mosaic of takeaways.

"What surprised you most during your time at CFAR?

Surprise 1: People are profoundly non-numerate.

And, people who are not profoundly non-numerate still fail to connect numbers to life.

I'm still trying to find a way to teach people to apply numbers for their life. For example: "This thing is annoying you. How many minutes is it annoying you today? how many days will it annoy you?". I compulsively do this. There aren't things lying around in my life that bother me because I always notice and deal with it.

People are very scale-insensitive. Common loci of scale-insensitivity include jobs, relationship, personal hygiene habits, eating habits, and private things people do in their private homes for thousands of hours.

I thought it'd be easy to use numbers to not suck.

Surprise 2: People don't realize they need to get over things.

There was a unit a CFAR called 'goal factoring'. Early in it's development, the instructor would say to their class: "if you're doing something continuously, fill out a 2x2 matrix", where you ask: 1) does this bother me? (yes or not), and 2) is it a problem? (yes or no).

"Some things will bother you and not be a problem. This unit is not for that."

The thing that surprised me, was telling the instructor: "C'mon instructor. It's not necessary to spell out that people just need to accept some things and get over them. People know that, it's not worth spending the minute on it."

At the next class, the instructor asked the class: "When something bothers you, do you ask if you need to get over it?". 10% of people raised their hand. People didn't know the "realize that some ...
Meetup In a Box: Year In Review
by Czynski — 2024-02-14T01:18:28.259Z

Summary: Three rounds of questions, where the meetup thinks about their past year in a few frames of reference, then discusses their insights.

Tags: Small, Medium, One-Off, Experiment

Purpose: Year in Review is something more useful than New Year's Resolutions - more like a team retrospective you might do at work, but personal. Taking a chance to take stock.

Materials: Have enough paper, pens, and tables or clipboards for everyone to have their own, and enough space for all or most to sit with their paper out of other's direct sight. Smartphones and laptops are also possible but assume they won't be around. Take time to consider the lists of questions ahead of time; you may want to vary them. Have a pomodoro timer which can track 10 minutes and 20 minutes.

A whiteboard or other big wall-hung paper where you can write the questions in everyone's view is not necessary but very helpful. A shared Google doc with the questions is a good substitute or addition.

Announcement: We'll spend some time looking at our past year and asking questions of ourselves. What did you mean your priorities to be, last year, and what did it look like they were? What did you start doing, and stop doing, and do you endorse those choices? If you were to 80/20 the year, what was the 20% and what was the 80%? Then we'll talk some about our answers and any questions they raise.

Description: Describe the format to the participants:

We're going to consider a few sets of questions, three at a time. I'll share the questions [and write them on the whiteboard]/[and the doc containing them], and then everyone will spend ten minutes considering them and noting down their thoughts. Then we'll spend twenty minutes discussing our answers and insights with each other, before moving on to the next set. Don't worry if there's nothing you have to say or are willing to share, and don't push anyone if they want to stay silent.

Check for understanding and that everyone has whatever writing tools they need, then read the first set of three questions. If you have somewhere to write these questions in common view, do that now - in my case I wrote abbreviated versions of the questions[1], since it's a reminder more than a description. Pause before starting the timer, in case anyone's confused what they mean; after any necessary clarifications, start the timer for 10m. Be fairly strict about this timer unless everyone ...
An EA used deceptive messaging to advance her project; we need mechanisms to avoid deontologically dubious plans
by Mikhail Samin — 2024-02-13T23:15:08.079Z

TL;DR: good-hearted EAs lack the mechanisms to not output information that can mislead people. Holly Elmore organized a protest, with the messaging centered on OpenAI changing their documents to "work with the Pentagon," while OpenAI only collaborates with DARPA on open-source cybersecurity tools and is in talks with the Pentagon about veteran suicide prevention. Many participants of the protest weren’t aware of this; the protest announcement and the press release did not mention this. People were misled into thinking OpenAI is working on military applications of AI. OpenAI still prohibits the use of their services to "harm people, develop weapons, for communications surveillance, or to injure others or destroy property". If OpenAI wanted to have a contract with the Pentagon to work on something bad, they wouldn't have needed to

change the usage policies

of their publicly available services and could've simply provided any services through separate agreements. Holly was warned in advance that messaging could be misleading in this way, but she didn't change it. I think this was deceptive, and some protest participants agreed with me. The community should notice a failure mode and implement something that would prevent unilateral decisions with bad consequences or noticeable violations of deontology.

See a comment from Holly in a footnote[1] and a post she published before I published this post. In both, she continued to ignore the core of what was deceptive- not the "charter" mistake, but the claim that OpenAI was starting to work with the Pentagon, without providing important context of the nature of the work. She further made false statements in the comments, which is easily demonstratable with the messages she sent me; I'm happy to share those on request[2], DM or email me.

The original version of this post didn't mention Holly's name and was trying to point at a failure mode I wanted the community to fix. But since Holly commented on the post and it's clear who it is about, and the situation hasn't improved, this became needlessly making the post more confusing, so I edited the post, removing earlier redactions of the name.

Correction: the post previously asserted[3] that a co-organiser of the protest is a high-status member of the EA community; people told me this might be misleading, as many in the community disagree with their strateg...
Useful starting code for interpretability
by eggsyntax — 2024-02-13T23:13:47.940Z

Want to try your hand at neural network interpretability? A very nice way to get started is to find an existing Python notebook using one or more interpretability techniques, hopefully one written with beginners in mind. In a click or two you can make a copy of it, which you can typically run without any modification, and then start tweaking it to look at what you're interested in.

Fortunately, many such notebooks already exist, thanks to helpful members of the interp community! This post is just a list of those, mostly Colab notebooks. Many of them I have no personal experience with, but all of them have been recommended by people who know what they're doing. This list will probably be acceptably current through late 2024 or so; after that you should use a more up-to-date resource if one exists (although if one existed now I would have used it instead of writing one, so there may or may not be another one then).

Suggestions for other similarly useful starter notebooks for other areas are extremely welcomed!

Notebooks for understanding machine learning (as background): Transformers From Scratch, some other ML technique notebooks, reinforcement learning.

And the main list is in no particular order, so no need to go top to bottom.

@Neel Nanda's exploratory analysis demo for TransformerLens walks you through many of the basic mech interp techniques, and is highly recommended, and he has others as well.
Another intro to mech interp from ARENA, along with several other excellent notebooks reproducing some important mech interp results:

Indirect Object Identification
Superposition & Sparse Autoencoders
Function Vectors & Model Steering
OthelloGPT
Balanced Bracket Classifier
Grokking and Modular Arithmetic

Two activation steering notebooks, based on "Steering GPT-2-XL by adding an activation vector" (bonus: several different implementations from @Annah) (extra bonus: quick and dirty representation engineering on Mistral)
Developmental interpretability and singular learning theory notebooks, from @Jesse Hoogland.
A smallish notebook on using the tuned lens technique (successor to the logit lens).
Mech interp on Mamba using nnsight.

Thanks to @Jesse Hoogland and @CallumMcDougall for extremely useful input!
Masterpiece
by Richard_Ngo — 2024-02-13T23:10:35.376Z
Link: https://www.narrativeark.xyz/p/masterpiece

A sequel to qntm's

Lena

. Reading Lena first is helpful but not necessary.

We’re excited to announce the fourth annual MMindscaping competition! Over the last few years, interest in the art of mindscaping has continued to grow rapidly. We expect this year’s competition to be our biggest yet, and we’ve expanded the prize pool to match. The theme for the competition is “Weird and Wonderful”—we want your wackiest ideas and most off-the-wall creations!

Competition rules

As in previous competitions, the starting point is a base MMAcevedo mind upload. All entries must consist of a single modified version of MMAcevedo, along with a written or recorded description of the sequence of transformations or edits which produced it. For more guidance on which mind-editing techniques can be used, see the Technique section below.

Your entry must have been created in the last 12 months, and cannot have been previously submitted to any competition or showcase. Submissions will be given preliminary ratings by a team of volunteers, with finalists judged by our expert panel:

Roger Keating, mindscaping pioneer and founder of the MMindscaping competition.
Raj Sutramana, who has risen to prominence as one of the most exciting and avant-garde mindscaping artists, most notably with his piece Screaming Man.
Kelly Wilde, director of the American Digital Liberties Union.

All entries must be received no later than 11.59PM UTC, March 6, 2057.

Award criteria

Our judges have been instructed to look for technique, novelty, and artistry. More detail on what we mean by each of these:

Technique. Mindscaping is still a young art, and there are plenty of open technical challenges. These range from the classic problem of stable emotional engineering, to recent frontiers of targeted memory editing, to more speculative work on consciousness funnels. Be ambitious! Previous winners of our technique prize have often pushed the boundaries of what was believed possible.

Even when an effect could be achieved using an existing technique, though, submissions that achieve the same outcome in more efficient or elegant ways will score highly on the technique metric. Conversely, we’ll penalize brute-force approaches—as a rough guide, running a few thousand reinforcement learning episodes is acceptable, but running millions isn’t. We also discourage approaches that involve overwriting aspects of MMAcevedo’s psyche with data f...
A Bridge Between Utilitarianism & Stoicism
by Jonathan Moregård — 2024-02-13T22:46:06.388Z
Link: https://honestliving.substack.com/p/a-bridge-from-utilitarianism-to-stoicism

I have been a practising Stoic for over 10 years. I adopted Stoicism before I encountered the sequences and LessWrong-style rationalism. To me, Stoicism and rationalism serve complementary purposes, tackling different aspects of life.

Rationalism, to me, is related to getting a better understanding of the world, and how the mind functions. It’s generally descriptive, attempting to make models that accurately describe aspects of existence.

Stoicism, to me, is all about the nitty-gritty aspects of self-improvement. It’s more about instruction than explanation, featuring perspectives and explanations that are conducive to embodying a set of character traits commonly referred to as “virtues”.

This post will start by analysing utilitarianism, the most common ethical approach among rationalists, followed by a pragmatic integration of utilitarianism, Stoicism, and rationality.

But first, a bit of background.

Stoicism vs Utilitarianism

Stoicism is a type of Virtue Ethics, focused on the workings of your cognition — an action is “good” if it develops a set of desirable character traits (virtues). A bad action, on the contrary, is an action that builds bad habits (vices). In this view, committing genocide is bad because you don’t want to train yourself to be a person who commits genocide.

Consequentialism is all about the consequences of your actions — an action is “good” if its positive consequences outweigh its negative consequences. An action is bad if the downsides outweigh the upsides. In this view, committing genocide is bad because you cause a lot of suffering to a lot of people, robbing their agency by killing them, and causing a bunch of negative secondary effects.

At first glance, these are two very different ways of looking at ethics. Stoicism is sometimes presented as incompatible with utilitarianism. Let’s illustrate this gap through a story I heard from a friend. My friend attended an ethics course at university, and told me about his professor’s view of virtue ethics (paraphrased):

“Utilitarianism is about looking at the result of actions and judging the outcome. Virtue ethics is about acting like a ‘good person’, whatever that means!”

Arguing for the superiority of one view over the other misses the point. Why argue about “who is right”, when you can aim for an integration, getting the upsides from both views? Let’s move on to a pragmatic integration!

The

...
The "context window" analogy for human minds
by Ruby — 2024-02-13T19:29:10.387Z

Years ago I already modeled (and wrote about) that emotions are mediated by attention. You have emotions about what's on your mind (or recently on your mind) rather than having emotions constantly about everything that you technically know and believe.
Recently, I've gotten a good deal of mileage from porting over the concept of "context window" to my own mind. It's kind of the same idea, but somehow makes use of more intuitions.
I see my current emotions, and more than that–my current evaluations and leanings on questions of what to believe and do (of course, emotions inform these) to be quite dependent on what's loaded into my context window.
A useful intuition is that my context window is only so large, and for sufficiently large and complex topics, I'm not capable of loading everything into my context at once (at least not without chunking, external memory at aids). This means there are now classes of decision that previously I might have attempted to resolve while walking somewhere, showering, falling asleep, that I'll now refrain from making any conclusion on until I've sat down with exobrain, in good circumstances, and tried to load all considerations into my context window in a short space of time.
I've found that some known techniques achieve this, namely anything "inner parts" related. Internal Double Crux, or just "writing each perspective back and forth on an issue" seems to help, at least in part via the load-it-all-into-context mechanism.
A different principle that I get motivation for downstream of this model is that taking my time with complex decisions. "Sleeping on it." This helps because it allows for different info to naturally cycle through my context window. (Especially relevant is that your present emotional state is part of the context window that feeds into beliefs, and that varies over time.)
The context window analogy assists with holding emotions as object and not necessarily strong evidence about reality, as I see strong emotions often being the result of just the current context window contents, and sure enough, as my attention shifts, so do emotions. (Which doesn't mean the emotions are informative, just not the whole picture.)
Some things are simple enough that they're easy to load into memory at once and just make a quick(ish) decision. Before I was using the context window analogy, I had briefly updated against making any big...