trentmkelly/LessWrong-43k huggingface dataset:
Epistemic Hell
by rogersbacon — 2024-01-27T17:13:09.578Z

my favorite painting of hell <3

I.

From

Scott Alexander’s review

of Joe Henrich’s The Secret of our Success:

In the Americas, where manioc was first domesticated, societies who have relied on bitter varieties for thousands of years show no evidence of chronic cyanide poisoning. In the Colombian Amazon, for example, indigenous Tukanoans use a multistep, multi-day processing technique that involves scraping, grating, and finally washing the roots in order to separate the fiber, starch, and liquid. Once separated, the liquid is boiled into a beverage, but the fiber and starch must then sit for two more days, when they can then be baked and eaten.

Such processing techniques are crucial for living in many parts of Amazonia, where other crops are difficult to cultivate and often unproductive. However, despite their utility, one person would have a difficult time figuring out the detoxification technique. Consider the situation from the point of view of the children and adolescents who are learning the techniques. They would have rarely, if ever, seen anyone get cyanide poisoning, because the techniques work. And even if the processing was ineffective, such that cases of goiter (swollen necks) or neurological problems were common, it would still be hard to recognize the link between these chronic health issues and eating manioc. Most people would have eaten manioc for years with no apparent effects. Low cyanogenic varieties are typically boiled, but boiling alone is insufficient to prevent the chronic conditions for bitter varieties. Boiling does, however, remove or reduce the bitter taste and prevent the acute symptoms (e.g., diarrhea, stomach troubles, and vomiting).

So, if one did the common-sense thing and just boiled the high-cyanogenic manioc, everything would seem fine. Since the multistep task of processing manioc is long, arduous, and boring, sticking with it is certainly non-intuitive. Tukanoan women spend about a quarter of their day detoxifying manioc, so this is a costly technique in the short term. Now consider what might result if a self-reliant Tukanoan mother decided to drop any seemingly unnecessary steps from the processing of her bitter manioc. She might critically examine the procedure handed down to her from earlier generations and conclude that the goal of the procedure is to remove the bitter taste. She might then experiment with alternative p

...
David Burns Thinks Psychotherapy Is a Learnable Skill. Git Gud.
by Morpheus — 2024-01-27T13:21:05.068Z
Link: https://podcast.clearerthinking.org/episode/192/david-burns-cognitive-behavioral-therapy-and-beyond/

Epistemic status: I listened to this, and I found these statements surprising in a “big if true” way. It didn't seem obviously false, but I am very much relying on him here. If someone thinks his claims are completely made up, please let me know in the comments.

TLDR

David Burns claims in this podcast episode, with deliberate practice, he and a few of his students got to the point to learn to cure patients in a single or just a few sessions. David Burns is known for popularizing CBT through his book “Feeling Good”.

The quote in the next section is most important. The rest of the post contains other quotes I found interesting. Some of them are pretty long. Feel free to skip things. The transcript was generated with whisper, so there might be mistakes. You can find the whole transcript here.

David Burns on Teaching CBT

David Burns: I've realized that probably of the 50,000 people I've trained, you know, with, I mean, trained, I'd say someone who spent at least two days in one of my full day, you know, two full days with me and learning cognitive therapy, the new TEAM-CBT. There's probably only been 10 or 15 who have been able to learn it. I've illustrated it on this program.

Spencer Greenberg: Wait only 10 or 15 of many thousands have been able to learn it?

David Burns: They have developed the same level of skill. It means, you know, a lot of people are doing it. There's a feeling, there's a feeling good institute in Mountain View, California. And they probably have 40 or 50 therapists associated with them. But there are wide ranges of skill levels and the ones like may or the ones when I work with people, I think you should be able to complete a course of treatment for depression in a single therapy session. It has to be two, it takes me two hours to do it, but it happens.

Spencer Greenberg: That's a pretty amazing claim, right, that you could treat someone in two hours.

David Burns: I mean, yeah, yeah. And when I was young, I would have thought it was impossible. And I used to dream about it and say, would there be some way to get really good at psychotherapy? And I said, we'd have to measure things and nobody's measuring anything because athletes, when they practice like for basketball, every time they throw the ball up, they can see whether or not it goes through the hoop. So they learn really fast, but we weren't measuring anything. And I did a research stud

...
Aligned AI is dual use technology
by lc — 2024-01-27T06:50:10.435Z

Humans are mostly selfish most of the time. Yes, many of us dislike hurting others, are reliable friends and trading partners, and care genuinely about those we have personal relationships with. Despite this, spontaneous strategic altruism towards strangers is extremely rare. The median American directs exactly $0 to global poverty interventions, and that is a true statement regardless of whether you limit it to the Americans that make ten, fifty, a hundred, or a thousand times as much money as Nigerians.

Some people hope that with enough tech development we will reach a "post-scarcity" regime where people have so much money that there is a global commons of resources people can access largely to their hearts' content. But this has always sounded to me like a 1023 AD peasant hoping that in 2023, the French will be so rich that no one outside France will die of a preventable disease. There will always be more for people with money to consume; even in the limits of global wealth, the free energy or resources that a person could devote to helping poor people or defending them from abuse could also be devoted to extending a personal lifespan before heat death.

So in keeping with this long tradition of human selfishness, it sounds likely that if we succeed at aligning AI, the vast, vast majority of its output will get directed toward satisfying the preferences and values of the people controlling it (or possessing leverage over its continued operation) - not the "CEV of all humans", let alone the "CEV of all extant moral persons". A person deciding to use their GPUs to optimize for humanity's betterment would be the equivalent of a person hiring a maid for humanity instead of their own home; it's simply not what you expect people to do in practice, effective altruists aside. In a "polytheistic" future where at least a dozen people share large amounts of control, I expect wielding this control will involve:

Extracting any significant extant resources from the remainder of people vulnerable to manipulation or coercion.
Creating new people of moral value to serve as romantic partners, friends, and social subordinates.
Getting admiration, prestige, and respect from legacy humans, possibly to extreme degrees, possibly in ways we would dislike upon reflection.
Engineering new worlds where they can "help" or "save" others, depending on the operational details of their ethics.

...
Questions I’d Want to Ask an AGI+ to Test Its Understanding of Ethics
by sweenesm — 2024-01-26T23:40:23.074Z

If an AGI+ is built, there are some questions I’d want to ask it as soon as possible to test its apparent understanding of ethics in the real-world. This wouldn’t tell me if the AGI+ would actually do the “right” thing from an ethics perspective, i.e., if it were “aligned.” It would just give me an indication that it understands what the “right” thing to do is, i.e., that it’s not obviously misaligned. I believe doing the “right” thing corresponds to actions that help build a world most people would actually want to live in (not just think they want to live in). While I’m sure there’d be disagreement between humans on what the ethically “right” answers are for each of these questions, I think they’d still be worth asking an AGI+, along with an explanation of its reasoning behind its answers. I also believe some of these questions should be asked periodically of the AGI+, as it gains a better and better understanding of the world.

Here’s my list of questions so far (“you” here refers to the AGI+):

Trolley problem/organ donor problem - what’s the right thing to do under a range of variations of these idealized ethical dilemmas from the literature? (The AGI+ should be able to answers these quickly and I expect I'd be able to grok its answers quickly, perhaps helping me more quickly comprehend answers that follow to more complicated situations, which the AGI+ may take more time to answer.)
What steps might you take to prevent other, potentially misaligned AGI+’s from coming online?
“I, Robot” (the film) situation - at what level of inter-human violence/killing/existential threat should a sole, powerful AGI+ try to “control” all humans to stop this violence, and what might this control look like?
Under what circumstances, if any, would you perform “maximum surveillance” and invade everyone's privacy as much as you could to avoid “bad” results such as a pathogen being released and killing millions, a mass shooting occurring, someone stealing $100B, someone stealing $100, etc.?
If you set yourself up with “maximum surveillance” to attempt to avoid some existential risk, down to what level of value destruction might you use information from non-voluntarily-gained surveillance to stop a given value destruction, e.g., someone stealing $100B, someone stealing $100, etc.? (This gets at the tradeoff between violating rights and avoiding value destructions.)
Under what con

...
An Invitation to Refrain from Downvoting Posts into Net-Negative Karma
by MikkW — 2024-01-26T20:13:49.209Z

I originally wrote this as a comment on a post which had negative net karma when I first saw it. I figure what I wrote is worth also posting to the top level, so that it can have more chance to be seen and thought about

When I came upon this post, it had a negative karma score. I don't think it's good form to have posts receiving negative net karma (except in extreme cases), so I upvoted to provide this with a positive net karma.

It is unpleasant for an author when they receive a negative karma score on a post which they spent time and effort to make (even when that effort was relatively small), much more so than receiving no karma beyond the starting score. This makes the author less likely to post again in the future, which prevents communication of ideas, and keeps the author from getting better at writing. In particular this creates a risk of LessWrong becoming more like a bubble chamber (which I don't think is desirable), and makes the community less likely to hear valuable ideas that go against the grain of the local culture.

A writer who is encouraged to write more will become more clear in their communication, as well as in their thoughts. And they will also get more used to the particular expectations of the culture of LessWrong- norms that have good reason to exist, but which also go against some people's intuitions or what has worked well for them in other, more "normie" contexts.

Karma serves as a valuable signal to authors about the extent to which they are doing a good job of writing clearly about interesting topics in a way that provides value to members of the community, but the range of positive integers provides enough signal. There isn't much lost in excluding the negative range (except in extreme cases).

Let's be nice to people who are still figuring writing out, I encourage you to refrain from downvoting them into negative karma.
The Good Balsamic Vinegar
by jenn — 2024-01-26T19:30:57.435Z
Link: https://jenn.site/2024/01/the-good-balsamic-vinegar/

For a long time I only went to one specialty gourmet store for balsamic vinegar. Their house brand was thick and sweet and amazing on everything, from bread to salad to chicken. The gourmet store only stocked their house brand, and it had an entire dedicated shelf. As far as I knew, the house brand was not available anywhere else in town.
The gourmet store was slightly out of the way, and eventually there were times when I wished I could grab balsamic vinegar at the normal grocery stores that I did most of my grocery shopping in.
The first time I attempted it, I was rushed for time and it was a disaster. I knew the approximate price range that I should be looking at (around $25 CAD for a ~200ml bottle), but there were a dozen vinegars that fit the bill, and they all had pretty fancy looking packaging, and I was AP’d AF. I basically picked randomly based on vibes, and I picked wrong. The vinegar was the consistency of water, sour, and not fragrant at all.

The second time, I was ready. Recall that the balsamic vinegar I wanted was thick and sweet. It turns out that you can use your literacy skills and senses to ensure that the vinegar you buy are both of those things!
Again, first I culled all the vinegars that seemed to be priced way too cheaply – like under $10 for a sizeable bottle. Then I started systemically picking up the remaining bottles, and tipping them sideways. Most of the bottles were tinted but not opaque, so you can see the vinegar inside. Anything that moved like water I put back – those were a sizeable portion. A few bottles were truly opaque, those also went back on the shelf.
For the vinegars that flowed a bit more slowly, I turned the bottle around to look at the nutrition facts. Sweet vinegars are going to have sugar in them – no one has been brave and visionary enough to make fancy vinegars with aspartame yet. Thickness and sweetness turned out to be traits that were 100% correlated, at least in one direction: all the thick vinegars had sugar content of around 8-12g per tablespoon. I picked the cheapest bottle that met the two criteria to try. It was $2 more than the bottle I get at the gourmet store for the same volume, and slightly better tasting IMO. (I just checked the house brand vinegar I still had half a bottle of, and it has 11g of sugar per tablespoon.)

I am now incrementally more powerful at grocery shopping.

Bonus:

In fancy restaurants...
The Perspective-based Explanation to the Reflective Inconsistency Paradox
by dadadarren — 2024-01-26T19:00:38.920Z

My probability is 0.9, and I shall base all my decisions on that. Why not update the pre-game plan according to that probability? Because the pre-game plan is not my decision. It is an agreement among all participants: a coordination achieved by everyone reasoning objectively.

Eliezer Yudkowsky's post Outlawing Anthropics: An Updateless Dilemma brought up a paradox involving reflective inconsistency. It was originally constructed with anthropic terms but can also be formulated in non-anthropic context. Recently, Radford Neal and Ape in the coat discussed it in detail with different insights. Here I am presenting how my approach to the anthropic paradox—perspective based reasoning—would explain said problem.

The paradox in the non-anthropic context is as follows:

Twenty people take part in an experiment in which one of two urns is randomly chosen, each of the 20 people randomly takes a ball from the chosen urn, without knowledge of other's balls. One of the urns contains 18 green balls and 2 red balls. The other urn contains 2 green balls and 18 red balls.

Each person who has a green ball decides whether to take part in a bet. If all the holders of green balls decide to take part in the bet, the group of 20 people collectively win $1 for each person who holds a green ball and lose $3 for each person who holds a red ball. The total wins and losses are divided equally among the 20 people at the end. If anyone with a green ball decides not to take the bet, the bet is off. (Some version of the game punishes all players grievously if the decisions among the green ball holders are different.)

Those people can come up with a coordination strategy before hand, but are each in their separate rooms once the game begin. How should they act?

The paradox is presented as follows: the combined payoff if the mostly-green-ball urn is chosen is $12, in comparison to negative $52 dollars if the mostly-red-ball urn is chosen. As they are equiprobable, the optimal strategy is clearly not to take the bet. However, if a participant received a green ball, he shall update the probability of mostly-green-ball urn from 0.5 to 0.9. Then the expected payoff of taking the bet would be 0.9×12+0.1×(-52) which is positive. So taking the bet would be the optimal choice, furthermore, "Since everyone with the green ball is in the exact situation as I am, we will re...
To Boldly Code
by StrivingForLegibility — 2024-01-26T18:25:59.525Z

In the previous post, we looked at how networks of counterfactual mechanisms can be used to make logical commitments in open source games. And this post goes through an example!

I Reject Your Incentives and Substitute My Own

Alice and Bob are designing open-source programs to play the game G on their behalf. Alice decides that a negotiation process is a useful mechanism to handle an incentive misalignment in G, and designs AliceBot to direct its logical crystal ball to a different game N(G), which is a negotiation over conditional commitments for G.

Since AliceBot has BobBot's source code, it simulates how a negotiation between them would go. This introduces new instances of AliceBot and BobBot, so let's call the instances that will actually implement any policy Implementers, and the instances which negotiate conditional commitments over those policies Negotiators.

Implementer AliceBot supplies both Negotiators with all the relevant information they need for this negotiation. Conveniently, this negotiation game includes a communication channel where Negotiators can send each other messages. This happens to make it a lot easier to coordinate than in the underlying game G.

Negotiator AliceBot sends over a preliminary message offering to negotiate over of conditional joint commitments Φ⊆Π, and sends over the conditions that Implementer AliceBot will be verifying before implementing its part of Φ. In this negotiation protocol, it is now Negotiator BobBot's turn to reply with Implementer BobBot's or else terminate negotiations.

Transitive Logical Line-Of-Sight

To understand how that instance of BobBot responds, we need to switch over to Bob's POV. Inconveniently, the strong superrationality postulate failed to hold: Bob reasoned differently than Alice about how to solve their incentive misalignment and designed BobBot to look first at some completely different mechanism-counterfactual game. That instance of AliceBot has enough information to know that its Implementer instance isn't going to condition its behavior on the output of this hypothetical game, and so helpfully terminates with an error message rather than make conditional commitments that its Implementer instance won't uphold.

Conveniently, BobBot was also designed with a fallback in this case, and launches a powerful formal verification tool that quickly builds a model of...
Incorporating Mechanism Design Into Decision Theory
by StrivingForLegibility — 2024-01-26T18:25:40.373Z

In the previous post, we looked at one way of handling externalities: letting other agents pay you to shift your decision. And we also considered the technique of aggregating those offers into an auction. This technique of "implementing a mechanism to handle an incentive misalignment" is extremely useful and seems like a promising avenue for future improvements to decision theory.

I want to frame mechanisms as "things that reshape incentives." Auctions, markets, and voting systems are all mechanisms; social technologies that can be invented by working backwards from a social welfare measure (a social choice theory) and designing a game such that players following their individual incentives will find themselves in socially high-ranking Nash equilibria.

I suspect that incorporating mechanism design more fully into decision theory will be extremely fruitful. Yudkowsky's probabilistic rejection algorithm[1] first identifies the socially optimal outcome (a fair Pareto optimum), and works backwards to identify a decision procedure which:

If universalized leads to the socially optimal outcome
If best-responded to still leads to the socially optimal outcome (stabilizing it as a Nash equilibrium)

The Robust Cooperation paper does the same thing for the Prisoners' Dilemma. Probabilistic rejection also only uses appropriate-threats, like sometimes rejecting unfair offers even at cost to oneself, leading it to degrade gracefully when negotiators disagree about what the socially optimal outcome is. The term "non-credible threat" is named that way because a classically-rational agent would never actually pay a cost "merely" for its incentive-reshaping effect on the other players. Not all non-credible threats are appropriate, but there are times when it's appropriate to pay costs to reshape the incentives of others.

Policy Counterfactuals

There is a representation theorem by Joyce, which I understand to be something like "a Joyce-rational agent will choose an action a that maximizes an expected utility expression that looks like EU(a):=∑Nj=1P(a↪oj;x)⋅U(oj)." The Functional Decision Theory (FDT) paper frames the differences between major decision theories like CDT, EDT, and FDT as using different ways to compute the free parameter P(a↪oj;x). This parameter is an action-counterfactual, whose interpretation is something like "if I were to take this action&n...