trentmkelly/LessWrong-43k huggingface dataset:
Resolving internal conflicts requires listening to what parts want
by Richard_Ngo — 2023-05-19T00:04:20.451Z

In the first four posts of this sequence, I outlined how fear can lead to internal conflict. In the next four posts I'll explore and defend the following claim: we can resolve internal conflicts by understanding what underlying fears are driving a conflict, then providing evidence that those fears won't happen, thereby

reconsolidating the memories

which caused them. A simple example of this process comes from

exposure therapy

for phobias, which works by demonstrating that the phobia is much less dangerous than the person had learned to believe. A wide range of different therapeutic approaches apply the same core mechanism to deal with more complex internal conflicts. I'll focus in particular on the

internal family systems (IFS) framework

—which, despite the slightly kooky name, is one of the most powerful methods for dealing with internal conflict.

The

core ideas of IFS

are essentially the ones I've outlined in the last few posts: that you should think of yourself as being composed of many parts, some of which are implementing protective strategies based on your previous experiences (especially from childhood). IFS particularly highlights the idea that there are “

no bad parts

”—we should treat all parts as deserving of sympathy, even when the strategies they’re using are harmful and deeply misguided. From an IFS perspective, the memory reconsolidation process I described above can also be see as a process of build trust between different parts of yourself. In this post I'll focus on the first step: identifying the underlying parts at play and what they want.

Our starting point can be any phenomenon that triggers an emotional response from some part of you. You might find one by thinking about an emotionally-loaded topic, like your work or relationships (especially with your parents); or

paying attention to how your body feels

; or paying attention to the way you choose your words or thoughts, and which ones you're suppressing; or to your dreams; or to character archetypes or symbolic motifs that particularly resonate with you. Many types of triggers work far better for some people than others—a fact which helps explain why so many different psychotherapy techniques exist.

However you find an emotional trigger, the next step is to work through the protective or defensive strategies associated with the part causing the response, to figure out what its unde...
How could I measure the nootropic benefits testosterone injections may have?
by shapeshifter — 2023-05-18T21:40:53.704Z

I came across a post written by Gwern in which he says he never injected Testosterone before but is curious to know more about its potential nootropic benefits. I'm suspecting that its more people than just him who are curious about that soo why not experiment myself?

I wanted to "hop on another cycle of gear" (use steroids for 16-20 wks and restore my natural hormonal production right after) anyway but don't know anything about experiment design so if someone wants to help me out with that that would be pretty cool.
Investigating Fabrication
by LoganStrohl — 2023-05-18T17:46:52.783Z

Note 1: This was originally published in 2021. I'm now re-publishing it in 2023 as the penultimate post in The Nuts and Bolts Of Naturalism, as a demonstration of the method. I intend to publish a collection of much-more-complete demonstrations in the future, and I plan for them to explicitly discuss each step as I've described it in the sequence. For now, though, I hope that this partial demo will give you a taste of one way the overall process can go.
(For a similarly partial demo by someone other than me, check out "Maps of Maps, and Empty Expectations" by Nora Ammann.)

Note 2: This essay was written in response to “

Lies, Damn Lies, and Fabricated Options

”. I expect it’ll be pretty confusing to read this one without having read that one first. However, I’m not the boss of you, and it isn’t obvious to me anyway which reading order is best overall. If prolonged confusion makes you grumpy, I recommend reading Duncan’s essay first. But if you try it backwards, I’d love to hear how that goes.

1. Motivation

When I read Duncan's essay "

Lies, Damn Lies, and Fabricated Options

," it seemed to me to point toward something that's worth knowing for real, rather than just worth knowing about.

I think it does a good job of helping the reader build a new concept that could conceivably be taken as a hypothesis in important situations. The hypothesis is, “Perhaps this thing that seems to me like an option isn’t really a possibility, and it only seems like one because I’m in the grips of a map/territory conflation.” (Or, at least, “Perhaps the person I’m arguing with thinks their made-up option is real because [etc.].”)

But I wasn’t satisfied with my new concept “fabricated options”. There were some super important questions that I’d need to answer before I could put it to use.

For instance,

Which situations are the crucial ones for promoting this hypothesis to attention?
How do I actually move my mind to recognize that the option I’m considering might be fabricated?
Once I’ve posed the hypothesis, how do I confirm or deny it?
When I’ve successfully identified a fabricated option, how can I prevent the usual harm?
And, my favorite: How can I stop making this kind of mistake in the first place?

So I set out to investigate option fabrication (the act), and what my brain is up to in the moments just before a fabricated option appears. I hoped that if I could learn where fabricated op...
Microsoft and Google using LLMs for Cybersecurity
by Phosphorous — 2023-05-18T17:42:36.939Z

Google

and

Microsoft

are integrating their LLMs, PaLM and GPT-4 respectively, into their cybersecurity services.

They basically have two uses cases:

1. Better User Interfaces + Analysis Help for Security Analysts

Using the LLM to explain incidents (cyberattacks or things that might be attacks) in natural language, and answer questions from analysts in natural language, very similar to how one might use a LLM chatbot as a research assistant, except for these systems are given access to all the data within your organization + other Google or Microsoft tools.

Security analysts are constantly beset by “

alert fatigue

” - they receive so many alerts for things that might be a cyberattack, that they can’t handle it all. These LLMs help address that by summarizing and contextualizing the alerts. They can also be used to automatically write queries (formal requests for information) to the network, which are more like code generation

Both companies also hope that explaining incidents in natural language will help address the talent shortage of cybersecurity professionals, by helping security analysts with less experience do better analysis.

This is a parallel to how much of the military is going to make use of LLMs - by saving time for human intelligence analysts (

CSIS estimates 45 days a year per analyst!

).

2. Reverse Engineering to Identify Malware

That is, you can feed it the machine code for a piece of software you think might be malware, and it will tell you what the machine code does, and provide summaries in natural language. This seems like one of the biggest use cases for defenders, since “you can do in minutes what used to take a whole day” (Microsoft tutorial).

Google's version of this tool is called VirusTota Code Insightl, more documentation

here

. On a broader level, you don’t just have to upload a piece of suspected malware to these tools to get a result - these tools can always be scanning your network looking for code that could be malware. In this way, they’re not just a user interface improvement or a gimmick, they’re the latest improvement in a long line of

malware detection tools

that are pretty central to how enterprises defend themselves.

What does this mean?

Context

Microsoft and Google, among other things, are huge cybersecurity players. So, this is a fairly big deal for the world of cyber. ...
The Benevolent Billionaire (a plagiarized problem)
by Ivan Ordonez — 2023-05-18T17:39:25.801Z

Hello, this is my first post, as far as I can remember (I may have been asleep). My news feed shows after a few years, new entries about the Sleeping Beauty problem, and this reminded me of this variation I wrote a few years ago.

One-paragraph summary: this is a story restatement of the Sleeping Beauty problem in (hopefully) more realistic terms, with exaggerated odds, and with real stakes: there is money involved. Philosophers who have contributed to the original problem are mentioned presenting their positions in, I hope, fair terms.

It includes some questions left to the reader. I would like very much to see what this community's answers to the questions are!

No further ado. Here it is.

On day 1, the Benevolent Billionaire (BB) makes an announcement that is broadcast in the news: she will send a postcard to every mailing address in the world as a gesture of goodwill. You estimate there are 1 billion mailing addresses or more in the world.

On day 2, BB announces she has reconsidered, after talking with her financial advisors. Instead, she now says, she has drawn a random address from the totality of them, using a perfectly uniform distribution and a perfect random number generator. She will mail a postcard to that address only, which is for now kept a secret that she only knows. Public outcry ensues.

On day 3, BB says she has reconsidered again. A Solomonic decision has been reached, she says. She will flip a perfectly fair coin on day 4. If the coin lands heads, she will mail a card only to the secret random address she pulled on day 2. However, if it lands tails, she will mail one to every mailing address in the world, as originally planned. The outcome will be kept a secret until day 10, a time when it is guaranteed that at least one card will have reached its destination.

On learning of all this, you smile to yourself thinking that, as eccentric as BB’s idea sounds, at least it does not involve any memory-erasing drugs used on anyone, beautiful or not, or reconstructing perfect doppelgängers, or any such thing that would strain credibility. Whatever BB is doing, it requires nothing but a coin, addresses and postcards.

On day 9, you receive a postcard in the mail. You examine it carefully and conclude, with certainty, that it is one of the promised postcards from BB.

You ponder what may have happened. Did the coin land heads, and yours happened to be the one cho...
Notes from the LSE Talk by Raghuram Rajan on Central Bank Balance Sheet Expansions
by PixelatedPenguin — 2023-05-18T17:34:23.871Z

In a recent seminar, I had the chance to listen to a talk by Raghuram Rajan discussing his paper titled Liquidity Dependence and the Waxing and Waning of Central Bank Balance Sheets. The paper examines the new situation of central bank policies, where the cycle between QE and QT creates financial instabilities instead of mitigating them.

Key Points from the Paper

The paper puts forward the idea of "liquidity dependence," where the supply of reserves during quantitative easing (QE) leads to an increase in bank liquidity claims. In turn, an even larger central bank balance sheet is needed. This buildup of liquidity claims can make quantitative tightening (QT) difficult, while also dulling the impact of QE on the broader economy.

Some key findings from the paper include:

During QE, commercial banks financed the increase in reserves by issuing more demand deposits and reducing time deposits. They also issue more credit lines.
When the Fed halted QE and started QT, there was no corresponding decrease in bank demandable claims. This left the system more sensitive to liquidity shocks.
Weakly capitalized banks were found to engage more in maturity shortening of deposits during QE, taking on more liquidity risk for modest returns. They were also more exposed during the COVID liquidity shock.
The authors suggest that more liquidity regulation and supervisory policy that accounts for changing conditions may be needed to mitigate these issues.

Visualizing the Change in Demandable and Time Deposits

Simply put, during QE, time deposits are replaced by demandable ones:

Asset	Liability
Reserves	▲Demandable Deposits
Securities	▼Time Deposits

And during QT, banks experience a shrink in asset value as a result of interest rate risk:

Asset	Liability
Reserves	= Demandable Deposits
▼Securities	= Time Deposits

This can explain the SVB bank case, where its long-term securities lose value. On its liability side, there remains a lot of demandable, instead of time deposits, which increases its liquidity risk.

Here is a simple visual analysis, where dimmed regions represent QEs. A similar figure can be found in the paper. Note that the QT period starting in 2015 does not change either the time or demandable deposits.

Time Deposits, Demand Deposits, and Reserves as a Percentage of GDP (2008-2021)

Notes from the Talk

A timely mention of Goodhart's Law and Murphy's La

...
We Shouldn't Expect AI to Ever be Fully Rational
by OneManyNone — 2023-05-18T17:09:13.530Z

Summary of Key Points[1]

LLMs are capable of being rational, but they are also capable of being extremely irrational, in the sense that, to quote EY's definition of rationality, their behavior is not a form of "systematically promot[ing] map-territory correspondences or goal achievement."

There is nothing about LLM pre-training that directly promotes this type of behavior, and any example of this behavior in fundamentally incidental. It exists because the system is emulating rationality it has seen elsewhere. That makes LLM rationality brittle. It means that there's a failure mode where the system stops emulating rationality, and starts emulating something else.

As such, LLM-based AGI may have gaps in their reasoning and alignment errors that are fundamentally different from some of the more common errors discussed on this forum.

Emulated Emotion: A Surprising Effect (In Retrospect)

Five years ago, if you had asked a bunch of leading machine learning researchers whether AGI would display any sort of outward emotional tendencies - in the sense that it would set goals based on vague internal states as opposed to explicit reasoning - I think the majority of them would have said no. Emotions are essentially a human thing, reflections of subjective internal experiences that would have no reason to exist in AI, particularly a superintelligent one.

And I still strongly believe that LLMs do not have emotions that resemble human internal states. What I think has become very clear, however, is that they can very much act as if they had emotions.[2]

Take, for instance, this exchange showing Bing AI getting "angry" at a user:

Source

Now, if you actually understand how LLMs work, this is an entirely unremarkable, fully expected (if not somewhat unfortunate) phenomenon. Of course they would output emotionally charged text, why wouldn't they? They've been exposed to such a huge number of emotionally-charged human interactions; the result is inevitable.

But if you take a step back and look at it in the grand scheme of things, considering our expectations from just a few years ago, I think it's an absolutely fascinating effect. Part of the goal of building an AGI is to distill the entirety of human knowledge into a single entity capable of reasoning, and if you could approach that goal in a direct way you wouldn't expect to find any outwardly emotional behavior becaus...
Relative Value Functions: A Flexible New Format for Value Estimation
by ozziegooen — 2023-05-18T16:39:31.132Z

Summary

Quantifying value in a meaningful way is one of the most important yet challenging tasks for improving decision-making. Traditional approaches rely on standardized value units, but these falter when options differ widely or lack an obvious shared metric. We propose an alternative called relative value functions that uses programming functions to value relationships rather than absolute quantities. This method captures detailed information about correlations and uncertainties that standardized value units miss. More specifically, we put forward value ratio formats of univariate and multivariate forms.

Relative value functions ultimately shine where single value units struggle: valuing diverse items in situations with high uncertainty. Their flexibility and elegance suit them well to collective estimation and forecasting. This makes them particularly well-suited to ambitious, large-scale valuation, like estimating large utility functions.

While promising, relative value functions also pose challenges. They require specialized knowledge to develop and understand, and will require new forms of software infrastructure. Visualization techniques are needed to make their insights accessible, and training resources must be created to build modeling expertise.

Writing programmatic relative value functions can be much easier than one might expect, given the right tools. We show some examples using

Squiggle

, a programming language for estimation.

We at QURI are currently building software to make relative value estimation usable, and we expect to share some of this shortly. We of course also very much encourage others to try other setups as well.

Ultimately, if we aim to eventually generate estimates of things like:

The total value of all effective altruist projects;
The value of 100,000 potential personal and organizational interventions; or
The value of each political bill under consideration in the United States;

then the use of relative value assessments may be crucial.

Presentation & Demo

I gave a recent presentation on relative values, as part of a longer presentation in our work at QURI. This features a short walk-through of an experimental app we're working on to express these values.
The Relative Values part of the presentation is is from 22:25 to 35:59.

This post gives a much more thorough description of this work than the presentati...
Some background for reasoning about dual-use alignment research
by Charlie Steiner — 2023-05-18T14:50:54.401Z

This is pretty basic. But I still made a bunch of mistakes when writing this, so maybe it's worth writing. This is background to a specific case I'll put in the next post.

It's like a a tech tree

If we're looking at the big picture, then whether some piece of research is net positive or net negative isn't an inherent property of that research; it depends on how that research is situated in the research ecosystem that will eventually develop superintelligent AI.

A tech tree, with progress going left to right. Blue research is academic, green makes you money, red is a bad ending, yellow is a good ending. Stronger connections are more important prerequisites.

Consider this toy game in the picture. We start at the left and can unlock technologies, with unlocks going faster the stronger our connections to prerequisites. The red and yellow technologies in the picture are superintelligent AI - pretend that as soon as one of those technologies is unlocked, the hastiest fraction of AI researchers are immediately going to start building it. Your goal is for humanity to unlock yellow technology before a red one.

This game would be trivial if everyone agreed with you. But there are many people doing research, and they have all kinds of motivations - some want as many nodes to be unlocked as possible (pure research - blue), some want to personally unlock a green node (profit - green), some want to unlock the nearest red or yellow node no matter which it is (blind haste - red), and some want the same thing as you (beneficial AI - yellow) but you have a hard time coordinating with them.

In this baseline tech tree game, it's pretty easy to play well. If you're strong, just take the shortest path to a yellow node that doesn't pass too close to any red nodes. If you're weak, identify where the dominant paradigm is likely to end up, and do research that differentially advantages yellow nodes in that future.

The tech tree is wrinkly

But of course there are lots of wrinkles not in the basic tech tree, which can be worth bearing in mind when strategizing about research.

Actions in the social and political arenas. You might be motivated to change your research priorities based on how it could change peoples' minds about AI safety, or how it could affect government regulation.
Publishing and commercialization. If a player publishes, they get more money and prestige, which boosts their abilit

...
The Unexpected Clanging
by Chris_Leong — 2023-05-18T14:47:01.599Z

There are two boxes in front of you. In one of them, there is a little monkey with a cymbal, whilst the other box is empty. In precisely one hour the monkey will clang its cymbal.

While you wait, you produce an estimate of the probability of the monkey being in the first box. Let's assume that you form your last estimate, p, three seconds before the monkey clangs its cymbal. You can see the countdown and you know that it's your final estimate, partly because you're slow at arithmetic.

Let Omega be an AI that can perfectly simulate your entire deliberation process. Before you entered the room, Omega predicted what your last probability estimate would be and decided to place the monkey in a box such as to mess with you. Let q be the probability of Omega placing the monkey in the first box. In particular, Omega, sets q=p/2, unless p=0 or you haven't formed a probability estimate, in which case q=1.

What probability should you expect that the monkey is in the first box?

I think it's fairly clear that this is a no-win situation. No matter what the final probability estimate you form before clanging, as soon as you've locked it in, you know that it is incorrect, even if you haven't heard the clanging yet. You can try to escape this, but there's no reason that the universe has to play nice.

This problem can be seen as a variation on Death in Damascus. I designed this problem to reveal that the core challenge Death in Damascus poses isn't just that another process in the world can depend upon your decision, but that it can depend upon your expectations even if you don't actually make a decision based upon those expectations.

I also find this problem as a useful intuition pump as I think it's clearer that it's a no-win situation than in other similar problems. In Newcomb's problem, it's easy to get caught up thinking about the Principle of Dominance. In Death in Damascus, you can confuse yourself trying to figure out whether CDT recommends staying or fleeing. At least to me, in this problem it is clearer it is a dead end and that there's no way to beat Omega.

This is also a useful intuition pump for the Evil Genie Puzzle. When I first discovered this puzzle, I felt immensely confused that no matter which decision that you made you would immediately regret it. However, the complexity of the puzzle made it complicated for me to figure out exactly what to make of it, so ...