trentmkelly/LessWrong-43k huggingface dataset:
Exploring Functional Decision Theory  (FDT) and a modified version (ModFDT)
by MiguelDev — 2023-07-05T14:06:13.870Z

TL;DR

This post explores the role of Functional Decision Theory (FDT) in AI alignment, showcasing its efficacy in complex decision-making scenarios. The results obtained highlight the promising potential of FDT as a subject for further research in addressing the alignment problem. Additionally, this post helps clarify the distinction between FDT and ATL, which can be tentatively viewed as a modified version of FDT (ModFDT).

Intro

Within the intersection of philosophy and artificial intelligence, decision theories play a crucial role in steering the behavior of hypothetical rational agents – entities that consistently strive to make the most rational decisions, given their current information and preferences. In this discussion, I will explore two widely recognized decision theories: Causal Decision Theory (CDT) and Functional Decision Theory (FDT). I will delve into their unique characteristics and articulate why FDT emerges as the most promising approach in tackling the complex issue of AI alignment. If you are familar with CDT and FDT, you can skip directly to my arguments on Corrigibility and FDT.

Causal Decision Theory (CDT)

CDT advises an agent to perform an action that would cause the best outcome. CDT suggests that an agent's decision should be based on the causal consequences of possible actions. If an action is likely to cause a desirable outcome, then that action is what should be performed.

Functional Decision Theory (FDT)

FDT is a decision theory proposed by Nate Soares and Eliezer Yudkowsky that argues that agents should choose actions that they expect (based on a fixed mathematical function) to correlate with good outcomes, even if they don't cause those outcomes. So, FDT might recommend a different action than CDT in situations where there's a clear correlation between your decision and some uncontrolled factor, but where your decision doesn't cause a change in that factor.

CDT vs FDT in AI Decision Making

CDT and FDT are essential in AI systems, as they prescribe the most rational action a decision-maker should take in various scenarios. However, their approaches can lead to divergent outcomes.

To illustrate this, let's examine the classic Newcomb's paradox. In this scenario, an AI has to choose between two boxes. Box A contains $1,000 and Box B contains $1,000,000. However, Box B will only be filled if a predictor, ...
Optimized for Something other than Winning or: How Cricket Resists Moloch and Goodhart's Law
by A.H. — 2023-07-05T12:33:07.166Z

Recently, there has been some controversy in the world of cricket. The full details are described well in this piece but the gist of it is as follows. During a game between Australia and England, after the ball had been bowled, the English batsman Jonny Bairstow stepped forward thinking that the game was no longer in play. This allowed the Australian wicketkeeper Alex Carey to throw the ball at the stumps and get Bairstow out. As a result, Bairstow had to leave the field and later that day, Australia won the game. Would England have won were it not for this piece of quick thinking from Carey? It's hard to say, but the game certainly would have been a lot closer.

So why is this controversial? After all, Carey's actions were completely within the rules of the game and all of the umpires and players agreed that Bairstow indeed out. The controversy stems from a nebulous concept known as 'the spirit of cricket'. The 'spirit of cricket' is mentioned in the preamble to the official Laws of Cricket and broadly encompasses good sportsmanship and fair play. But the fact that it is not properly defined means that different people will have different views on what kind of actions are within the spirit of the game. Pat Cummins, the Australian captain believes that Carey's actions were within the spirit of the game, and the Ben Stokes, the English captain believes that they were not. Various media outlets have taken sides on this issue.

To be clear, the 'spirit of cricket' is not just 'good sportsmanship applied when convenient'. Players often make choices that are within the spirit of the game which put their team at a severe disadvantage. Indian player MS Dhoni famously withdrew his appeal against Ian Bell. Within the rules of the game Bell had been run out in a moment of confusion but Dhoni believed that it was not within the spirit of the game, and let Bell come back to bat. The spirit of the game is something that spectators and players genuinely care about. Certain historical cricketing episodes, such as the underarm incident, and the bodyline tour were within the rules but still evoke strong emotions to this day[1]. Other tactics, like Mankading (where a bowler hits the stumps during his run-up to 'run out' an over-eager batsman who has started running) are allowed by the rules but rarely occur, though they still happen occasionally, generating controversy when...
Puffer-pope reality check
by Neil  — 2023-07-05T09:27:11.200Z

This post is humor. It's meant to be entertaining and nothing more. tl;dr: nothing quite summarizes the weirdness of the 21rst century as well as this picture.

I was sorting through my mind dump recently and found a "first-principles rant" on the widely circulated image of the pope in a puffer jacket.

This image was:

Built using technology that effectively amounts to magic.
Magic controlled by a guy on shrooms. The talking space monkeys that we are have developed magic, only to use it to make silly-looking pictures as we chemically disrupt our brains for the fun of it.
Was generated thousands of kilometers away from where its high creator came up with the prompt. (For free.)
The idea behind computers is essentially: "what if we put lightning through a maze? If the maze is built in just the right way we can make the lightning do math, because that's how geometry works apparently."
This image was generated blindly through multiplying matrices. It is purely the result of calculations, nothing more.
While it took a ton of fine-tuning to get here, we frankly have no idea how Midjourney works. Fine: by changing the geometry of the universe we are capable of making it generate puffer-pope pictures for us; but we're not sure how it works exactly because we're training our models at the macro-level (and the micro-level is really really complex, and not as efficient as it could be, but that doesn't matter to us because we have a lot of spare compute).
The Pope is a figure that most people on Earth know. Why? Because we have this thing called religion. Religion is a meme which relies on believing--without evidence-- that the universe is essentially sentient and came up with us on purpose.
Religion is really viral apparently. The drippy guy in the pic is the spiritual leader ("") of over 1,300 million humans, distributed all over the planet.
Human language is oddly efficient. The Midjourney prompt was a single sentence long: "Pope Francis in a puffer jacket walking through the streets of Rome".[1] This reminds me that you can also get incredibly specific results with just 5 words or so on Google.
I've seen this picture and you've seen this picture and somehow this picture has become famous o

...
MXR Talkbox Cap?
by jefftk — 2023-07-05T01:50:01.059Z

I have an

MXR
M222 talkbox that I'm pretty happy with, but when the tube is
disconnected the speaker is very exposed:

I'm worried about damage to the speaker, and would like some sort of
cap to protect it. When I search for "MXR M222 talkbox cap" or
similar I don't find anything. I wonder if it would be possible to
make something?

Measuring, the outer diameter [1] is 24mm:

It sticks 10mm up from the plane of the box.

Would 3D printing be the way to go, or are the materials not a good
match for a friction fit? Elastic + cloth is another
idea. Suggestions?

[1] In case anyone's curious, I also measured the inner diameter while
I was at it, and it's 19mm, almost exactly.

Comment via: facebook, mastodon
"Reification"
by herschel — 2023-07-05T00:53:48.984Z

I regularly use the concept of reification, and often get back confusion as to what I mean, even from friends who are rather literate

The OED entry for ‘reify’:

transitive. To make (something abstract) more concrete or real; to regard or treat (an idea, concept, etc.) as if having material existence.

Here’s an attempt at a definition for my usage:

To make a perception, feeling, belief, or conception more real, solid, or salient, by any means.

The map is not the territory. Furthermore, our attempts at apprehension of the territory are refracted through our ontological stack. The perception of patterns occurs on our end, regardless of the correctness of that perception. Perceptions are substantially flexible and subject to influence culturally and socially.

The definition from the OED describes one of the grossest forms of reification but there are gradations and subtleties. In general the tendency is for the grandest and grossest reifications to rely on constructing a pattern from the territory as ontologically or metaphysically real. Plato’s theory of forms is the epitome of this kind of reification. Other obvious examples are God, heaven/hell, and karma/rebirth.

One form of reification that’s relatively malleable for most people is perception of visual patterns. If you look at a square grid, you can also tilt your head and perceive the 45 degree subgrid overlaid, or other subgrids at smaller angles. We can choose to reify latent possible perceptions of the same object. I don’t believe it’s appropriate to say that any of these are the “true” perception, indeed they’re all viable and consistent perceptions, which may naturally flit back and forth anyway, and which can be made more salient with some play of intention and attention.

I would say here that a view can exaggerate certain features or properties of perceptions, or just make them more salient etc.

One marvelous kind of reification is the standard “because I said so”, said by a parent to a child. Certainly because one said it is no more evidence about the truth of the claim, but this move brings more oomph to the emotional/relational dynamic, carrying (or at least emphasizing) real power. This is actually a barer kind of reification: it carries no justification, it simply makes a command more real, as in, carrying more emotional weight. It’s easy enough to se...
Dominant Assurance Contract Experiment #2: Berkeley House Dinners
by Arjun Panickssery — 2023-07-05T00:13:15.255Z
Link: https://arjunpanickssery.substack.com/p/dominant-assurance-contract-experiment?sd=pf

As an afterthought at the end of my previous post on dominant assurance contracts, I decided on a whim as I was typing to append a small dominant assurance contract:

In theory, writers could kickstart posts using dominant assurance contracts. An example (this is a real offer): If you send $20 to arjun.panickssery at Gmail via PayPal by noon New York time on January 21st, I'll send you back $25 if fewer than 10 people sent me money. If 10 or more people send me money, I'll post a review of Steven Pinker's The Sense of Style: The Thinking Person's Guide to Writing in the 21st Century by the end of the month. I'm not sure whether I'm just giving away free money right now.

I received exactly nine out of the ten funders needed and lost $45.

Now a second attempt, this time in the real world: at Andromeda House we plan to host large weekly dinners on Monday evenings for the local EA/rationality/etc community at our house in Southside Berkeley.

Here’s how it works:

You can Venmo me (@Arjun-Panickssery if the link doesn't work) or PayPal me any amount of at least $20 with the subject line "dinner" or similar.

If I get at least $700 total by noon Pacific time on July 15, I'll host dinners from July 17 till the end of August (seven dinners).

If I get less than $700 total, I'll give you a 25% return (e.g., if you sent me $100, I'll send you back $125).

Example: If you live in town and this service would be worth more than $20 per dinner to you and you'd expect to come to three before September, you should pay at least $60, since you either get a service worth more than that amount or you make a 25% return. You could try to free-ride but the fact that my previous contract received just one less than the target number suggests that I'm well-calibrated.

If you don't live in town—or even if you do—you can also idly speculate if you think that I'm unlikely to hit my target and likely to pay out.

You could also just pseudo-donate money without regard to whether you think I'm calibrated just because you think that DACs are cool and should proliferate.
Three camps in AI x-risk discussions: My personal very oversimplified overview
by Aryeh Englander — 2023-07-04T20:42:12.829Z

[I originally wrote this as a Facebook post, but I'm cross-posting here in case anybody finds it useful.]

Here's my current overview of the AI x-risk debate, along with a very short further reading list:

At a *very* overly simplified but I think still useful level, it looks to me like there are basically three "camps" for how experts relate to AI x-risks. I'll call the three camps "doomers", "worriers", and "dismissers". (Those terms aren't original to me, and I hope the terminology doesn't insult anybody - apologies if it does.)

1) Doomers: These are people who think we are almost certainly doomed because of AI. Usually this is based on the view that there is some "core" or "secret sauce" to intelligence that for example humans have but chimps don't. An AI either has that kind of intelligence or it doesn't - it's a binary switch. Given our current trajectory it looks entirely possible that we will at some point (possibly by accident) develop AIs with that kind of intelligence, at which point the AI will almost immediately become far more capable than humans because it can operate at digital speeds, copy itself very quickly, read the whole internet, etc. On this view, all current technical alignment proposals are doomed to fail because they only work on AIs without the secret sauce, and they'll completely fall apart for AIs with the secret sauce because those AIs will be fundamentally different than previous systems. We currently have no clue how to get a secret-sauce-type-AI to be aligned in any way, so it will almost certainly be misaligned by default. If we suddenly find ourselves confronted with a misaligned superintelligence of this type, then we are almost certainly doomed. The only way to prevent this given the state of current alignment research is to completely stop all advanced AI research of the type that could plausibly lead to secret-sauce-type-AGIs until we completely solve the alignment problem.

People in this camp often have very high confidence that this model of the world is correct, and therefore give very high estimates for "P(doom)", often >95% or even >99%. Prominent representatives of this view include Eliezer Yudkowsky and Connor Leahy.

For a good, detailed presentation of this view, see An artificially structured argument for expecting AGI ruin by Rob Bensinger.

[EDIT: Another common reason for being a Doomer is if you have really sh...
Six (and a half) intuitions for SVD
by CallumMcDougall — 2023-07-04T19:23:19.688Z

​The long-awaited[1] sequel to my "Six (and a half) intuitions for KL divergence" is finally here!

Thanks to the following people for feedback: Denizhan Akar, Rudolf Laine, Simon Lermen, Aryan Bhatt, Spencer Becker-Kahn, Arthur Conmy, and anonymous members of my ARENA cohort (-:

If you want to 80-20 this post (and you already know what SVD is), then just read the "Summary" section below.

Summary

The SVD is a matrix decomposition M=USVT, where U and V are orthogonal, and S is diagonal (with non-negative diagonal elements, which we call singular values). The six intuitions are:

1. Rotations and Scalings (geometric picture)

Orthogonal matrices U and V can be thought of as rotations of our basis, and S as a scaling of our new basis. These operations are easier to geometrically visualise than the entire operation of a matrix, with all its nuances (e.g. things like shear).

2. Best rank-k Approximations

Truncating the SVD sum gives us in some sense the "best way" to approximate M as a sum of rank-1 matrices. In other words, the first few matrices in the SVD sum are the "most important parts of M".

3. Least Squares Regression

If we're trying to minimize ∥Mx−b∥ (either with a constraint on the norm of x or with no constraint), we can express the solution for x in terms of the SVD of M. When ∥x∥ is constrained, it becomes most important to get the components of x right in the directions of the largest singular values of M; the smaller singular values matter less.

4. Input and Output Directions (like MLPs!)

An MLP can be thought of as a collection of neurons, where each neuron has an input direction (thought of as detecting a feature) and an output direction (outputting a result if that feature is observed). vi and ui are like the input and output directions (minus the nonlinearity in the middle), and furthermore each "neuron" in this case is completely independent (orthogonal).

5. Lost & Preserved Information

SVD tells you what information (if any) is lost when you use M as a linear map. It also tells you which bits of information are easier / harder to recover.

6A. Principal Component Analysis

If M is actually a matrix of data, we can interpret the SVD matrices in terms of that data. V tells you whic

...
Animal Weapons: Lessons for Humans in the Age of X-Risk
by Damin Curtis — 2023-07-04T18:14:24.166Z

“Reminder that you can listen to this post and other EA Forum/LessWrong posts on your podcast player with The Nonlinear Library.”

In case you study X-Risks and haven’t yet read the book Animal Weapons: The Evolution of Battle, this post is meant to flag it as an important book for understanding the factors which lead to arms races, and to summarize the book’s findings as they relate to the study and prevention of X-Risks. The book is written by Douglas Emlen, an evolutionary biologist researching the circumstances which lead to the evolution of large and expensive weapons in animals (think elephant tusks, beetle horns, extravagant deer antlers, and oversized crab claws.) Emlen demonstrates how these same circumstances also lead to arms races in human combat technologies.

The main takeaways for x-risk concerned readers, as I see them:

In order to occur,

arms races require three ingredients: evolutionary competition, economic defensibility, and one-on-one duels

. These three ingredients apply to the evolution of large weapons across the animal kingdom, as well as across historical arms races in human combat technology.

When these three ingredients cease to be in place, the arms race ends

. (This makes this book important for any EAs concerned about technology-driven X-risks, as it can help us identify causes of arms races and potential paths to ending them).

The high cost of making large weapons make them honest signals of strength, and therefore effective deterrents

. As a result, in animal species with large weapons, conflicts rarely escalate to costly, full-scale combat. For humans, this means we should focus on decreasing access to existential-risk-bearing weapon developments.

Ever

since humans developed weapons of mass destruction, we have been playing an unprecedented game

, wherein if a single pair of individuals (states) ever does choose to fight, the whole species may go extinct.

So far, deterrence has worked, but deterrence depends on large weapons being costly to obtain.

The cost of obtaining nuclear warheads has fallen significantly in recent decades, eliminating a key ingredient of deterrence

. As more and more countries obtain them, the unprecedented danger grows.

And a personal note: the author warns about the ease with which a state can obtain a nuclear warhead. I would add that developments in biosynthe...
Apocalypse Prepping - Concise SHTF guide to prepare for AGI doomsday
by prepper — 2023-07-04T17:41:41.401Z
Link: http://prepper.i2phides.me/

Greetings LessWrong community. I have written this prepping guide, to explain the threats of AGI in simple terms, and to propose the best course of action on an individual level, based on rational and logical thinking and strategies.

I am very concerned with how the topic is handled in the public sphere and on a personal level. I have identified and tried to explain many illogical fallacies that people commit to, which includes many if not most experts who give statements on this topic.

One of those fallacies is the principle to remain in inaction or to rely on herd mentality in a situation where the individual feels helpless. Another one concerns how to handle threats, depending on their magnitude, in absence of proof and certainty of knowledge. Another one would be to base your actions purely on faith into your personal beliefs about the outcome, when you cannot sufficiently rule out other outcomes to any meaningful degree.

I have seen many people argue very illogical things on this site, such that it makes no sense to concern yourself with societal collapse brought on by AGI, because it would end the world as we know it anyway. Or that we are incapable to know what to do, because we have not yet established what will truly happen.

While it might be true that some people would be content to really do nothing and invest nothing to address any risks, simply on a gamble like in Russian roulette, I don't think this suicidal mentality is what most people would recognize as an actually reasonable strategy for themselves and their family, if they only give the topic sufficient thought and consideration.

I would love to hear your feedback on my guide and have it analyzed through a rigorous logical lens as well, that fits the principles of LessWrong about logic and reason. I have been reading LessWrong from time to time, but I am not actually part of this community.

Unfortunately I have not been able to further support most of the logic proposed in this guide by some form of more academic framework to assess and manage (existential) risk, because such a framework just doesn't seem to exist.

Here is the direct onion link for faster access:

http://prepiitrg6np4tggcag4dk4juqvppsqitsvnwuobouwkwl2drlsex5qd.onion/

Edit: Please if you downvote, can you explain your rationale?