trentmkelly/LessWrong-43k huggingface dataset:
Sam Harris’s Argument For Objective Morality
by Zero Contradictions — 2024-12-05T10:19:03.701Z
Link: https://thewaywardaxolotl.blogspot.com/2024/07/sam-harriss-argument-for-objective.html

Apparently, the following is an argument made by Sam Harris on twitter, in a series of tweets. Unfortunately, the original tweets have been deleted, so I relied on a secondary source.

Let’s assume that there are no ought’s or should’s in this universe. There is only what *is*—the totality of actual (and possible) facts.
Among the myriad things that exist are conscious minds, susceptible to a vast range of actual (and possible) experiences.
Unfortunately, many experiences suck. And they don’t just suck as a matter of cultural convention or personal bias—they really and truly suck. (If you doubt this, place your hand on a hot stove and report back.)
Conscious minds are natural phenomena. Consequently, if we were to learn everything there is to know about physics, chemistry, biology, psychology, economics, etc., we would know everything there is to know about making our corner of the universe suck less.
If we *should* do anything in this life, we should avoid what really and truly sucks. (If you consider this question-begging, consult your stove, as above.)
Of course, we can be confused or mistaken about experience. Something can suck for a while, only to reveal new experiences which don’t suck at all. On these occasions we say, “At first that sucked, but it was worth it!”
We can also be selfish and shortsighted. Many solutions to our problems are zero-sum (my gain will be your loss). But *better* solutions aren’t. (By what measure of “better”? Fewer things suck.)
So what is morality? What *ought* sentient beings like ourselves do? Understand how the world works (facts), so that we can avoid what sucks (values).

Before going on, let’s pause to consider that Sam Harris is a famous public intellectual, with a BA in philosophy from Stanford and a PhD in neuroscience from UCLA.

Now, let’s consider how flawed his argument is.

The argument contains the following errors:

It begs the question. It presupposes objective good and bad.
It conflates a subjective value judgment (“it sucks”) with objective value.
It conflates pain with subjective value.

Essentially, the argument presupposes hedonism and altruism, and then pretends to derive a combination of those two assumptions (objective morality) from pure reason plus experience.

See Hedonic Utilitarianism.

Let’s go through the argument, point by point.

(see the rest of the post in the link)
Morality as Cooperation Part III: Failure Modes
by DeLesley Hutchins — 2024-12-05T09:39:27.816Z

This is a Part III of a long essay. Part I introduced the concept of morality-as-cooperation (MAC) in human societies. Part II discussed moral reasoning and introduced a framework for moral experimentation.

Part III: Failure modes

Part I described how human morality has evolved over time to become ever more sophisticated. Humans have moved from living within small tribes which were engaged in near-constant warfare, to peacefully cooperating within massive nations that contain billions of people, with laws and constitutions that protect basic human rights. But while human morality has gradually improved over time, progress has not been smooth. World War II was a catastrophic moral failure, and the subsequent cold war brought the constant threat of nuclear annihilation. Authoritarian countries still exist today, and the world's major powers seem to be backsliding towards nationalism and authoritarianism again. The evolution of morality clearly has failure modes.

Part II proposed a framework for both reasoning about morality and for doing experiments, with the goal of automatically deriving universal moral codes for both human and non-human agents, including AGI. As with any theory, though, a theory fails if any of its premises fail, which immediately suggests what some of those failure modes might be.

This section will again make heavy use of examples from human society to discuss failure modes, particularly international relations and geopolitics, and thus may seem more like a treatise on political philosophy than an essay on AI alignment. This is deliberate: nations and corporations are excellent analogues for the problems that we may face with AGI.

First of all, nations and corporations are the two main legal structures that humans use to cooperate, and thus provide a case study in cooperation. Second and more importantly, nations and corporations are both non-human legal entities which have far more resources (both computational and otherwise) than any individual human. Moreover, they are capable of self-directed action, with internal goals (such as power and profit) that may or may not align with human values in general. As a result, nations and corporations are the closest real-world analogue that we currently have to super-human AGI agents. The task of constraining national or corporate power, and studying systems for international cooperation, bears many si...
Morality as Cooperation Part II: Theory and Experiment
by DeLesley Hutchins — 2024-12-05T09:04:12.167Z

This is a Part II of a long essay. Part I introduced the concept of morality-as-cooperation (MAC), and discussed how the principle could be used to understand moral judgements in human societies. Part III will discuss failure modes.

Part II: Theory and Experiment

The prior discussion of morality was human-centric, and based on historical examples of moral values that human societies have traditionally adhered to. This section will try to expand that reasoning to derive universal moral values from first principles, following a chain of logic that any rational intelligent agent should be able to follow.

The ideas presented here were influenced by Through the Moral Maze, By Robert Kane, the Kantian categorical imperative, and a dash of social contract theory. The particular setup described here also firmly follows the principle of Morality as Cooperation (MAC), as described in Part I.

In short, we wish to show that if there are tangible benefits to cooperation, then a group of intelligent agents will be forced to derive a protocol for cooperating in order to obtain those benefits. We will call this protocol a moral code. It is possible to reason about moral codes by using hypothetical thought experiments, following a process that will be described below.

It is also possible to test moral codes using real experiments. The "best" moral codes are those which provide the highest expected value to individual agents, within a group of cooperating agents that choose to adopt the code. Just as with human morality, moral codes can change and evolve over time. As a result, it is even possible to learn better moral codes by hillclimbing, e.g. by using evolutionary algorithms or gradient descent. In other words, we can use the same techniques to answer moral and alignment questions (i.e. simple optimization methods and lots of compute) that we use to train the AI agents themselves.

Finally, moral codes operate as a social contract which agents voluntarily agree to, in order to obtain the benefits of cooperation. Thus, the agents themselves may even provide a hillclimbing mechanism, by negotiating tweaks to the moral code among themselves in order to improve their own individual fitness.

We would further like to show that good moral codes are stable Nash equilibria. And if at all possible, we would like to show that hillclimbing on moral codes will result in a stable equilibrium t...
Morality as Cooperation Part I: Humans
by DeLesley Hutchins — 2024-12-05T08:16:51.778Z

Abstract

The AI alignment problem is usually specified in terms of power and control. Given a single, solitary AGI, how can we constrain its behavior so that its actions remain aligned with human interests? Unfortunately, the answer, to a first approximation, appears to be "we can't." There are myriad reasons, but they mostly boil down to the fact that it is very hard, perhaps impossible, to devise a loss/reward function, or any other means of control, that will effectively constrain an agent that is potentially more intelligent than its creator.

However, an alternative way of viewing the problem is through the lens of cooperation, ecology, and civilization. Over thousands of years, humans have developed an increasingly sophisticated moral philosophy, based on respect for the lives and beliefs of other people. Increasingly, we extend moral judgements even to non-human animals. Murder is wrong. Genocide is wrong. Allowing other species to become extinct is wrong. As human destructive technology has advanced, we have simultaneously developed ever more sophisticated social technologies, such as laws, constitutions, courts, democracy, and non-proliferation treaties, in order to resolve conflicts, and prevent other humans from doing bad things.

Is human morality a quirk of primate evolution, and unlikely to be shared by AGI? Or are there universal moral values? If the latter, then AGI may be able to derive human-aligned moral values on its own, without the need to impose an external loss function. Instead of researching "AI alignment," perhaps we should research "social AI," with the aim of developing AGI that can reason about social interactions and moral values. A distinguishing feature of viewing the problem through a social lens is that it envisions a society, or ecology, of AI agents, cooperating both among themselves, and with humans. Morality does not exist in a vacuum; instead, it arises as an emergent property when multiple agents, which may have different goals and values, must coordinate to decide what constitutes acceptable behavior.

Emergent behaviors are notoriously difficult to engineer, and are prone to unanticipated results, so "emergent morality" will not be easy, nor does it offer iron-clad guarantees. In the absence of better alternatives, however, it seems like a path that is worth exploring. This essay explores the plausibility of emergent moral...
I Finally Worked Through Bayes' Theorem (Personal Achievement)
by keltan — 2024-12-05T02:04:16.547Z

Two years ago I found this community, which prompted me to start self teaching math. For reference, I didn't know what a fraction was in early 2022. I knew what they looked like, and what they were called. I didn't know what they meant.

The story of why I lacked basic math skills is complex enough for its own post. But my motivation to learn was simple. TSUYOKU NARITAI!

My goal was to understand Bayes' Theorem. And today, I have done that for the first time. Today, I achieved that milestone. This post chronicles the final reasoning steps that got me there.

I don't expect the average LWer will gain value from this post.

It is embarrassing to me to post this here. I admire the people in this space very much. Seeing you work problems has been critical in my learning, and learning to learn. Yet it can be daunting to stand among you all when I'm just beginning to grasp fundamentals many have mastered long ago.

If it's so embarrassing, why would I even make this post? 3 reasons.

Two years ago, I couldn't find anyone "at my level" on LW - no posts about learning math from an elementary school starting point. This is my contribution to fill that gap, This is my quick attempt to fill that void and encourage others who may be in past keltan's shoes.

I want to put more of myself into the training data where possible.

Simply put, I'm celebrating. This achievement has me all excited! I wanted to share that with this community.

Relating to 1, I feel myself really hesitating to post this draft. Maybe that is part of the reason I couldn't find anyone "on my level" two years ago. Maybe there are actually more people like me. Who also get sweaty thinking about their friends realizing they don't know what a percentage represents. For this reason, I'm forcing myself to post this now.

Below, you'll find my reasoning steps as I work through problems generated by Claude. I've tried to think "out loud" on the page as much as possible. While I don't expect most readers to gain technical insights from what follows, if you're interested in seeing simple, deliberate thinking and active application of rationalist skills - well, here you go.

Question 1: Basic Probability

[!Claude]
Perfect approach - let's test your understanding systematically, starting with basic probability concepts. Given your work with rationality and systematic thinking, I'll frame these in a way that builds up to mor

...
The Dream Machine
by sarahconstantin — 2024-12-05T00:00:05.796Z

Midjourney, “the dream machine”

I recently started working at Renaissance Philanthropy. It’s a new organization, and most people I’ve met haven’t heard of it.[1] So I thought I’d explain, in my own words and speaking for myself rather than my employers, what we (and I) are trying to do here.

Modern Medicis

The “Renaissance” in Renaissance Philanthropy is a reference to the Italian Renaissance, when wealthy patrons like the Medicis commissioned great artworks and inventions.

The idea is that “modern Medicis” — philanthropists — should be funding the great scientists and innovators of our day to tackle ambitious challenges.

RenPhil’s role is to facilitate that process: when a philanthropist wants to pursue a goal, we help them turn that into a more concrete plan, incubate and/or administer new organizations to implement that plan, and recruit the best people in the world to work on that goal and make sure they get the funding and support they need.

I like to use the Gates Foundation as an example of a really strong philanthropic organization. When Bill Gates decided he wanted to do philanthropy, he did a ton of research, decided what was important to him and what strategies he thought were effective, and built a whole new organization that he leads full-time.

But not every philanthropist is going to go that route. Some donors still work; some want to enjoy their retirement. The default path for donation, the one that takes the least effort for the donor, is to give to an existing, trusted nonprofit organization.

And that’s not necessarily bad, but it does make it hard to do new, bold, effective things.

Most philanthropy is fundamentally steady-state. Whether you’re donating to the opera or to anti-malaria bednets, you’re supporting an existing organization to do pretty much the same thing this year that they did last year.

It’s more difficult — but more interesting — to use your donations to create something in the world that did not exist before.

This can range from things like the Vesuvius Prize for digitally scanning and translating the long-lost Herculaneum Scrolls, to the Cystic Fibrosis Foundation’s “venture philanthropy” investments that funded the development of the first cystic fibrosis drugs back in the 1990’s.

On a smaller scale, ACX Grants are an example of an individual donor funding a variety of early-stage projects (from developing an open-source intra...
Should you have children? A decision framework for a crucial life choice that affects yourself, your child and the world
by Sherrinford — 2024-12-04T23:14:23.950Z

The need for a structured framework for deciding whether you should have children

In the rationality community and in the EA community, it is normal to analyze all kinds of decisions in detail, and this of course includes their ethical consequences - that is, the impact of decisions on other people. Yet even though there are a number of forum posts on the decision whether to have children, they often focus on the effects on personal productivity and resources or personal happiness of the potential parents. There are also some ethical considerations, but the effect of being brought into this world on the child itself receives surprisingly little attention. In general, there does not seem to be a place where all the factors of the "should you have children" (SYHC) question are brought together in structured way. This can lead to implicit normative assumptions and vagues ideas heavily influencing discussions.

Deciding whether to have children is one of the most important decisions in your life as it will require a large amount of your time and financial resources and will likely be a life-changing experience that will transform your worldview and preferences.[1] Of course, not having children can also drain your resources because, depending on your personality and values, you might feel incomplete and unhappy.

With this post, I aim to organize the SYHC discussions and offer a framework for the considerations that are necessary for them, in order to enable a more structured and rational discussion for those who are interested in the topic.

First, I discuss how to approach the question as a rational decision, and discuss the considerations that should, arguably, be part of a rational decision on whether to have kids. I split this into considerations regarding the parents, the child, and the world, discuss how these considerations relate and which role time horizons, discounting and timing play. After an interim summary, I discuss failure modes of rational discourse and rational decisionmaking in this context, with one particular fallacy that I suspect plays an important role in the SYHC decision. I also mention some alternatives to having children.

My conclusions do not include an object-level answer to the question whether you should have kids. In particular, this post is not intended to say that you should not have kids or that you can easily sum costs and bene...
CCing Mailing Lists on External Communication
by jefftk — 2024-12-04T22:00:02.038Z

In my current work, and at various previous companies, I've done a lot
of coordination with external groups. We generally use email for
this: it's very standard and works well enough.

One place where it doesn't work very well, though, is making it
possible for others working on a project to read and find relevant
threads. For example, if I write to our partner I can CC coworkers
who are currently on the project, but what if my boss is interested?
What about other people who might join the project later? What if I
want to share a link to the conversation with a co-worker?

My preferred solution here is email lists. For example, if I were at
ExampleCorp working on Project Cabbage with OtherCorp, I would create
a cabbage-other@example.com list and CC it on all my Cabbage-related
messages to OtherCorp. If the Cabbage-Other collaboration isn't
sensitive I might have the list set so anyone at ExampleCorp can join,
or I might open it only to specific people at ExampleCorp. (This is
overkill if Cabbage is a small project; then I'd just use
cabbage-collaborations@ or some other single address for everything
external Cabbage-related.)

This also makes CCing someone a much clearer signal: without this,
it's hard to tell the difference between being CC'd on a message "so
you can read it if you want to" vs "because you probably should be
following this thread." But if I receive a message because I'm on the
cabbage-other@ list that's not something I necessarily need to read,
while if I'm explicitly listed in the CC line I'll pay more attention.

I can also tell my mail client about this: I'll subscribe to all of
these lists but set them to skip my inbox if I'm not specifically
mentioned. [1] Then they show up in my Gmail search, and if I do
later want to reply to one (ex: we're moving work around; someone's
out) it's easy.

[1] In Gmail: on a message from the list click "filter messages like
this", verify it has a "has the words" of something like
list:(...), expand that to append -to:me
-cc:me, click "create filter, check "skip the inbox".
Picking favourites is hard
by dkl9 — 2024-12-04T20:46:47.470Z
Link: https://dkl9.net/essays/favourites.html

If you ask for my "favourite", or whatever is "best", in any broad domain, I will refuse to answer, or else give an answer I know to be probably wrong.

Goodness-judgments are fuzzy, i.e. subjective and intuitive. Fuzzy values only compare one way or the other when values compared are far enough apart. The more items there are to compare in a set, the closer together their respective qualities will be, on average. Hence, as sets get larger, fuzzy comparisons among its members get less reliable.

Something confidently known to be best in its category is (definitionally) known to be strictly better than each other item in its category. So we can only be certain about the best item in a category if the category is small, or if we expend great effort to compare items.

More precisely, suppose you ask about a domain of n objects, with qualities (x1 thru xn) sampled from a normal distribution. In the average scenario, the normal CDF of x1 thru xn is evenly-spaced by 1n. To find the best item, we must (at worst) compare best to second-best. That entails comparing inverse CDF values at 2n−12n to those at 2n−32n.

Comparing is difficult in inverse proportion to the difference. As n increases, that difference in inverse CDF values approaches 12.

Questions about favourites have included foods, neighbours, and programming languages. In each case, I'm familiar with dozens of options. That large n prohibits finding a confidently, correctly-known best option in any convenient timescale.

"X" isn't [my favourite] word, but it's the first word that comes to mind.

— Chuck Palahniuk, approximately

What I can do instead is alter the question from "which is your favourite?" to "which is distinctly good?", picking an option which compares as either greater-than or ambiguous against each other option, and so is in the top few.
How can I convince my cryptobro friend that S&P500 is efficient?
by AhmedNeedsATherapist — 2024-12-04T20:04:44.521Z

Just to clarify, we are both high schoolers. No one is at risk of losing a bunch of money here :)
I have a friend who's fascinated by cryptocurrency trading, especially with recent events like crypto going up. He doesn't actually invest in crypto though (that would be illegal). But he does spend a lot of time running trading simulations of the S&P500 using simple strategies (Claude's wording of this post described it as "things like setting buy/sell limits based on moving averages that could be implemented with basic Python code").

I want to convey the idea of market efficiency to him, but his unintuition is unintuitive to me. I don't know what building blocks he would need to understand market efficiency.

1.
How do you develop an intuition for why strategies that look good after-the-fact might not actually be good a priori? Do I bother explaining overfitting, for instance?

2.
How do I communicate that PhDs with way more computer resources are going to "eat up" any market inefficiency? How do I convince him that this isn't theoretical nonsense?

3. There seems to be something fundamental about market efficiency that's hard to grasp until you've developed certain intuitions - like understanding that you can't simply exploit a market inefficiency to generate unbounded returns, even with unbounded capital. How does one develop these intuitions?

4. Should I even try to discourage this? Perhaps paper trading in this way, even if it's just peering at noise, has educational value? On the contrary, peering at noise seems to neither discourage gamblers nor provide substantial educational value for them.

This post co-written by Claude. The original draft from Claude is viewable here.