trentmkelly/LessWrong-43k huggingface dataset:
How Money Fails to Track Value
by JustinShovelain — 2022-04-02T12:32:35.968Z

This post was finished for Good Heart Week and speaks to perhaps the most typical context for Goodharting.

Context: Theory of Money

Money is one of the largest factors in determining what gets done in the world and a clearer understanding of it and how it interacts with what we value may pay dividend$.

It is commonly claimed that money is an estimate of value and some seem to even think that

it

is

value

. While clearly this isn’t entirely true, let's examine it in more detail, and see what money really is and what it really tracks.

What is money? At first glance it seems to be something you can convert into arbitrary

goods

and services by paying for things. In that sense it seems kind of similar to, say, having iron ore that you may convert into nails. It’s a bit more subtle though. Firstly it’s something you may convert into practically anything, a kind of universal resource. For another, it isn’t itself generally destroyed in the process and is instead simply exchanged: money moves towards the person giving the good or service and away from the person receiving the good or service. It is a kind of resource that can be created from nothing or destroyed and be of any number from positive to negative infinity. Finally, it is supported by a kind of universal belief and convention; if you fail to believe in it, then it ceases to exist, and the belief that exists is distributed amongst all the people involved in transactions. Given how it works I would say it is the natural generalization of an IOU or social status change and probably originated in that (

History of money

).

From this then it’s reasonable to say that there are five significant operations on money:

Getting it in exchange for goods or services
Giving it to receive goods or services
Storing it for future use like you may most resources
Creating it
Deleting it

Giving these operations, it then looks like money is a kind of anti resource that you exchange for other resources; when you get a good or service that act annihilates a certain amount of money for you. In some derivative-like sense it is the opposite of the change in what you got in exchange.

How do these money calculations then ground out so you get concrete prices? They ground out in the decisions of people to exchange it for things they value intrinsically, the brute facts of resource conversion efficiency, and...
I'm writing a novel to promote Effective Altruism
by Timothy Underwood — 2022-04-02T08:50:22.438Z

I’m currently writing a fantasy novel to encourage people to join the EA community.

I’ve made a living self publishing novels for years (mainly Pride and Prejudice fan fiction, but also two litrpg-esque works), and I’ve been interested in writing something to directly promote Effective Altruism for a while.

I applied for one of Scott Alexander’s ACX grants to fund taking three - five months off from other projects to write this, and I received an ACX plus grant. Now that I’m actively developing the story, I’m hoping to get some thoughts from the community on ideas (especially about AI safety and regulation) that are important to signal boost/ treat as info hazards. Also, I think it would be cool to try brainstorming and bouncing my ideas off of people who are interested in this sort of fiction outreach project. I’ve never really showed my work to people who might meaningfully critique it while it was still in the outline phase, and I’d like to see if I get a lot of value from doing that.

I’ve set up a discord to talk about the story in, and here is a link to a shared google doc that has my outline. It’s

here

— if you are quick you might be the first person there who isn’t me. Come say hi!

Here

is a link to my current outline if you want to look at it and leave comments:

Also, I’m planning on having a discussion event on the discord server on Tuesday April 5 at 6 pm Central European Time (which is, I’m afraid, 9 am in California). The plan is for me to talk a bit more about the project, and then talk about any questions or ideas people have. So it would be awesome if you join, and even if you are just vaguely curious, I’d love you to come.

The theory underlying why writing a novel like this is a good idea has two components:

HPMOR, Atlas Shrugged, and a variety of books that did not influence me personally such as The Alchemist, Who Moved the Cheese, or Ishmael so the value of works of fiction in getting people to take ideas seriously, start talking about issues, and engage with a community.
A good novel for exposing people to these ideas needs to be successful as a novel, and hence writing something that targets an existing market niche makes sense.

Two years ago I actually wrote a whole essay about the topic of using fiction to change the world that I cross posted on the

EA forum

and

Less Wrong

.

I’m looking for some help in terms of what...
Good Heart Week: Extending the Experiment
by Ben Pace — 2022-04-02T07:13:48.353Z

The Good Heart Leaderboard, at the time of writing.

Yesterday we launched Good Heart Tokens, and said they could be exchanged for 1 USD each.

Today I'm here to tell you: this is actually happening and it will last a week. You will get a payout if you give us a PayPal/ETH address or name a charity of your choosing.

Note that voting rings and fundraising are now out of scope, we will be removing and banning users who do that kind of thing starting now. More on this at the end of the post.

Also, we're tentatively changing posts to be worth 4x the Good Heart Tokens of comments (Update: we decided on 3x instead, and it just went live, at around 4:20PM PT on April 2nd).

Why is this experiment continuing?

Let me state the obvious: if this new system were to last for many months or years, I expect these financial rewards would change the site culture for the worse. It would select on pretty different motives for being here, and importantly select on different people who are doing the voting, and then the game would be up.

(Also I would spend a lot of my life catching people explicitly trying to game the system.)

However, while granting this, I suspect that in the short run giving LessWrong members and lurkers a stronger incentive than usual to write well-received stuff has the potential to be great for the site.

For instance, I think the effect yesterday on site regulars was pretty good. I'll quote AprilSR who said:

I am not very good at directing my monkey brain, so it helped a lot that my System 1 really anticipated getting money from spending time on LessWrong today.

...There’s probably better systems than “literally give out $1/karma” but it’s surprisingly effective at motivating me in particular in ways that other things which have been tried very much aren’t.

I think lots of people wrote good stuff, much more than a normal day. Personally my favorite thing that happened due to this yesterday was when people published a bunch of their drafts that had been sitting around, some of which I thought were excellent. I hope this will be a kick for many people to actually sit down and write that post they've had in their heads for a while.

(I certainly don't think money will be a motivator for all people, but I suspect it is true for enough that it will be worth it for us given the Lightcone Infrastructure team's value of money.)

I'm really interested to find out what happens over a...
Sydney AI Safety Fellowship Review
by Chris_Leong — 2022-04-02T07:11:45.130Z

The Sydney AI Safety Fellowship was a 7-week program with a coworking space, speakers, mentors and social activities. The idea was for each participant to have a "project", but only one had a research project, whilst the others simply dedicated their time towards figuring out their plans for the future. More specifically, they were thinking about whether they wanted to shift their careers or prospective PhDs towards AI safety and if so what they focus on and what would be their next step.

Inspiration:

There were four main influences:

I interviewed on improving the AI Safety Pipeline and I noticed that EAs in more cities seemed to be setting up coworking spaces.
Kat Woods suggested that there should be a lightweight research agency and I modified the idea to be in-person as I'm not really a fan of remote programs and I shortened it to a summer program as a minimal viable product.
JJ hosted an AI Safety meet-up at WeWork and I noticed that the space was really nice and I heard that WeWork was offering a discount on coworking memberships which meant that I could potentially self-fund this.
Michael Aird indicated that mentorship was crucial for a research development program and since I had no idea who we'd be able to find as mentors[1], I pivoted hard away from this. We advertised mostly non-research projects, but we indicated that we would accept research projects from people who already have proven research experience[2].

Target Audience:

Given that it is a 7-week program the natural target audience is students or people in the middle of a career transition. I'm thinking of complementing this program with a retreat later in the year to suit busy professionals so that our local activities cover different market segments.

Last Minute Organising:

Unfortunately, this idea came to me quite late and since the university summer break is fixed, I couldn't have pushed it back without having to wait until next year. We also didn't know until very late whether we were going to be funded by the Long Term Future Fund (LTFF) or not. The result was that this fellowship was organised at the last minute.

I expect this suppressed applications because many potential candidates would already have lined up something else, but also because our initial advertising was very vague about how much funding we'd be able to provide to participants. I made a mistake here by being too reluctant to sen...
Community Building: Micro vs. Macro
by Chris_Leong — 2022-04-02T07:10:57.216Z

We can identify two different[1] approaches to community building. The micro approach involves deeply engaging[2] with individuals in order to change their views or to assist in their journey. When adopting this approach, programs tend to be flexible, with guidelines rather than rules and decisions sometimes being driven by the needs of individuals. This approach puts participants at the center.

The macro approach involves a level of standardization in order to ensure consistency and to keep the program manageable. While exceptions may be made to rules there is more of an effort to limit these exceptions in order to limit deliberation time if nothing else. Individual accommodations will likely need to be limited given that organizers only have so much time. This approach is typically is centered more around the needs of organizers.

Here are some examples of decisions that would make more sense from a micro approach than a macro approach[3]

Asking people who have expressed interest in the ideas you're sharing whether they want to grab a coffee. Or offering to have a call with anyone who wants to know more about a program.
Moving lunch an hour earlier than originally proposed so a participant can make a lecture.
Offering to cover a participant's plane ticket personally in order to encourage them to attend a program.
Choosing mentors based on individual interests rather than just choosing a group of generally competent mentors.
Individually helping participants who miss part of a program catch up on key the points they may have missed.
In response to a participant expressing a keen interest in reinforcement learning and asking if there will be any talks on the subject, saying that you can't make any guarantees, but promising to try your best to find someone[4].
Increasing the funding available to all participants in order to sway one who is on the fence, whilst also being fair to the others[5].
Learning each person's individual interests and providing personalised recommendations of things to check out or people connect with.

Here are some examples of decisions that would make sense from a macro approach:

Requiring participants to attend 6 out of 8 sessions in order to graduate from a program, strictly limiting exceptions to medical reasons.
Fixing a formula for the amount of funding provided to each participant.
Politely refusing to reschedule a session because re

...
Inflection AI: New startup related to language models
by Nisan — 2022-04-02T05:35:24.759Z

Inflection AI (website, announcement) is a new startup founded by Mustafa Suleyman, Reid Hoffman, and Karén Simonyan. It appears they'll be doing something with language models. They're also looking to hire a safety team; I bet they'll need to do RLHF as well as toxicity filtering, etc.
How can a layman contribute to AI Alignment efforts, given shorter timeline/doomier scenarios?
by AprilSR — 2022-04-02T04:34:47.154Z

Versions of this question have probably been asked before, but MIRI at least is getting more pessimistic and I’ve seen Eliezer express multiple times that he doesn’t know what actually useful advice to give people who aren’t in the field yet.

I don’t want the world to end, but I am only a decently intelligent person of medicore competence. I could try to read and grok alignment research until I can have productive thoughts, but I do not anticipate that helping (though I will probably start doing more reading anyways; I’m tentatively planning to try reading Jaynes). Should I go into some kind of advocacy? I don’t know how I would do productive work there either, really.

I would guess there are others in a roughly similar situation to mine. Does anyone have ideas?
April 2022 Welcome & Open Thread
by Ben Pace — 2022-04-02T03:46:13.743Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the new Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
A Summary of "Four Thousand Weeks" by Oliver Burkeman
by Magnus — 2022-04-02T02:17:22.376Z

This is my first post here, so please don't bully me. I know all you math-boys with your big, wrinkly brains could do me a serious bruising. I beg thee mercy, for I am but a first time poster.

First things first - Oliver boy, I am on to you. I see what you've done here, disguising your work of legitimate philosophy in self-help's clothing.

Just when I thought I could listen to another audiobook telling me to take cold showers, drink green juice, and meditate all with a "scientists have even discovered..." sprinkled atop, you snuck one past me.

Not only did you avoid mentioning green juice even once, but you avoided laundry lists of life-hacks all together. Instead, you decided to write a thought-provoking analysis of our finitude relying only on appeals to my common sense, using only observations that any typical human would have.

All I was looking for were some new platitudes to add to my "Inspirational Quotes" document, but you just had to really educate me on how think about time management. You had to make me change my opinions on commitment, pacing, rest, and time's importance in being human.

I'll let you get away with it this time, educating me, changing my perspective for the better. I just hope next time you can write something that promises me the impossible via a series of bizarre rituals. Improving my work life balance by eating more broccoli, maybe? Up to you.

In all seriousness, this is a really good book. The audiobook is about six hours. I would recommend it.

I've included my summary of the book below. I've broken it down into a few different categories. The summary is chronological.

Finitude

You have to face finitude, pick a few things to dive deep on, and really try to distance yourself from distractions.
You have to cherish the simple fact you're alive. The time you've been given in limited, but at least it's there at all. You didn't have to be here, don't take your time for granted.

Commitment

Committing is as much a positive action as it is a negative. You are doing something as much as *you will not be doing other things.*
Also, you will be destroying the perfect fantasies in your head about what could be. So commitment is both a negative and destructive act.
Three tenants of commitment

Do your most important task first.
Limit works in progress. Do not have ten projects going simultaneously. Yo

...
Optimality is the tiger, and agents are its teeth
by Veedrac — 2022-04-02T00:46:27.138Z

You've done it. You've built the machine.

You've read the AI safety arguments and you aren't stupid, so you've made sure you've mitigated all the reasons people are worried your system could be dangerous, but it wasn't so hard to do. AI safety seems a tractable concern. You've built a useful and intelligent system that operates along limited lines, with specifically placed deficiencies in its mental faculties that cleanly prevent it from being able to do unboundedly harmful things. You think.

After all, your system is just a GPT, a pre-trained predictive text model. The model is intuitively smart—it probably has a good standard deviation or two better intuition than any human that has ever lived—and it's fairly cheap to run, but it is just a cleverly tweaked GPT, not an agent that has any reason to go out into the real world and do bad things upon it.

It doesn't have any wants. A tuned GPT system will answer your questions to the best of its ability because that's what it's trained to do, but it will only answer to the best of its abilities, as it doesn't have any side-goals to become better at doing that in the future. Nowhere is the model motivated to gather more resources to become a better thinker. There was never an opportunity during training to meta-learn that skill, because it was never the optimal thing for the model to be when it was trained.
It doesn't plan. GPTs have no memories. Its mental time span is precisely one forward pass through the network, which at a depth of a few thousand means it can never come up with anything that requires more than the equivalent of maybe 10-ish human-time equivalent coherent seconds of thought at once. There is a fearful worry that perhaps the model could start forming plans that span multiple instantiations, using one output to feed into the next input, but it's a text-prediction model, and that's directly at odds with its trained goal. The system was trained primarily by asking it to maximize actual probabilities of actual texts, where such a skill would never be useful, and only fine-tuned in the autoregressive regime, in a way that held most of the model parameters fixed. It would be a stretch to assume the model could develop such sophisticated behaviors in such a small fraction of its training time, a further stretch that it could be done while training such a reduced fraction of the model, and an even g

...
MIRI announces new "Death With Dignity" strategy
by Eliezer Yudkowsky — 2022-04-02T00:43:19.814Z

tl;dr: It's obvious at this point that humanity isn't going to solve the alignment problem, or even try very hard, or even go out with much of a fight. Since survival is unattainable, we should shift the focus of our efforts to helping humanity die with with slightly more dignity.

Well, let's be frank here. MIRI didn't solve AGI alignment and at least knows that it didn't. Paul Christiano's incredibly complicated schemes have no chance of working in real life before DeepMind destroys the world. Chris Olah's transparency work, at current rates of progress, will at best let somebody at DeepMind give a highly speculative warning about how the current set of enormous inscrutable tensors, inside a system that was recompiled three weeks ago and has now been training by gradient descent for 20 days, might possibly be planning to start trying to deceive its operators.

Management will then ask what they're supposed to do about that.

Whoever detected the warning sign will say that there isn't anything known they can do about that. Just because you can see the system might be planning to kill you, doesn't mean that there's any known way to build a system that won't do that. Management will then decide not to shut down the project - because it's not certain that the intention was really there or that the AGI will really follow through, because other AGI projects are hard on their heels, because if all those gloomy prophecies are true then there's nothing anybody can do about it anyways. Pretty soon that troublesome error signal will vanish.

When Earth's prospects are that far underwater in the basement of the logistic success curve, it may be hard to feel motivated about continuing to fight, since doubling our chances of survival will only take them from 0% to 0%.

That's why I would suggest reframing the problem - especially on an emotional level - to helping humanity die with dignity, or rather, since even this goal is realistically unattainable at this point, die with slightly more dignity than would otherwise be counterfactually obtained.

Consider the world if Chris Olah had never existed. It's then much more likely that nobody will even try and fail to adapt Olah's methodologies to try and read complicated facts about internal intentions and future plans, out of whatever enormous inscrutable tensors are being integrate...