trentmkelly/LessWrong-43k huggingface dataset:
What are you getting paid in?
by Austin Chen — 2024-07-17T19:23:04.219Z
Link: https://www.approachwithalacrity.com/what-are-you-getting-paid-in/

Crossposting this essay by my friend, Leila Clark.

A long time ago, a manager friend of mine wrote a book to collect his years of wisdom. He never published it, which is a shame because it was full of interesting insights. One that I think a lot about today was the question: “How are you paying your team?”

This friend worked in finance. You might think that people in finance, like most people, are paid in money. But it turns out that even in finance, you can’t actually always pay and motivate people with just money.

Often, there might just not be money to go around. Even if there is, managers are often captive to salary caps and performance bands. In any case, it’s awkward to pay one person ten times more than another, even if one person is clearly contributing ten times more than the other (many such cases exist).

With this question, my manager friend wanted to point out that you can pay people in lots of currencies. Among other things, you can pay them in quality of life, prestige, status, impact, influence, mentorship, power, autonomy, meaning, great teammates, stability and fun. And in fact most people don’t just want to be paid in money — they want to be paid some mixture of these things.

To demonstrate this point, take musicians and financiers.

A successful financier is much, much richer in dollars than a successful musician. Some googling suggests that Mitski and Grimes, both very successful alternative musicians, have net worths of about $3-5m. $5m is

barely notable

in the New York high society circles that most financiers run in. Even Taylor Swift, maybe one of the most successful musicians of all times, has a net worth of generously $1b; Ken Griffin, one of the most successful financiers of all time, has a net worth of $33b.

But more people want to be musicians, and I think it’s because musicians are paid in ways that financiers aren’t.

Most obviously, musicians are way cooler. They get to interact with their fans. People love their work. They naturally spend their days hanging out with other cool people – other musicians. They can work on exactly what they want to, largely when they want to – they’ve won the American Dream because they get to work on what they love and get paid! And in that way, they get paid in radical self-expression. (This is a little unfair, because I know some financiers who think that work is a means of radical self-expr...
Individually incentivized safe Pareto improvements in open-source bargaining
by Nicolas Macé — 2024-07-17T18:26:43.619Z

Summary

Agents might fail to peacefully trade in high-stakes negotiations. Such bargaining failures can have catastrophic consequences, including great power conflicts, and AI flash wars. This post is a distillation of DiGiovanni et al. (2024) (DCM), whose central result is that agents that are sufficiently transparent to each other have individual incentives to avoid catastrophic bargaining failures. More precisely, DCM constructs strategies that are plausibly individually incentivized, and, if adopted by all, guarantee each player no less than their least preferred trade outcome. Figure 0 below illustrates this.

Figure 0: Payoff space representation of a bargaining scenario. Horizontal (vertical) coordinate: payoff to the first (second) agent. In this toy scenario, two agents can either trade (outcomes on the thick black segment) or wage a costly war (red point). Although going to war hurts everyone, each player might want to accept the risk of war to improve their bargaining position. However, we’ll show how to construct individually incentivized strategies that, if adopted by both, can guarantee each player is no worse off than under their least preferred trade outcome (star).

This result is significant because artificial general intelligences (AGIs) might (i) be involved in high-stakes negotiations, (ii) be designed with the capabilities required for the type of strategy we’ll present, and (iii) bargain poorly by default (since bargaining competence isn’t necessarily a direct corollary of intelligence-relevant capabilities).

Introduction

Early AGIs might fail to make compatible demands with each other in high-stakes negotiations (we call this a “bargaining failure”). Bargaining failures can have catastrophic consequences, including great power conflicts, or AI triggering a flash war. More generally, a “bargaining problem” is when multiple agents need to determine how to divide value among themselves.

Early AGIs might possess insufficient bargaining skills because intelligence-relevant capabilities don’t necessarily imply these skills: For instance, being skilled at avoiding bargaining failures might not be necessary for taking over. Another problem is that there might be no single rational way to act in a given multi-agent interaction. Even arbitrarily capable agents might have different priors, or different approa...
Profit and Value
by kwang — 2024-07-17T18:06:57.048Z
Link: https://open.substack.com/pub/kevw/p/14-profit-and-value?r=ufp7i&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

In 1950, the eponymous founder George Wilhelm Merck said that “We try never to forget that medicine is for the people. It is not for the profits. The profits follow, and if we have remembered that, they have never failed to appear.”

In 1970, Milton Friedman wrote an NYT column titled “The Social Responsibility of Business Is to Increase Its Profits”

In the same historical era, Nun-CEO Sister Irene Kraus coined an apparent synthesis: “No margin, no mission”.

Here are some claims I think are true about Profit and Value:

Profit and Value aren’t the same

The difference between capitalism and shmapitalism, as I call it, is in precisely whether Profit or Value is more important:

Capitalists view organizations as financial instruments with which they maximize Profit. Capitalists are concerned first and foremost with extracting Profit. Capitalists seek to maximize the Profit they capture. Capitalists sacrifice their long term sustainability in order to increase near term Profit. Capitalists use Profit to maximize the Profit they extract. A capitalist justifies a human life by its economic value.
Shmapitalists view organizations as generative instruments with which they maximize Value. Shmapitalists are concerned first and foremost with delivering Value. Shmapitalists ensure their long term sustainability at the expense of their near term Profit. Shmapitalists seek to maximize the Value they deliver. Shmapitalists use Profit to maximize the Value they deliver. A shmapitalist grounds economic value in the human lives which constitute it.

Boeing’s ongoing troubles are perhaps the best case study to understand this difference (1, 2, 3, 4). Boeing chases Profit at the expense of Value when it minimizes costs at the expense of passenger safety. Boeing helps us understand the failure of asserting Profit as ultimately and intrinsically valuable.

Profit and Value can sometimes align

Profit and Value can’t be the same if they can be traded off against each other.

So behold, my friends: yet another 2x2. This one is of things that humans do.

A capitalist promotes Profit as the highest good, and profitable activities as good, whether or not they’re valuable.

The neutrality of capitalist ends suffuses the capitalist’s means. A pure capitalist is equally happy to profit by delivering value or by committing fraud. Capitalism by itself won’t stop them either way. As capitalists, we pr...
So You've Learned To Teleport by Tom Scott
by landscape_kiwi — 2024-07-17T18:04:37.272Z
Link: https://www.youtube.com/watch?v=zJt8yzR2aoY

Tom Scott video on creative uses of teleportation (if you were a superhero). It is from August 20, 2015 and there have been no similar videos despite it claiming to be Episode 8 at t=16s.

It reminded me of HPMOR when I saw it.
How does generalized accessibility compare to targeted accessibility?
by ErioirE — 2024-07-17T17:07:09.829Z

Status:
I don't have much to give in the way of answers. I try asking useful questions.
How do we efficiently determine the optimal amount of resources to allocate for accessibility purposes in public spending/business requirements?
What proportion of the general population is [blind|deaf|wheelchair bound|highly allergic to common additive], and are we currently doing an appropriate amount of accommodation for those things relative to the costs of doing so?

A city I've lived near has several large pedestrian bridges crossing a few major streets, some or most of them near schools. I'm sure these provide some utility in convenience and safety but I question the efficiency of their cost, particularly because they were built with huge switchback ramps rather than stairs, increasing the required amount of land+material significantly.
Now, while a ramp is accessible to both those walking as well as bicycles and wheelchairs and in most cases provides a better service, is that sufficient to justify the increased cost of a ramp vs stairs?
While a completely objective decision cannot be easily made here because different utility functions give variable valuations, there are some variables we can meaningfully quantify to inform our subjective evaluation of the situation.
Prior to something like this being built, the city planners (hopefully) made some considerations such as:
What proportion of those in the area near the road are likely to use a bridge if one is made?
Of those, how many are likely to use bicycles on it or need wheelchair accessibility?
Did they assign a dollar value for how much they are willing to spend for how large of a proportion of the population they'd be covering? If not, I have a sinking suspicion (Estimation P>(0.50)) that the difference between the initial + maintenance (cost of bridge with ramp - cost of bridge with stairs) is greater than the (cost of providing para-transit for every wheelchair-bound person who is likely to need the bridge in its expected use-time[1])[2].

Then again, anything touching bureaucracy is plausibly more expensive than my intuition first expects, and maybe the ramp-bridge actually is cheaper.
My prior for the primary decision making factor being a utilitarian breakdown at least this good[3] is a lot lower than the prior that it was motivated by the desire to signal how thoughtful and magnanimous the City Leaders ...
Housing Roundup #9: Restricting Supply
by Zvi — 2024-07-17T12:50:05.321Z

I’d split the latest housing roundup into local versus global questions. I was planning on waiting a bit between them.

Then Joe Biden decided to propose a version of the worst possible thing.

So I guess here we are.

What is the organizing principle of Bidenomics?

Restrict Supply and Subsidize Demand (1)

This was the old counterproductive Biden proposal:

Unusual Whales: Biden to propose $5,000 credit for first-time home buyers, per WaPo.

The Rich: House prices about to go up $5,000 everywhere.

Under current conditions this is almost a pure regressive tax, a transfer from those too poor to own a home to those who can afford to buy one, or who previously owned one and no longer do.

If there were no restrictions on the supply of housing, such that the price of a house equalled the cost of construction plus the underlying price of land, then the subsidy would work, but also you would not need it.

Yes, this implies that many good policy interventions are available.

Biden then literally doubled down on this proposal, which Tyler Cowen correctly labeled ‘from the era of insane economic policy’: Joe Biden proposes to pay new homeowners $9600 of taxpayer money at $400/month for two years. Why are we transferring money from existing homeowners and those who cannot afford to home to this particular middle group?

To be fair to Joe Biden here, and I initially made this mistake as well, no prices will not rise by almost $9600 per house allowing existing homeowners to capture all the loot in an accounting sense. Only about half of home buyers are first time purchasers, and by buying they are using up a valuable one-time resource. So my guess would be this only increases home values by about $5000.

Also note that if all home values go up by $5000, that does not mean that homeowners are that much wealthier unless they own more house than they need. If you live in one house and own one house, you are closer to economically flat housing than to being long one house. You have to live somewhere.

I also note that Biden’s proposal of $400/month for two years means that the first time homebuyer likely cannot use that $9600 as part of their down payment on the house. So the one actual plausible thing this might have done, to allow first time buyers to assemble their down payment, doesn’t even work. Madness.

Here is the best counterargument I have seen.

Zack Kanter:

...
We ran an AI safety conference in Tokyo. It went really well. Come next year!
by Blaine — 2024-07-17T06:55:39.620Z

Abstract

Technical AI Safety 2024 (TAIS 2024)

was a conference organised by

AI Safety 東京

and

Noeon Research

, in collaboration with

Reaktor Japan

,

AI Alignment Network

and

AI Industry Foundation

. You may have heard of us through ACX.

The goals of the conference were

demonstrate the practice of technical safety research to Japanese researchers new to the field
share ideas among established technical safety researchers
establish a good international reputation for AI Safety 東京 and Noeon Research
establish a Schelling conference for people working in technical safety

We sent out a survey after the conference to get feedback from attendees on whether or not we achieved those goals. We certainly achieved goals 1, 2 and 3; goal 4 remains to be seen. In this post we give more details about the conference, share results from the feedback survey, and announce our intentions to run another conference next year.

Okay but like, what was TAIS 2024?

Technical AI Safety 2024 (TAIS 2024)

was a small non-archival open academic conference structured as a lecture series. It ran over the course of 2 days from April 5th–6th 2024 at the International Conference Hall of the Plaza Heisei in Odaiba, Tokyo.

We had

18 talks

covering 6 research agendas in technical AI safety:

Mechanistic Interpretability

Developmental Interpretability

Scaleable Oversight
Agent Foundations

Causal Incentives
ALIFE

…including talks from Hoagy Cunningham (Anthropic), Noah Y. Siegel (DeepMind), Manuel Baltieri (Araya), Dan Hendrycks (CAIS), Scott Emmons (CHAI), Ryan Kidd (MATS), James Fox (LISA), and Jesse Hoogland and Stan van Wingerden (Timaeus).

In addition to our invited talks, we had 25

submissions

, of which 19 were deemed relevant for presentation. 5 were offered talk slots, and we arranged a poster session to accommodate the remaining 14. In the end, 7 people presented posters, 5 in person and 2 in absentia. Our best poster award was won jointly by Fazl Berez for

Large Language Models Relearn Removed Concepts

and Alex Spies for

Structured Representations in Maze-Solving Transformers

.

We had 105 in-person attendees (including the speakers). Our live streams had around 400 unique viewers, and maxed out at 18 concurrent viewers.

Recordings of the conference talks are hosted on

our youtube channel

.

How did it go?

Very well, thanks for asking!

We sent out a feedback survey a...
Agency in Politics
by Martin Sustrik — 2024-07-17T05:30:01.873Z

This is a crosspost from https://250bpm.substack.com/p/agency-in-politics

Matt Yglesias explains why history is not a steam roller with a steering wheel welded tight, unyielding and incorrigible.

Of course, only a very naive person would see history as the unfolding of random occurrences driven purely by individual choices. But I think sophisticated people tend to overcorrect. Once something happens — like, for example, Joe Biden getting himself renominated — smart people are often eager to explain how this was "always going to happen."

This is, of course, at its worst, the historicism criticized by Popper, the idea that history is a deterministic process following a predetermined plan. Marx famously refrained from suggesting how a future communist society should function because if progress is driven by historical necessity, then the system is going to be what it's going to be and there's no point of trying to change the unevitable.

Yglesias offers numerous examples of individual decisions that have changed history:

Roosevelt was only 50 years old and he was also, to the extent we could measure public opinion at that time, very popular. There was no constitutional prohibition on him running for a third term. [...] But he decided to stand down and more or less handpicked Taft as his successor.

Taft lost, and Roosevelt’s decision not to run thus altered the course of history.

Even more strikingly, if Franz Joseph had decided to cede some contested Austrian territories to Italy in 1913, Italy might have joined the Central Powers as it promised to. This could have altered the timing of war, potentially preventing the Bolshevik Revolution in Russia and changing subsequent historical events.

For want of nail the horseshoe was lost, for want of horseshoe the rider was lost, for want or rider the battle was lost, because of the battle the kingdom was lost.

On the other hand, Kurt Vonnegut writes:

During the Vietnam War... every respectable artist in this country was against the war. It was like a laser beam. We were all aimed in the same direction. The power of this weapon turns out to be that of a custard pie dropped from a stepladder six feet high.

History is sometimes maleable, sometimes it is not.

But what does that mean exactly? How does it work? And how to think about it?

Here's my intuition pump.

History is a chaotic process, non-linear, full of feedback loops. To get a s...
Arrakis - A toolkit to conduct, track and visualize mechanistic interpretability experiments.
by Yash Srivastava — 2024-07-17T02:02:50.492Z

Introduction

“The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge.” -Daniel J. Boorstin

Understanding how we think is a question that has perplexed us for a long time. There have been countless theories, many thought experiments, and a lot of experiments to try to unravel the mysteries of the brain. With time, we have become more aware of our brain’s working, and in my honest, and a little biased opinion, Artificial Intelligence has come the closest to model our mystery organ.

This is one of the reasons why Interpretability as a field makes a lot sense to me. It tries to unravel the inner working of one of the most successful proxy of human brain - Large Language Model. Mechanistic Interpretability is on the approach in AI alignment to reverse engineer neural networks and understand the inner workings.

Although this field is really exciting(and challenging), researchers have made quite a significant progress in coming up with hypothesis and conducting experiments to prove their validity. Heavyweights like Anthropic, Google Deepmind, EleutherAI, Leap Labs and many open source organizations have been pushing the boundaries of knowledge in this field, and many researchers are pivoting to Interpretability to advance this field. With recent works breakthroughs in the field such as Dictionary Learning by the Anthropic team, there is a lot of scope in MI

What is Arrakis and Why do we need it?

Arrakis is one such tool which I've been making from the past four months(even longer if I include the ideation period) for the community to provide a means for researchers working in the MI to quickly iterate on their ideas and run experiments. Arrakis is a complete suite to conduct MI experiments. It is still very much in it’s infancy, and I hope to evolve this project with the support of the community.

Key Differentiator

The speed at which a researcher innovates is limited by its iteration speed.

The life of a r...
Announcing Open Philanthropy's AI governance and policy RFP
by Julian Hazell — 2024-07-17T02:02:39.933Z
Link: https://www.openphilanthropy.org/request-for-proposals-ai-governance?utm_source=LW%2FAIF&utm_medium=EA&utm_campaign=AIGPRFP724

AI has

enormous

beneficial

potential

if it is governed well. However, in line with a

growing

contingent

of

AI

(and

other

)

experts

from academia, industry, government, and civil society, we also think that AI systems could

soon

(e.g. in the next 15 years) cause

catastrophic harm

. For example, this could happen if malicious human actors

deliberately misuse

advanced AI systems, or if we

lose control

of future powerful systems designed to take autonomous actions.[1]

To improve the odds that humanity successfully navigates these risks, we are soliciting short expressions of interest (EOIs) for funding for work across six subject areas, described here.

Strong applications might be funded by Good Ventures (Open Philanthropy’s

partner organization

), or by any of >20 (and growing) other philanthropists who have told us they are concerned about these risks and are interested to hear about grant opportunities we recommend.[2] (You can indicate in your application whether we have permission to share your materials with other potential funders.)

Click here to read the full RFP.

As this is a new initiative, we are uncertain about the volume of interest we will receive. Our goal is to keep this form open indefinitely; however, we may need to temporarily pause accepting EOIs if we lack the staff capacity to properly evaluate them. We will post any updates or changes to the application process on this page.

Anyone is eligible to apply, including those working in academia, nonprofits, industry, or independently.[3] We will evaluate EOIs on a rolling basis. See below for more details.

If you have any questions, please

email us

. If you have any feedback about this page or program, please let us know (anonymously, if you want) via this

short feedback form

.
Turning Your Back On Traffic
by jefftk — 2024-07-17T01:00:08.627Z

We do a lot of

walking
around the neighborhood with kids, which usually involves some
people getting to intersections a while before others. I'm not
worried about even the youngest going into the street on their
own—Nora's been

street trained for
about a year—but we have to be careful about what signals we
send to cars. Someone standing at an intersection facing traffic
looks to a driver like they're waiting for the opportunity to cross.

Waving drivers to continue doesn't work well: they tend to slow down
significantly, and many of them will wave back in a misguided attempt
at "no, you first" politeness. Instead, what seems to work well is
turning your back to the street:

This isn't perfect: some drivers still read anyone stationary near an
intersection as intending to cross, but it's pretty good. And it's
especially good for little kids: not only do they often like to look
intently at passing traffic in a way that is concerning to drivers and
passers by, but it's a clear signal to the parent that the kid knows
it's not time to cross yet.

Comment via: facebook, mastodon
Opinions on Eureka Labs
by jmh — 2024-07-17T00:16:02.959Z

I would be currious to hear from people about their views or reactions to the Eureka Labs planned AI training course.

While a very trivial level, I must confess to liking the name. When I saw the mention on MR the first thing I thought of was the TV show Eureka which I found quite entertaining and enjoyable.

More seriously I can currious about what quality the cource might be, I assume some here know something about Kartaphy and how his company might present the information. I would also be interested in thought from a pure AI Safety perspective. One might suspect some see this a bit as teaching any or everyone how to build a nuclear bomb or make TNT. But perhaps some see him as perhaps helping to get students thinking in terms of safety and alignment.

Update, I should probably also note that this probably doesn't really move the dial on anything much given the existing availability of online AI training from many. many sources. So "Big yawn." to the question might be the popular response.