trentmkelly/LessWrong-43k huggingface dataset:
Failing to fix a dangerous intersection
by alyssavance — 2022-06-30T18:09:18.565Z

Over the last few years, many people have written about why America can't build things anymore (eg. here, although this is just one of hundreds of relevant essays). Ten years ago, when I was 21 and had just graduated, a friend told me about a dangerous intersection in Berkeley. I tried writing to the city and asking for it to be fixed; I'm posting the email and reply here as a useful data point.

My request:

To the Berkeley Department of Transportation:

I write to urge the City to address a dangerous intersection, at San Pablo and Gilman Streets. When approaching the intersection from the west (on Gilman), there are two lanes of roadway, of which one is left-and-straight and the other is right-and-straight. However, on the opposite side of the intersection (east of San Pablo), there is only one lane of roadway. Hence, cars going straight are forced to merge in the middle of the intersection (with no warning), which is time-consuming and hazardous.

I and some fellow Berkeley residents propose that the lane arrows be modified, such that either the left lane is left only, or the right lane is right only. This way, there is only one lane of forward traffic, and cars do not have to merge in the intersection. This modification would cost virtually nothing, and would make driving easier for the thousands of City residents who use this intersection daily, as well as preventing a potentially fatal accident. We greatly appreciate your consideration.

Their reply is below. (Email is generally private, but in this case the communication, from a city employee about a government issue, should be an open record under the California Public Records Act.)

Dear Alyssa,

I forwarded your email to the Supervising Traffic Engineer, and have been asked to respond to your request with the following information.

San Pablo Avenue (State Highway 123) is under Caltrans jurisdiction, and any significant changes to the intersection must be approved by Caltrans. Recent communications from Caltrans indicate they have no immediate plans to upgrade their non-freeway facilities.

In theory the City could develop a modified striping plan concept and ask for approval from Caltrans to proceed. If Caltrans were to agree to the concept, the City would need to assign resources to develop the conceptual design, then a detailed design, obtain approvals from Caltrans, then put out and pay for a contrac

...
Most Functions Have Undesirable Global Extrema
by En Kepeig — 2022-06-30T17:10:02.672Z

(Crossposted from my blog Centerless Set)

It’s hard to come up with a workably short function that includes human civilization in its global maximum/minimum.

This is a problem because we specify our goals to AI using functions. For any practical function we might use, there’s a set of strange and undesirable worlds that satisfy that function better than our world.

For example, if our function measures the probability that some particular glass is filled with water, the space near the maximum is full of worlds like “take over the galaxy and find the location least likely to be affected by astronomical phenomena, then build a megastructure around the glass designed to keep it full of water”.

On top of this, what the AI would actually do if we trained it on that function is hard to predict, because it would learn values that instrumentally help it in our training data but not necessarily in new environments. This would result in strange, unintuitive values whose maximum is equally if not more unlikely to include human civilization.

Here I’ll list some potential ideas that might come to mind, along with why I think they’re insufficient:

Idea:

Don’t specify our goals to AI using functions.

Flaw:

Current deep learning methods use functions to measure error, and AI learns by minimizing that error in an environment of training data. This has replaced the old paradigm of symbolic AI, which didn’t work very well. If progress continues in this direction, the first powerful AI will operate on the principles of deep learning.

Even if we build AI that doesn’t maximize a function, it won’t be competitive with AI that does, assuming present trends hold. Building weaker, safer AI doesn’t stop others from building stronger, less safe AI.

Idea:

Use long, complicated functions that represent our actual goals.

Flaw:

This is even more difficult than it sounds. It’s hard to specify a goal like “don’t affect anything except this glass and this pitcher of water” using a function. Every action that fills the glass with water also affects everything within the local causal sphere of influence, although in very minor ways. “Only affect everything else a little” has very strange worlds near its global maximum. “Only use T seconds to form a plan to fill the glass with water” could result in a plan like “copy paste my code without the planning time limit”.

Also, the longer our function gets, the harder ...
Hedonistic Isotopes:
by Trozxzr — 2022-06-30T16:49:49.573Z

Abstract

Simple scaling of hedonistic values can become fairly imprecise when given scales, especially when comparing elevations of euphoria or emptiness. Hedonistic Values may be more precise and specific than other methods of description. If there is more variation in describing experiences, it can fill gaps in tacit knowledge that fails to be articulated into its comprehensible constituent, if any at all. Firstly, the person of experience should list every emotion they experience in x situation. Such that we converge the plethora of words into an underlying topic. This can be done by scaling different temperaments in relation to each other on a scale, which then chooses the most ideal “feeling” closest to all the emotions the user had mentioned.

For example if we have the word distraught and annoyed we may converge them to the word “temperamental”. This may be done heuristically, but to avoid major biases, it’s best to plot the vast variation of hedonistic traits on a graph (Which is time consuming, at the expense of precision).

Lastly, each trait in itself that was in result of the users experiences would be scaled out of 10 or 20, but this value can be endlessly adjusted towards the overall hedonistic framework of the individual which is far more accurate. Note you may utilize a value for each semantic that comprises an experience, but to get obtain the isotope/value of the output trait, you take the semantic list the user mentioned and obtain its mean (Sum of each scale value / Total amount of words(description) ).
Abadarian Trades
by David Udell — 2022-06-30T16:41:12.232Z

Spoilers for planecrash (Book 2).

I propose an equal split of the gains from this trade, and will reject lesser splits with a probability corresponding to how disproportionately they reserve the gains for you, such that you can't actually do better by pretending to underrate me, but we'll still work something out with high probability if we honestly disagree, paid in Wishes and spellsilver above and beyond the ordinary payment of permanent Arcane Sight. …and permanent Tongues."

…first of all, mortals aren't supposed to know about any of that, however garbled and incomplete it sounds, and second, if they stumble over a piece of it, you're supposed to shut them down hard and refuse to bargain for their soul and ideally let them get executed by Cheliax.

…

"That is not the way of Hell," he rumbles. "Asmodeus is not Abadar, little mortal, no matter what company you have been keeping of late. You can try to hold what secrets you like, and Hell will keep its own, and whoever is closer to Asmodeus in wit and ways is the one to win the compact. Name to me the price you seek for yourself."

--planecrash

If I offer you tutoring for $40, you have the option of either turning me down and keeping $40 or gaining tutoring and losing $40. Whichever option is more desirable to you is the one you'll take. Similarly, I have the option of offering tutoring for money or not offering tutoring with my time. I'll do whichever sounds net best to me. Because people only ever willingly switch alternatives when the new alternative is a net improvement, someone taking me up on my tutoring offer means that we both prefer what we bought ($40, tutoring) to what we sold (tutoring, $40). This is one of the best reasons to be enthusiastic about free markets: transactions are mutually beneficial to both traders.

Because they constitute improvements by both trader's lights, both traders want to make every transaction they can. Once no more trades can clear, no more mutual improvements can be made. However much happier each trader is now, by their own lights, is how beneficial free markets were to them.

If tutoring is worth $100 to you in total, then I can offer tutoring for up to $99 and you'd still buy from me. If I'd rather have unpaid leisure time than work for $24, I'll only ever offer tutoring for $25 or more. So there are many prices I can offer that you'd buy from me at, that wou...
Covid 6/30/22: Vaccine Update Update
by Zvi — 2022-06-30T14:00:01.476Z

This week’s news is that the FDA advisory committee voted overwhelmingly to update the vaccine for Omicron, after a delay of only six months, which means they’ll get to deciding which way it should be updated Real Soon Now and then they can tell the pharma companies what requirements they want to place on that. There’s some chance the update will happen at a time that isn’t never, and even a tiny chance the update might not be hopelessly behind the virus when it happens. You never know.

Executive Summary

BA.5 now the dominant strain.
We might eventually update the vaccines. Maybe.
Authorities are still using misinformation to justify policy.

Let’s run the numbers.

The Numbers

Note that I am increasingly forced to make adjustments to account for obviously delayed or wrong reporting. Florida reports every two weeks, and a number of states did something requiring at least some form of adjustment. These decisions are

Predictions

Prediction from last week: 675k cases (+14%) and 2,015 deaths (+14%).

Results: 679k cases (+14%) and 2,048 deaths (+16%).

Prediction for next week (4th of July!): 600k cases (-13%) and 1,800 deaths (-16%).

This is a holiday prediction. I expect the true situation to hold roughly steady, with drops due to the extra reporting this week going away plus the missing reporting next week.

Deaths

Cases

As expected this is a bounce back from the holiday weekend. Now we are headed into another holiday weekend, so I expect a drop back down again. Things are less smooth at the state level than they appear at the regional level, but nothing is that dramatic.

We are entering the period of most rapid percentage growth in BA.5 on a week-to-week basis, so if we are going to see that transition cause a lot of problems then these next two weeks are when we will see clear signs of that.

BA.1.2.3.4.5

BA.4 and BA.5 are now the dominant strains.

Technically BA.2.12.1 is bigger than either of them in the latest data set, but by the time the data gets reported BA.5 will have passed it. None of this was ever in doubt.

The good news is that so far there is no sign of the next variant.

Eric Topol covers our new dominant strain, BA.5.

If Omicron really is off in a complete different direction than any of the previous dominant strains, then that is importantly good news. It means that O...
How should I talk about optimal but not subgame-optimal play?
by JamesFaville — 2022-06-30T13:58:59.576Z

I’d like an easy way to distinguish the behavior of payoff-maximizing players who would or would not play a strategy in an extensive form game that deviates from the

subgame-perfect

(Bayesian) equilibrium strategy profile of the game (when their strategy is known to their opponent, and their opponent is also payoff-maximizing).

Example

An example of what I’m interested in can be seen in an ultimatum game where a proposer presents a responder with an offer of the form (a,1−a), where a and 1−a for a∈[0,1] are the respective payoffs of the proposer and the responder.

For now, let’s call a strategy subgame-optimal if for every subgame of the game, the strategy’s restriction to that subgame is still optimal within the subgame. In other words, at each decision node of the game, a payoff-maximizing player with a subgame-optimal strategy chooses the action which maximizes their expected payoff (as calculated at that decision node, rather than as calculated according to their prior). A payoff-maximizing player who can commit to a subgame-suboptimal strategy will play the action which maximizes the payoff they expected at their initial decision node, without the need to play a strategy that holds up to

backward induction

.

Say that the responder’s strategy is known to the proposer ahead of time, and the proposer is restricted to only subgame-optimal strategies. What strategy should the responder use? A (subgame-suboptimal) strategy to reject all proposals with a > ε for arbitrary ε would force the proposer to make an offer arbitrarily in the responder’s favor. But this is impossible for a responder who can only play subgame-optimal strategies. Updateless agents don’t have to worry about this problem, but updateful agents without commitment devices or values other than payoff maximization do.

Possible terms for this distinction

Ex interim or ex post rationality vs. ex ante rationality
I like the clarity of focusing on from what perspective the expected payoff of an action is being calculated. A big advantage to these terms is that they are in use to some extent. For example,

Posner (1997)

classifies retaliation in a non-repeated interaction as ex ante rational but ex post irrational. On the other hand, these terms might sound overly jargon-y or be confused with different uses of “

...
Formal Philosophy and Alignment Possible Projects
by Daniel Herrmann — 2022-06-30T10:42:23.245Z

Context

We (Ramana, Abram, Josiah, Daniel) are working together as part of PIBBSS this summer. The goal of the PIBBSS fellowship program is to bring researchers in alignment (in our case, Ramana and Abram) together with researchers from other relevant fields (in our case, Josiah and Daniel, who are both PhD students in Logic and Philosophy of Science) to work on alignment.

We’ve spent a few weeks leading up to the summer developing a number of possible project ideas. We’re writing this post in order to both help ourselves think through the various projects and how they might actually help with alignment (theory of change), and to (hopefully!) get feedback from other alignment researchers about which projects seem most promising/exciting.

We’ve discussed five possible project directions. The first two in particular are a bit more fleshed out. For each project we’ll describe the core goals of the project, what a possible plan(s) of attack might be, and how we’d expect a successful version of the project to contribute to alignment. Many of our projects inherit the theory of change for all agent foundations work (described here by John). In the descriptions below we focus on slightly more specific ways the projects might matter.

Possible Projects

Project 1: Inferring Algebras from Behavior

Description

Standard representation theorems in decision theory (for example, Savage and Jeffrey-Bolker) show that when an agent’s preferences satisfy certain rationality and structural constraints, then we can represent the preferences as if they were generated by an agent who is maximizing expected utility. In particular, they allow us to infer meaningful things about both the probability and the utility function. However, these representation theorems start off with the agent’s conceptual space (formally, an algebra[1]) already “known” to the person trying to infer the structure. The goal of this project would be to generalize representations theorems so that we can also infer things about the structure of an agent’s algebra from her preferences or choice behavior.

Theory of Change

A representation theorem is a particular kind of selection theorem. John has argued that selection theorems can help us understand agency in a way that will help with alignment. Inferring an agent’s conceptual space from her behavior also seems like it might be useful for ELK (for ELK, we might want to th...
Cultivating And Destroying Agency
by hath — 2022-06-30T03:59:27.239Z

Something that really surprised me about high school is how little agency the majority of the students there had. This was even more pronounced among the students at the local magnet school, Academies of Loudoun; they were selected for on intelligence and work ethic, but generally had a "the world affects me" mindset as opposed to "I affect the world". It took me a while to understand what led to this, and over time my way of describing this went from "NPCs/sheeple (somewhat joking)" to "NPCs/sheeple (almost completely serious)" to my current way of describing it, which is "kids who had the agency burned out of them".

In this post, I'll go through the fully general way of increasing agency, some actual examples of that, and what it was that burned the agency out of my peers.

Building Agency

The academic term for agency, or the "I affect the world" vs. "The world affects me" continuum is internal vs. external locus of control, and I'll be using that interchangeably with agency.

Charles Duhigg's Smarter Faster Better has a chapter dedicated to locus of control. He talks about how the Marines restructured their basic training around building an internal locus of control:

Krulak [The head of the Marines] believed basic training needed to change. "We were seeing much weaker applicants," he told me. "A lot of these kids didn't just need discipline, they needed a mental makeover. They'd never belonged to a sports team, they'd never had a real job, they'd never done anything. They didn't even have the vocabulary for ambition. They'd followed instructions their whole life."

(...)

This was a problem, because the Corps increasingly needed troops who could make independent decisions... In today's world, that means the Corps requires men and women capable of fighting in places such as Somalia and Baghdad, where rules and tactics change unpredictably and marines often have to decide—on their own and in real time—the best course of action.

This is one of two times I've seen an organization explicitly try to maximize agency in its members; the second is the Atlas Fellowship.

Duhigg went on to explain what researchers had found to be the core of creating an internal locus of control; choice. Presenting people with a choice, as opposed to just picking something for them, gives them a feeling of power over the outcome, instead of someone else picking their path for them.

If

...
$500 bounty for alignment contest ideas
by Orpheus16 — 2022-06-30T01:56:49.863Z

*Up to $500 for alignment contest ideas*
Olivia Jimenez and I are composing questions for an AI alignment talent search contest. We want to use (or come up with) a frame of the alignment problem that is accessible to smart high schoolers/college students and people without ML backgrounds.

$20 for links to existing framings of the alignment problem (or subproblems) that we find helpful.

$500 for coming up with a new framing that meets our criteria or that we use (see below for details; also feel free to send us a FB message if you want to work on this and have questions).

We’ll also consider up to $500 for anything else we find helpful.

Feel free to submit via comments or share Google Docs with

oliviajimenez01@gmail.com

and

akashwasil133@gmail.com

. Awards are at our discretion.

-- More context --

We like Eliezer’s strawberry problem: How can you get an AI to place two identical (down to the cellular but not molecular level) strawberries on a plate, and then do nothing else?

Nate Soares noted that the strawberry problem has the quality of capturing two core alignment challenges: (1) Directing a capable AGI towards an objective of your choosing and (2) Ensuring that the AGI is low-impact, conservative, shutdownable, and otherwise corrigible.

We also imagine if we ask someone this question and they *notice* these challenges are what makes the problem difficult, and maybe come at the problem from an interesting angle as a result, that’s a really good signal about their thinking.

However, we worry if we ask exactly this question in a contest, people will get lost thinking about AI capabilities, molecular biology, etc. We also don’t like that there aren’t many impressive answers besides full answers to the alignment problem. So, we want to come up with a similar question/frame that is more contest-friendly.

Ideal criteria for the question/frame (though we can imagine great questions not meeting all of these):

It can be explained in a few sentences or pictures.
It implicitly gets at one or more core challenges of the alignment problem.
It is comprehensible to smart high schoolers/college students and not easily misunderstood. (Ideally the question can be visualized.)
People don’t need an ML background to understand or answer the question.
There are good answers besides solving the entire alignment problem.
Answers might reveal people’s abilities to notice the hard parts of the align

...
any good rationalist guides to nutrition / healthy eating?
by Ben A — 2022-06-30T00:50:53.543Z

It's clear to me that there is a great deal of nutritional advice grounded in weak or worthless evidence. A few reasons that the evidence is weak: Long-durations RCTs are very hard to run (they would ideally run for decades, it's generally hard to blind people to what they are eating, it's hard to make people stick for long periods to diets they haven't chosen themselves). Observational studies are badly confounded and you can't control perfectly for all confounders. The usual "Why Most Published Research Findings Are False" problems are exacerbated by the existence of food producers with big budgets and huge incentives to influence the findings.

Is there any trustworthy guide to well-grounded nutritional science that leads to actionable advice? (beyond, don't eat things that are poisonous, try to avoid deficiencies in a range of vitamins and minerals that are frankly difficult to avoid getting enough of if you live in a rich country and eat a vaguely normal diet) Or do I have to sift through it by myself? I've found it hard to find any trustworthy consensus on questions as basic as, is there any health consequence to eating lots of saturated fat, or, does drinking a modest amount of alcohol provide strong health benefits compared to not drinking alcohol?

I thought this piece made a good argument about nutritional pseudoscience: https://metarationality.com/nutrition.