trentmkelly/LessWrong-43k huggingface dataset:
Cross-Void Optimization
by pneumynym — 2022-11-04T17:47:44.499Z

I. Tea-master

You've probably heard this tale before, but please bear with me.

A tea-master was strolling through Kyoto when he bumped into a ronin on the busy street. He apologized profusely, of course; but the swordsman, like many who lack honor, had filled himself with vanity instead. As he raised his katana to slaughter the hapless man, a sudden whim took his fancy: drawing his unblooded wakizashi, he slapped its hilt into the master's quaking palm. "Meet me by the gate at noon tomorrow," grinned the ronin, "and we'll have some sport. Otherwise I'll hunt you down."

By a stroke of what we (who cannot see the Pattern) must call fortune, the tea-master's house was being honored by a visit from a legendary samurai, an old friend from Lord Yamanouchi's court. In desperation, the master begged his guest to teach him the ways of the blade. "Very well," the warrior said grimly. "Fetch my bokken at once."

But after a few hours, it was miserably clear that the tea-master had no aptitude. "It is hopeless," he mumbled. "Tomorrow at noon, I shall be slain."

The old samurai nodded. "Yes. Come, friend, let us have tea together one last time."

Bent with misery, the master began to perform the ancient rituals of his Art. And the samurai noticed something strange: as the tea-master handled the fragrant leaves and steaming pots, every trace of terror drained away. Past and future lifted from his shoulders--all techniques became one technique--he moved correctly with no constraint.

"Stop!" barked the samurai. "Do you want to kill your enemy tomorrow?"

The master stared. "You said there is no hope."

"Not of survival. But do you want to die like a man? Do you want to take your enemy with you?"

". . . Yes. Yes, I do."

"Then do what you are doing now," said the warrior.

"But I am not doing anything!" the tea-master objected.

"Exactly. Do not think, do not feel. Simply raise your sword, cut, and die."

The master, being a master, understood.

When the ronin arrived the next day at the place ordained for the combat, he found the tea-master already waiting. The honorless swordsman chuckled and stepped closer, unscabbarding his weapon--but then he paused. Where was the cringing buffoon from only a day before? The tea-master was naked to the waist, his wakizashi gleaming in the sunlight, motionless. The ronin d...
Monthly Shorts 10/22
by Celer — 2022-11-04T16:30:07.616Z

India?

Map of Indian poverty, raw data on page 136.

For how important poverty/wealth in those states are, you can look at

That data was drawn from Wikipedia, while the population map distortion was done with this program, which I suspect I will use myself one of these days. Comparing the two isn’t trivial, but you can certainly see that, for example, it’s pretty important that Bihar is poor and Kerala is not. OK, so I checked the List of Indian State and Union Territories by GDP per Capita just in case my assumptions were wrong, and wow they were. Kerala has a multidimensional poverty index a fifth that of Goa’s while Goa has twice the per capita GDP of Kerala. Goa on a PPP adjusted basis is still substantially poorer than the poorest American state, but it’s richer than Greece while Kerala is closer to pre-invasion Ukraine. Something interesting is going on. The answer seems to be “functional communism”? Unfortunately, limited time and many things to do.

Wikipedia

In 2022, Canada acquired a border with a second country. This came out of reading the surprisingly interesting Talk Page for the Wikipedia article on 2022. I was pinged because I’d made a small edit, but it’s fun to read Jim Michaels 2 argue for deleting almost everything, while other people try to defend smalltime filmmakers for inclusion. Normally I’m very strongly anti-deletionist, but when it comes to content on a year page I’m more sympathetic.

The edit that got me dragged in was changing a disambiguation link to correctly point towards the January 2022 coup d'état in Burkina Faso, as opposed to the 2022 coup: in January “the 2022 coup” was sufficient but by September there had been a second. You know you’re having problems when Russia calls for a return to a framework with “legitimacy”.

I do recommend reading Wikipedia pages for a year to get a sense of what is happening. It’s not the only thing, but if you want to be “well-rounded” in some sense, it’s a better starting point than most alternatives. If you don’t understand something, click the link, and read the first paragraph.

On the topic of Wikipedia, the two largest cities in New England (excluding the parts of New York that extend over Connecticut) are Caribou and Ellsworth Maine! This is because the largest cities by land area in America are *not* in New England.

Militaries

Something that can happen when you get too into political theory is th...
Weekly Roundup #4
by Zvi — 2022-11-04T15:00:01.096Z

Relatively light roundup this week, likely due to Elon Musk buying Twitter plus the incoming midterms sucking all the oxygen from everything else. I do have a Twitter/Musk post in progress tentatively scheduled for Monday.

Prediction Markets Would Probably Be Great

Journalist Ben Collins says ‘very good example of how illogical betting markets are’ and gives classic example of prediction markets being completely logical and doing good probabilistic reasoning.

Metaculus timelines for AI winning a gold medal in the International Math Olympiad keep compressing, down this year from 2036 to 2026. This is certainly super scary and exciting on the AI front. It is also disappointing as a former aspirational math competitor, as it reveals that the IMO problems are far more formulaic and hackable than we realized. When AI makes super rapid progress in some areas (e.g. math competitions) while stalling in others (e.g. self-driving cars) relative to expectations, a lot of that is learning about the underlying task space.

People Are Trying To Destroy the Internet

Joe Manchin taking his turn? Proposing to force filing of ‘suspicious activity’ reports for essentially everything. From a few weeks ago, forgot to include before.

Sports Go Sports

It is real, and it is spectacular. Worth watching.

Pat McAfee @PatMcAfeeShow

Is this real?

9:11 PM ∙ Oct 30, 2022

112,354Likes8,984Retweets

He learned this from playing NASCAR on Game Cube growing up. It works in real life.

Competitors mostly thought it was super cool, which is correct.

The question is, it was super cool that time. What happens when it happens on a regular basis? Looks like there is currently no appetite for a rules change until the off season. I think this is a mistake. We honor such crazy one-time moves by saying ‘yes, that was super cool and super legal, with great execution. Kudos. Now let us make sure that never happens again.’ If it happens a few more times before the season is over, that lessons the moment, and also seems dangerous and like it would rapidly get kind of dumb.

Bad News

On China stocks last week, I noticed my confusion that stocks were down after the party congress. This shows how long it has been since I have been trading. Back in the day, I would not have missed what a reader reminded me about, which is that stocks are not permitted to fall during a party congress, so...
A new place to discuss cognitive science, ethics and human alignment
by Daniel_Friedrich — 2022-11-04T14:34:15.632Z
Link: https://www.facebook.com/groups/652418766529925

One could frame EA as the project of human alignment - of deconfusing ourselves about what we care about and figuring out how to actualize it. And there seems to be an interconnected bundle of problems at the core of this project:

What should we fundamentally value?
How to distinguish rational intuitions from biases?
How to make an AGI care about these questions?
Which ideas should we spread to help humanity process these questions more complexly?
How certain can we be about all of this?

These questions are particularly important, as we seem to live at the hinge of history and at a time of a growth in consciousness research[1]. They're interconnected but require a range of disciplines perhaps too wide for a single person to fully grasp - which suggests there could be a great added value in stronger cooperation.

So if you like impossible problems and want to work on them together, you're warmly welcome at the Mind & Values Research Group. The group is about the intersections of EA, cognitive science, the nature of consciousness and intelligence, moral philosophy and the formulation & propagation of ethical and rational principles.

How can these areas help nudge humanity in a positive directions?

1. The philosophy of mind angle

Deconfusing humanity about what we mean by values and intelligence could:

Help solve the technical side of AI alignment[2].
Advance the broad-longtermist mission introduced in What We Owe the Future - getting a clearer picture on which values should we lock in in these important times. Important topics here could be the nature of valence, net-positiveness of experiences, animal and digital sentience.
Help global prioritization by investigating the assumptions behind interventions for instance recommended by Happier Lives

2. The social change angle

Cognitive enhancement research - how to support rationality & moral circle expansions in society by formulating the most elegant case for rational ethics
Studying people's biases about values could help here[3]. This area can also be advanced with some inferences from experimental philosophy or even history & sociology of ideas.

What could it lok like?

Meetups: The group will vote on topics to discuss. For the following month or so, people will be welcome to collect materials from different angles. We'll discuss them in a virtual meetup and collect what was mentioned in a document pe

...
A newcomer’s guide to the technical AI safety field
by zeshen — 2022-11-04T14:29:46.873Z

This post was written during

Refine

. Thanks to Jonathan Low, Linda Linsefors, Koen Holtman, Aaron Scher, and Nicholas Kees Dupuis for helpful discussion and feedback.

Disclaimer: This post reflects my current understanding of the field and may not be an accurate representation of it. Feel free to comment if you feel that there are misrepresentations.

Motivations

I remember being fairly confused when I first started reading AI safety related posts, especially when it pertains to specific ideas or proposals, as there may be implicit assumptions behind those posts that relies on some background understanding about the research agenda. I have since had the opportunity to clear up many of those confusions by talking to many people especially while I was participating in Refine. Looking back, there were many background assumptions about the field I wish I had known earlier, so here’s the post I never had.

This post does not intend to cover topics like why AI safety is important, how to get into the field, or an overview of research agendas, as there are

plenty of materials

covering these topics already.

Some terminology

Artificial intelligence (AI) refers to intelligences that are created artificially, where intelligence measures an agent's ability to achieve goals in a wide range of environments as per

Shane Legg and Marcus Hutter’s definition

.

Artificial general intelligence (AGI)

refers to a machine capable of behaving intelligently over many domains, unlike narrow AIs which only perform one task.

Artificial superintelligence (ASI)

, or “superintelligent AI”, refers to an intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills. On the other hand,

transformative AI (TAI)

is an AI that precipitates a transition at least comparable to the agricultural or industrial revolution, where the concept refers to its effects instead of its capabilities. In practice however, the terms AGI, ASI, and TAI are sometimes used interchangeably to loosely mean “an AI that is much more powerful than humans”, where “power” (related to “

optimization

”) is often used to loosely mean “intelligence”.

AI safety is about making the development of AI go safely. It is often used to refer to AGI safety or

AI alignment

(or just “alignment” because “A...
Are alignment researchers devoting enough time to improving their research capacity?
by Carson Jones — 2022-11-04T00:58:21.349Z

(Epistemic Status: Anecdotal)
If we want to reduce AGI x-risk, it seems pretty intuitive to me that alignment researchers should be regularly dedicating time to improving their research capacity. But I'm suspicious that many of them don't do this.
Am I wrong? I would love to know if I am.
(I'm using the phrase "research capacity" vaguely, to mean both research skill and productivity.)

Over the last 3 - 4 years, I've had something like 12 - 20 informal conversations with researchers at various alignment orgs on the subject of how they improve their research capacity. In these conversations, I'd ask questions like "How do you personally go about getting better at research?", or "What have you done recently to improve your research process?" or "What is one thing you could do to get better at your job?". And more than half of the responses are one of the following:
1. a blank stare
2. a long, thoughtful pause followed by no actual answer
3. an argument that "this kind of messy, abstract work doesn't lend itself to direct improvement in the way other skills do"
4. an argument that "the best way to improve at research is to simply do the work and keep your eyes peeled for opportunities to improve as you go."
To be clear, some researchers do have different responses. But these answers were surprisingly common, and... they seem wrong?

My Response To #3 :

"this kind of messy, abstract work doesn't lend itself to direct improvement in the way other skills do"

I'm not a researcher, but I can't think of any skill I've learned, whether intellectual or physical, that doesn't benefit from some amount of regular and intentional focus on improving it.

Additionally, over the last couple months, I've started having debugging conversations with researchers who want help thinking through how to improve, and in these conversations most of them generate lots of ideas and claim the discussion is quite productive. That's not what I would expect if doing explicit skill improvement just didn't work for alignment research.

Here are a few examples of opportunities for improvement that researchers have identified in talking with me:

improving their ability to find research collaborators
improving at prioritizing research tasks (e.g. should they spend the next hour reading a textbook, or spend this time writing out their current ideas)
improving their abili

...
Don't you think RLHF solves outer alignment?
by Charbel-Raphaël — 2022-11-04T00:36:36.527Z

I think the RLHF solves 80% of the problems of outer alignment, and I expect it to be part of the solution.

But :
- RLHF doesn't fully solve the difficult problems, which are beyond human supervision, i.e. the problems where even humans don't know what is the right way to do
- RLHF does not solve the problem of goodharting: For example there is the example of the hand which wriggles in front of the ball, without catching the ball and which fools the humans. (Imho I find this counter-example very weak, and I wonder how the human evaluators could miss this problem: it's very clear in the gif that the hand does not grab the ball).

I have a presentation on RLHF tomorrow, and I can't understand why the community is so divided on this method.
Mechanistic Interpretability as Reverse Engineering (follow-up to "cars and elephants")
by David Scott Krueger (formerly: capybaralet) — 2022-11-03T23:19:20.458Z

I think (perhaps) the distinction that I was trying to make in my previous post "Cars and Elephants": a handwavy argument/analogy against mechanistic interpretability is basically the distinction between engineering and reverse engineering.

Reverse engineering is analogous to mechanistic interpretability; engineering is analogous to "well-founded AI" (to borrow Stuart Russell's term).
So it seems worth exploring the pros and cons of these two approaches to understanding x-safety-relevant properties of advanced AI systems.
As a gross simplification,[1] we could view the situation this way:

Using deep learning approaches, we can build advanced AI systems that are not well understood. Better reverse engineering would make them better understood.
Using "well-founded AI" approaches, we can build AI systems that are well understood, but not as advanced. Better engineering would make them more advanced.

Under this view, these two approaches are working towards the same end from different starting points.
A few more thoughts:

Competitiveness arguments favor reverse engineering. Safety arguments favor engineering.
We don't have to choose one. We can work from both ends, and look for ways to combine approaches.
I'm not sure which end is easier to start from. My intuition says that there is the same underlying difficulty that needs to be addressed regardless of where you start from,[2] but the perspective I'm presenting seems to suggest otherwise.
There may be some sort of P vs. NP kind of argument in favor of reverse engineering, but it seems likely to rely on some unverifiable assumptions (e.g. that we will in fact reliably recognize good mechanistic interpretations).

^

I know people will say that we don't actually understand how "Well founded AI" approaches work any better. I don't feel equipped to evaluate that claim beyond extremely simple cases, and don't expect most readers are either.

^

At least if your goal is to get something like an AGI system, the safety of which we have justified confidence in. This is perhaps too ambitious of a goal.
Could a Supreme Court suit work to solve NEPA problems?
by ChristianKl — 2022-11-03T21:10:48.344Z

According to Eli Dourado, the original interpretation of the NEPA law was reasonable. The earliest Environmental Impact Statements (EIS) were often less than ten pages. Afterward, through a combination of district court decisions and the decisions of the Council on Environmental Quality the work required to create an EIS ballooned to make it unreasonably expensive to comply with the EIS requirements.

It's noteworthy that neither laws passed by congress nor court decisions by the Supreme court are responsible in the growth of the work.

Whenever cases reached the Supreme Court, the court usually ruled against expansive interpretations of NEPA. Zachary Green describes the court decisions in his master thesis "NEPA in the Supreme Court: A history of defeat" by saying:

Since its enactment in 1970, the National Environmental Policy Act (NEPA) has been the focus of seventeen Supreme Court cases. Government and industry have defeated environmental organizations in each of those seventeen cases.

In United States v. Students Challenging Regulatory Agency Procedures the Supreme Court argued:

But 694*694 nowhere, either in the legislative history or the statutory language, is there any indication that Congress intended to restore to the federal courts the power temporarily to suspend railroad rates, a power that had been clearly taken away by § 15 (7) of the Interstate Commerce Act.

The statutory language, in fact, indicates that NEPA was not intended to repeal by implication any other statute.

This is a clear statement against an overreaching reading of NEPA. Given this history of Supreme Court cases, it seems to me that there's a lot of room for the Supreme Court to rule against the current interpretation of NEPA without having to rule against any precedent. Given the current trend of ruling very pro-business, the current court would likely be even more willing than past courts to rule against expansive interpretations NEPA.

This year the Supreme Court decided that the EPA overreached when they decided to regulate CO2. In the process they created precedent that reduces the amount of freedom agencies have to just do what they want.

One argument might be to say that the Council on Environmental Quality overreaches the authority that congress has given it?
[Video] How having Fast Fourier Transforms sooner could have helped with Nuclear Disarmament - Veritasium
by mako yass — 2022-11-03T21:04:35.839Z
Link: https://www.youtube.com/watch?v=nmgFG7PUHfo

Veritasium presents an argument that if we'd had Fast Fourier Transforms sooner, this would have made it possible to detect underground nuclear tests without on-site audits, somewhat better nuclear non-proliferation treaties could have been signed, and nuclear proliferation would have been slowed.

This was really interesting, I hadn't heard the story of these very early, hopeful, tragically failed attempts at nuclear disarmament. I'm troubled to learn that the Soviets rejected the Baruch Plan, measures analogous to my geopolitical AGI non-proliferation story. They saw on-site audits as "espionage". They saw this proposal to form an international nuclear authority as a threat to their sovereignty. To some extent, it could have been. There's a lot to chew on there.

A particularly interesting detail is that Fast Fourier Transforms actually were discovered earlier, but then forgotten, so no one can argue for fatalism here, it was not inevitable that we'd end up with this progress differential. Other histories were possible.
Further considerations on the Evidentialist's Wager
by Martín Soto — 2022-11-03T20:06:31.997Z

Thanks to Caspar Oesterheld for the discussion on which this post is largely based.

In a previous post I presented an objection to The Evidentialist's Wager. Reading it is probably necessary to understand the following.

A counter-argument to my objection

In the post I broadly claim the following:

Imagine we have absolutely no idea whether more acausally correlated agents in the universe are positively correlated to us/Good Twins (our one-boxing is evidence for them increasing our utility function) or negatively correlated to us/Evil Twins (our one-boxing is evidence for them decreasing our utility function). That is, our credence on the two events is 50%[1]. Then, when facing a Newcomb decision, the stakes for EDT and CDT are equal. This is because any evidence of additional utility gains provided by EDT will balance out in the expected value calculation, given our complete uncertainty about the interpretation of that evidence (whether one-boxing is evidence for a universal increase or decrease in our utility).

Shortly after writing the post, I discovered the following counter-argument: In that situation, EDT urges you to research further whether more positively or negatively correlated agents exist, to break the symmetry and then act accordingly. That is, it is plausible (it has non-zero probability) that dedicating more resources to studying this issue ends up breaking the symmetry, and changing your credences away from 50-50 in some of the two possible directions. If that happens, then the Wager applies and the stakes for EDT will be higher (which will either urge you to one-box or to two-box, depending on how the symmetry was broken). And so, even in the described situation, the stakes of EDT are higher, and if it's not immediately obvious it's just because EDT doesn't recommend neither of one-boxing or two-boxing, but the third option of researching further into the nature of the multiverse.

But the above argument presents a flaw, related to other issues with the concept of negative correlation, which is generally problematic and ill-defined (as I pointed out in the first post, to the extent that the definition of negatively correlated agents is not clear, but that's not an issue for what follows).

Considering meta-reasoning

Suppose the correlation between us and the Good Twins is perfectly positive (identical copies), and with the Evil Twins is perfectly negat...