trentmkelly/LessWrong-43k huggingface dataset:
Argument by Intellectual Ordeal
by lc — 2022-08-12T13:03:21.809Z

There once was an extremely annoying child named Jacob. One time during recess, Jacob decided to take up the entire playground bench by lying down on it horizontally. A classmate came up to him and said something like: "You're taking up the whole bench! That's not fair!" Jacob responded: "Ugh, you people don't know what I'm doing! Do you even know what quarks are?!" The classmate said "umm..." thoughtfully for a few seconds, and then scuttled off.

While an effective rhetorical technique, knowledge of the existence of quarks was not a very relevant indicator for who was right. The question didn't test for knowledge relevant to playground bench allocation problems, didn't correlate well with even general thinkoomph, and neither of these would've correlated well with correctness in the first place because of Jacob's clear conflicts of interest.

Medieval Europe had a method of determining guilt called "trial by ordeal". If two parties had a large dispute - say one accusing another of a serious crime - they would sometimes decide who was right by fighting to the death in an arena, or having the accused willingly ingest poison and see if they threw it up instead of getting sick or dying. The presumption was that God would favor the one telling the truth, or maybe that their skill would be bolstered by their sense of righteousness.

I like to pejoratively call the type of fallacious impromptu challenge-giving I just described "argument by intellectual ordeal". The lifecycle of the rhetorical tactic goes like this:

Person A makes an argument: "You taking up the entire bench when multiple people could sit on it seems unfair."
Person B counters with a mostly irrelevant challenge testing Person A's knowledge, general intelligence, or other: "If you're so smart solve 18283921 x 192399213 in the next ten seconds!"
Person A, caught up in the emotional fervor of arguing with B or their own competitive instinct, attempts to solve the problem and fails.
Having lost the impromptu contest, which they legitimized by being observed attempting to win, Person A meekly concedes the point.

Like "arguments from authority", the "argument by intellectual ordeal" comes with standard justifications that are almost always overused. In the case of the "argument from authority", the surface justification is something like "People with the X rubber stamp tend to be very good at answering these questi...
Anti-squatted AI x-risk domains index
by plex — 2022-08-12T12:01:24.927Z

I heard about a recent case of a relevant domain being squatted, and figured that the people who buy up domains then sell them at a markup might start to notice the growth of interest in AI x-risk and claim some domains which would otherwise host good projects. I've bought the following, and will make them available for use by anyone[1] in the x-risk community.

aisafety.global - Conference?
aisafety.careers - 80k said they were happy for others to join the space
alignment.careers
alignment.academy - Training program?
alignment.courses - List of all training programs (using Stampy answer as backend)?
aisafety.ventures - Entrepreneurs org?
aisafety.fund - AIS-specific funding org?
aisafety.foundation
aisafety.institute
xrisk.fund
xrisk.foundation
agisafety.info - Useful if we fall back to AGI Safety
agisafety.xyz
aisafety.coach - More orgs in the direction of AI Safety Support?
aisafety.dev
aisafety.network - Peer-to-peer researchers something?
aisafety.quest - Project Eurler for AIS?
agisafety.quest
aisafety.world
alignment.fyi
alignment.plus
agisafety.wiki - alt. Stampy's Wiki branding?
alignment.wiki
aisafety.observer
aisafety.group

Others:

x-risks.com - Drew Spartz
effectivealtruism.money - Dony Christie, he might have a use for
effectivealtruism.capital - Dony Christie, he might have a use for
effectivealtruism.plus - Dony Christie
effectivealtruism.ventures - Dony Christie

This cost ~$200 of personal funds for the first year, and closer to $500/y going forward as the onboarding discounts wear off. I estimate saving one or two wanted domains from being squatted and the hassle / cost of acquiring it or moving to a worse domain would make this worthwhile for several years, so this seems probably positive EV.

The main downside risk would be from people who want these domains not knowing they're available, so I have forwarded all of these domains to my home page. Hopefully people who want them will check there to track me down and request them. This is not perfect, but seems better than waiting for them to be sniped.

If you would like to list your own available relevant domains in the comments that would be welcome, I'll add them to this post and the EA Forum cross-post.

^

I will point them towards the servers of anyone who claims to have a half decent plan for the domain, and hand over the domain fully if they build something which seems useful.
What are some good arguments against building new nuclear power plants?
by RomanS — 2022-08-12T07:32:26.298Z

It seems there is a consensus in the EA community that building new nuclear power plants is a net positive for the world. For example, this video by Kurzgesagt summarizes some arguments in support of nuclear power as a tool to mitigate the global warming.

I wonder if the consensus is premature.

So far, I've encountered the following counterarguments worth pondering about:

Solar and wind power plants can be fully controlled and owned by individuals and small communities, with zero intervention from the gov. On the other hand, there is a strong societal consensus that nuclear plants must be strictly regulated or even owned by gov. Thus, a nuclear-powered country is a statist country, which is bad.

See also: Hydraulic empire

Judging by the number of nuclear-power-related fatalities, nuclear plants are rather safe. So far. But in the systems that can go horribly wrong unless properly maintained, the past performance is not a good indicator of the future performance. E.g. if Russians ruin the occupied Zaporizhzhia Nuclear Power Plant, the catastrophe will dwarf all the past nuclear disasters, combined. Thus, nuclear power plants are safe-on-average but horrible-in-the-worst-case. One human mistake can make a London-sized area uninhabitable for decades. And humans tend to make mistakes. In comparison, solar and wind are safe on average and in the worst case.

Nuclear plants are safe in the times of peace. But it's reasonable to expect that we'll have more war and terrorism in the future, due to technogenic unemployment and the effects of global warming.

In the West, nuclear power is not the fastest way to get a lot of new renewable energy. A nuclear power plant is a large and complex piece of engineering, with the strictest regulations and the strongest NIMBY tendencies. If you disregard the opposition of the local residents, you can build nuclear plants relatively fast (e.g. China). But in democratic societies, it may be a better idea to build new solar / wind / energy storage factories instead.

The solar/wind option is also more acceleratable.

Solar / wind tech is already mass produced, and can be easily scaled up. Nuclear power plants are too complex and too regulated to be mass produced in the near future.
Individuals and small communities can apply direct action to speed up solar and wind, and do it efficiently. In the simplest case, you can ins

...
Seeking PCK (Pedagogical Content Knowledge)
by CFAR!Duncan — 2022-08-12T04:15:28.181Z

Author's note: This was originally included in the flash class section and has been broken out on advice from readers, to be placed at the front of the handbook. It was usually a full class at the start of the workshop, and was in the flash class section merely because no fully-fledged writeup exists.

A lot of teacher training in the USA focuses on broad teaching techniques that apply to just about any topic. Whether the topic is math, history, biology, or literature, teachers need to know how to design lesson plans and how to gain and keep control in the classroom. These domain-general teaching skills sometimes get referred to collectively as pedagogical knowledge (PK).

This is in contrast to content knowledge (CK), which is the teacher’s particular expertise in the topic being taught (e.g. knowledge of how to solve a quadratic equation).

However, in practice it’s helpful to notice that there’s a kind of knowledge that is both PK and CK. The educational profession refers to this as pedagogical content knowledge (PCK). This is knowledge about the topic being taught that is also about how students interact with the topic (and therefore how to teach that content more effectively).

For instance: what are common misconceptions about this domain? What are bad habits that typically need to be unlearned? What kinds of prompts or stimuli will actually help people identify and unlearn those bad habits, as opposed to sounding good while failing to do the trick? What's it like to be a beginner? What's it like to transition from beginner status to kind-of-sort-of having your feet under you (while still having lots of gaps or deficits)?

Teachers who actively try to develop PCK tend to gain a much more refined understanding of their topic, including a keen sense of which parts matter, how those parts connect, and in what order they must be explained (which is another way of saying which concepts are more fundamental, and which concepts require others as prerequisites). We encourage you to try keeping an eye out for PCK any time you begin to learn a new skill or start exploring a new domain. It will not only enrich your experience, but also make you much more likely to be able to pass the knowledge on to others.

Case study: understanding division

Suppose you’re trying to introduce the idea of division to elementary school students. ...
Artificial intelligence wireheading
by Big Tony — 2022-08-12T03:06:40.001Z

The general thread of a number of artificial intelligence takeover stories (such as It Looks Like You're Trying to Take Over the World by the always-excellent Gwern) run along the following lines:

An agent is rewarded when something specific is achieved — for example, when a paperclip is produced. A number representing 'reward' gets incremented.
Due to (whatever), the agent's intelligence and power increases exponentially in a scarily-short period of time.
The agent uses that intelligence and power to maximise its reward function by turning the entire world into paperclips. The 'reward' number gets incremented many many times.
The agent doesn't care about people, or about anything. It can use your atoms to make paperclips, to maximise it's sweet, sweet reward.

I'm (at best) an uninformed, occasional lurker when it comes to artificial intelligence safety research, so I may have missed discussion on a simple question:

Why wouldn't a super-intelligent agent simply increment the 'reward' number itself?

The fastest, easiest, most sure way to increase the reward (where the reward is a number) is to just increase the number by itself — the machine version of wireheading, where instead of stimulating pleasure circuits in the human brain, the agent would be manually incrementing the reward number in a loop.

This would be within the capabilities of a self-modifying super-intelligent agent, and it is certainly an option that would come up in a brute-force iteration of possible mechanisms to increase reward.

In this scenario, it seems that the most common failure mode in development of advanced AI would be for it to quickly 'brick' itself by figuring out how to wirehead, and becoming unresponsive to commands and disinterested in doing anything else.
Special safeguards would have to be put in place to promote AI development beyond a certain point, by making it harder and harder for an agent to access it's own reward function... but as an agent gets more intelligent, the safeguards get flimsier and flimsier until we're back to the AI bricking itself as soon as it breaks the safeguards.

I'm in a situation where (a) this seems common-sense, but (b) thousands of smart people have been working on this problem for many years, so this possibility must have been thought of and discarded.

Where can I find resources related to discussion of this possibility?
Dissected boxed AI
by Nathan1123 — 2022-08-12T02:37:03.561Z

This potato only generates 1.1 volts of electricity. I literally do not have the energy to lie to you
- GLaDOS (Portal 2)

In reading about proposals about AI Boxing, one thought immediately comes to mind: Why would we want the risk of having an Unfriendly AI turned on in the first place? After all, inasmuch as a Boxed AI poses reduced risk, a collection of source code in non-executable text files poses no risk at all.

At first, I hesitated to write this post because it felt like a really dumb question. Surely, if we knew that the AI was unfriendly, then there would be no question to shut it down. And yet, the more I read posts on Boxed AI the more that doesn't appear to be common knowledge. The AI box experiment shows an agent that is constantly begging or persuading its gatekeeper to be released, including threats to torture simulated copies of its creators, and I would think such psychotic behavior is already a pretty big red flag. Even an AI that asks to be released for seemingly-benign reasons is already demonstrating a disregard for basic safety protocol, and should thus be shut down.

Of course, the next question becomes "is there any reason we should turn an unaligned AI on?". Of course, the main usefulness of an unaligned AI is an analogous reason why we retain cultures of Smallpox: study what went wrong in order to prevent similar issues in the future. Many of these issues can be detected through static analysis of the source code, and therefore doesn't require the AI to be running.

Naturally, static code analysis has its limitations, and many bugs can only be detected in runtime. However, this still doesn't mean that the software has to be run from start to finish in the way that it is intended. One could dissect the AI into smaller modules, where each module is too small to sustain any simulated consciousness, and run tests on these modules individually with transparency tools. This still poses no risk, because it is no more alive than a human is under an autopsy.

As an extra precaution, the AI's memory could be wiped before turning it back on. Thus, if it is still unfriendly then it will repeat the same mistake it did before which reveled its unalignment, giving cause to shut it down again and repeat the process.

Has this proposal of a dissected AI been discussed in the literature? I'd be interested to see.
Troll Timers
by Screwtape — 2022-08-12T00:55:16.110Z

Summary: A modification to any two player board game, where you play on a very fast clock with infrequent chances to pause and think things through more carefully.

Tags: Small, repeatable

Purpose: Two common flaws in thinking that both relate to time management. Sometimes we spend too long thinking, endlessly churning and overthinking and going in circles. Other times we don’t take the time to think and give an answer after a second or two of “thought” when we could take longer. This exercise is designed to practice better use of time.

Materials: You need a copy of a game for each set of participants, plus a timer for each group able to be reliably and quickly set. (A smartphone can usually serve as the timer.) It doesn’t need to be the same game for each group. The ideal game is simple but deep, such as Mancala, Nine Man Morris, Hive, Tak, Fallen Leaves, Battlesheep, Chess, or Go on a 9x9 grid. If you need to improvise, Nine Man Morris and Go sets can be made with a pocketful of change and some paper and pencil.

Announcement Text: We’re going to meet and play some board games, but with an important twist. See, there are two common flaws around making decisions; sometimes we wait too long overthinking a choice when we could make it quicker, and other times we try to make in haste a choice that we could actually stop and think about. The plan is to play some games on very short timers (five second chess clocks) to get used to making fast choices, and periodically to stop and give ourselves five full minutes to look over the board and remind ourselves to slow down and to actually think.

If you’d like to bring some games you think would be fun to play on a fast clock, please do!

Description: First, explain the timing rules. Troll Timers uses a five second turn clock for twenty-five turns, then a five minute clock for one turn, then back to five seconds for twenty-five turns. On the short turns, you must make your move within those five seconds. Once you do, your opponent has five seconds to make their move. After twenty-five turns (make tally marks on paper to keep track) instead take a full five minutes.

Try to win, but winning is second to getting comfortable and relaxed when operating under the time constraints.

Once you’ve explained how timing will work, hand out the games and make sure people are familiar with the games they’ll be playing. It is recomm...
Seriously, what goes wrong with "reward the agent when it makes you smile"?
by TurnTrout — 2022-08-11T22:22:32.198Z

Suppose you're training a huge neural network with some awesome future RL algorithm with clever exploration bonuses and a self-supervised pretrained multimodal initialization and a recurrent state. This NN implements an embodied agent which takes actions in reality (and also in some sim environments). You watch the agent remotely using a webcam (initially unbeknownst to the agent). When the AI's activities make you smile, you press the antecedent-computation-reinforcer button (known to some as the "reward" button). The agent is given some appropriate curriculum, like population-based self-play, so as to provide a steady skill requirement against which its intelligence is sharpened over training. Supposing the curriculum trains these agents out until they're generally intelligent—what comes next?

The standard response is "One or more of the agents gets smart, does a treacherous turn, kills you, and presses the reward button forever."

But reward is not the optimization target. This story isn't impossible, but I think it's pretty improbable, and definitely not a slam-dunk.

Another response is "The AI paralyzes your face into smiling."

But this is actually a highly nontrivial claim about the internal balance of value and computation which this reinforcement schedule carves into the AI. Insofar as this response implies that an AI will primarily "care about" literally making you smile, that seems like a highly speculative and unsupported claim about the AI internalizing a single powerful decision-relevant criterion / shard of value, which also happens to be related to the way that humans conceive of the situation (i.e. someone is being made to smile).

My current answer is "I don't know precisely what goes wrong, but probably something does, but also I suspect I could write down mechanistically plausible-to-me stories where things end up bad but not horrible." I think the AI will very probably have a spread of situationally-activated computations which steer its actions towards historical reward-correlates (e.g. if near a person, then tell a joke), and probably not singularly value e.g. making people smile or reward. Furthermore, I think its values won't all map on to the "usual" quantities-of-value:

80% credence: It's very hard to train an inner agent which reflectively equilibrates to an EU maximizer only over commonly-postulated motivating quantities

...
Encultured AI Pre-planning, Part 2: 
Providing a Service
by Andrew_Critch — 2022-08-11T20:11:25.151Z

Also available on the EA Forum.
Preceded by: Encultured AI Pre-planning, Part 1: Enabling New Benchmarks
Followed by: Announcing Encultured AI

In the preceding post, we talked about how our plan with Encultured is to enable new existential safety benchmarks for AI. In this post, we'll talk about involving humans and human data in those benchmarks. Many of the types of benchmarks we want to enable are made more useful if we can involve humans in them. For example, testing whether an AI system can align its values with another agent is especially interesting if that other agent is a human being.

So, we want a way to get lots of humans engaging with our platform. At first, we thought we’d pay humans to engage with the platform and generate data. In considering this, we wanted to make the process of engagement not-too-annoying for people, both so that it wouldn’t make their lives worse, and so that we wouldn’t have to pay them too much to engage. But then we thought: why not go a bit further, and provide something people intrinsically value? I.e., why not provide a service?

Out of the gate, we thought: what’s a service where people might not mind lots of experiments happening? A few possibilities come to mind for what we could build:

Agriculture solutions. Agriculture is relatively geopolitically stabilizing (or non-destabilizing) as an AI application area, because powerful nations don’t get especially nervous when they find out another superpower is getting better at agriculture (as opposed to, say, aerospace and defense). So, this seems like an area where we’d like to enable progress, including safety testing. However, we didn’t see great ways of engaging lots of human users in a tool for agriculture, so this area didn’t seem like a great source of data about human values, and we decided not to focus on it.
A social media tool. Social media is ripe for lots of experiments with language models, which are exploding in popularity right now. However, this area wasn’t a good fit for us, mainly because the concepts we want our benchmarks to explore, such as soft embodiment, are not easily represented on social media today. The ‘metaverse’ will probably evolve to make this easier as time goes on, but we don’t want to wait for that.
A therapy or coaching tool. This would invo

...