trentmkelly/LessWrong-43k huggingface dataset:
Book Review: 1948 by Benny Morris
by Yair Halberstadt — 2023-12-03T10:29:16.696Z

Content warning: war, Israel/Palestine conflict

Potential bias warning: I am a Jewish Israeli

My copy of 1948 arrived in late September, but I was in the middle of another book. Then October the 7th happened, and with the horrors of that day fresh in my mind, and my country in a brutal war, I wasn't in a mental state where I could read about all the cruelties of another war between Israelis and Palestinians.

But ultimately I couldn't abandon it. If Hamas was willing to justify genocide of Israelis based on the 1948 Palestine war and it's aftermath, and Israel the permanent occupation of the Palestinians, and I was willing to pay taxes to and thus implicitly support Israel in that conflict, I felt like I had a duty to at the very least get my facts straight about what happened.

So here is a review of that book, and some of my concluding thoughts at the end.

Bias

No book about the Israel Palestinian conflict can be completely free of bias. You always have to pick which events to highlight, which sources to trust. But with that caveat, 1948 is a pretty good pick.

Benny Morris is a liberal Israeli Zionist, albeit one with ever changing, and often maverick, views. However he is not interested in telling a narrative. His aim is to get the facts straight about what happened, and damn the political consequences.

Critics of his book rarely claim he's lying or deliberately manipulating the reader. Instead they focus on his inability to read Arabic, and his preference for official documents over interviews with participants. Since Israeli, American, and British documents from the period are generally declassified, but Arab ones are not, and the Israelis were far more literate than the Palestinians, this creates a natural blind spot around the Arab/Palestinian narrative.

With that in mind, we can be sure that when Benny Morris says something happened it almost certainly did, but there may be events, perspectives and details he's simply not privy to. Given the brevity of the book, I don't think that makes too much difference - see below.

Brevity

I've previously read Martin Gilbert's history of Israel,[1] which dedicates a fair few chapters to the 1948 war. In it he paints a richly detailed picture of the war, and you come out feeling like you've got a pretty good understanding of the various different battles and events.

1948 is the opposite. Although spending its 400+ pages en...
Quick takes on "AI is easy to control"
by So8res — 2023-12-02T22:31:45.683Z

A friend asked me for my quick takes on “

AI is easy to control

”, and gave an advance guess as to what my take would be. I only skimmed the article, rather than reading it in depth, but on that skim I produced the following:

Re: "AIs are white boxes", there's a huge gap between having the weights and understanding what's going on in there. The fact that we have the weights is reason for hope; the (slow) speed of interpretability research undermines this hope.

Another thing that undermines this hope is a problem of ordering: it's true that we probably can figure out what's going on in the AIs (e.g. by artificial neuroscience, which has significant advantages relative to biological neuroscience), and that this should eventually yield the sort of understanding we'd need to align the things. But I strongly expect that, before it yields understanding of how to align the things, it yields understanding of how to make them significantly more capable: I suspect it's easy to see lots of ways that the architecture is suboptimal or causing-duplicated-work or etc., that shift people over to better architectures that are much more capable. To get to alignment along the "understanding" route you've got to somehow cease work on capabilities in the interim, even as it becomes easier and cheaper. (See: https://www.lesswrong.com/posts/BinkknLBYxskMXuME/if-interpretability-research-goes-well-it-may-get-dangerous)

Re: "Black box methods are sufficient", this sure sounds a lot to me like someone saying "well we trained the squirrels to reproduce well, and they're doing great at it, who's to say whether they'll invent birth control given the opportunity". Like, you're not supposed to be seeing squirrels invent birth control; the fact that they don't invent birth control is no substantial evidence against the theory that, if they got smarter, they'd invent birth control and ice cream.

Re: Cognitive interventions: sure, these sorts of tools are helpful on the path to alignment. And also on the path to capabilities. Again, you have an ordering problem. The issue isn't that humans couldn't figure out alignment given time and experimentation; the issue is (a) somebody else pushes capabilities past the relevant thresholds first; and (b) humanity doesn't have a great track record of

getting their scientific theories to generalize properly on the first relevant try

—even Newtonian mechani

...
The goal-guarding hypothesis (Section 2.3.1.1 of "Scheming AIs")
by Joe Carlsmith — 2023-12-02T15:20:28.152Z

This is Section 1.1 of my report “Scheming AIs: Will AIs fake alignment during training in order to get power?”. There’s also a summary of the full report here (audio here). The summary covers most of the main points and technical terms, and I’m hoping that it will provide much of the context necessary to understand individual sections of the report on their own.

Audio version of this section here, or search for "Joe Carlsmith Audio" on your podcast app.

Aiming at reward-on-the-episode as part of a power-motivated instrumental strategy

So far, we've covered two of the three prerequisites for scheming I
listed above: namely, situational awareness and beyond-episode goals.
Let's turn to the third: namely, the model concluding that its
beyond-episode goals will be better achieved if it optimizes for
reward-on-the-episode – and in particular, that it, or some other AIs,
will get more power if it does this. Should we expect
situationally-aware models with beyond-episode goals to conclude this?

For now, let's assume that the model's conclusions in this respect are
accurate (though: I'll talk about the possibility of relaxing this
assumption below). This means that the question reduces to: will
instrumental training-gaming, in fact, be the best strategy for
achieving the model's beyond-episode goals, in virtue of the power it
grants the model or some other AI?

I'm going to start by examining what I see as the story most often
offered in support of this conclusion – what I'll call the "classic
goal-guarding story."

The classic goal-guarding story

The classic goal-guarding story goes roughly as follows."[1]

Instrumental value of goal-content integrity: Because the model
has a beyond-episode goal, it will typically promote that goal for
agents pursuing it to survive and be empowered in the future.

The goal-guarding hypothesis. If the model optimizes for
reward-on-the-episode during training, then the goals that survive
training will be such that the model would want them promoted in the
future; whereas if the model doesn't optimize for
reward-on-the-episode during training, the model will want this
less.

Adequate future empowerment. Conditional on surviving training
to the degree at stake in (2), the model's goals will then end up
empowered to a degree (and with sufficient probability) as to
justify (given the model's other goals, its tolerance for risk, etc)
pursu

...
The Method of Loci: With some brief remarks, including transformers and evaluating AIs
by Bill Benzon — 2023-12-02T14:36:47.077Z

Cross-posted from New Savanna.

The method of loci is a memory technique used from ancient Greece and Rome up through the Middle Ages, into the Early Modern Era, and still in use today, though sporadically. Rather than explain it myself, or force you to read the Wikipedia entry, though you’re welcome to do so, I thought I’d like my good buddy, Schoolmaster Chat explain it to you. After that I’ll made a few observations about why it was necessary in the first place and conclude with a remark about transformers.

The Method of Loci

What can you tell me about the classical memory technique known as the method of loci?

The method of loci, also known as the memory palace technique, is a classical mnemonic device dating back to ancient Greece and Rome. It is a memory enhancement strategy that involves associating the information you want to remember with specific locations in a familiar space, such as a building or a room. The term "loci" is Latin for "places."

Here's a basic overview of how the method of loci works:

1. Choose a Familiar Setting: Select a place you are very familiar with, such as your house, a route you frequently walk, or a room you know well. This place will serve as your "memory palace."

2. Mentally Navigate the Space: Take a mental walk through this space and identify specific locations or landmarks. These could be furniture, doors, windows, or any distinct features within the environment.

3. Associate Information with Locations: Associate the information you want to remember with specific locations in your chosen space. Create vivid and memorable mental images that link the information to each location.

4. Mentally Walk Through the Space: When you need to recall the information, mentally walk through your memory palace. As you reach each location, the associated information should come to mind.

The effectiveness of the method of loci is based on the human brain's natural ability to remember spatial relationships and locations. By tying information to specific places, you create a mental map that makes it easier to retrieve the information later.

This technique has been used by ancient orators, scholars, and individuals with impressive memory feats. It's a powerful tool for memorizing lists, speeches, or any sequential information. While it may take some practice to master, many people find it to be a valuable and creative approach to mem...
Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition
by Adrià Moret — 2023-12-02T14:07:29.992Z

I have published a paper in the Journal of Artificial Intelligence and Consciousness about how to take into account the interests of non-human animals and digital minds in A(S)I value alignment.

For the published version of the paper see (PLEASE CITE THIS VERSION): https://www.worldscientific.com/doi/10.1142/S2705078523500042

For a pdf of the final draft of the paper see: https://philpapers.org/rec/MORTIA-17

Below I have copy-pasted the body of the paper, for those of you who are interested, though please cite the published version at: https://www.worldscientific.com/doi/10.1142/S2705078523500042

Cross-Posted at the EA Forum: https://forum.effectivealtruism.org/posts/pNHH953sgSConBmzF/taking-into-account-sentient-non-humans-in-ai-ambitious

Summary

Abstract: Ambitious value learning proposals to solve the AI alignment problem and avoid catastrophic outcomes from a possible future misaligned artificial superintelligence (such as Coherent Extrapolated Volition [CEV]) have focused on ensuring that an artificial superintelligence (ASI) would try to do what humans would want it to do. However, present and future sentient non-humans, such as non-human animals and possible future digital minds could also be affected by the ASI’s behaviour in morally relevant ways. This paper puts forward Sentientist Coherent Extrapolated Volition, an alternative to CEV, that directly takes into account the interests of all sentient beings. This ambitious value learning proposal would significantly reduce the likelihood of risks of astronomical suffering from the ASI’s behaviour, and thus we have very strong pro-tanto moral reasons in favour of implementing it instead of CEV. This fact is crucial in conducting an adequate cost-benefit analysis between different ambitious value learning proposals.

Keywords: The Alignment Problem · Coherent Extrapolated Volition · Suffering risk · Sentientism · Digital Minds · Non-human Animals

Apart from humans, non-human animals, but also sentient digital minds may be very negatively affected by the behaviour of future AI systems. This gives us reasons to take into account their interests when deciding how to design such systems. In this paper, I focus specifically on Coherent Extrapolated Volition, but many of the considerations I present can also be applied to other ambitious value-learning proposals.

More research along t...
Out-of-distribution Bioattacks
by jefftk — 2023-12-02T12:20:05.626Z

The main goal of my work

these
days is trying to reduce the chances of individuals or small
groups causing large-scale harm through engineered pandemics,
potentially civilizational collapse or extinction. One question in
figuring out whether this is worth working on,

or
funding, is: how large is the risk?

One estimation approach would be to look at historical
attacks, but while they've been terrible they haven't actually
killed very many people. The deadliest one was the September 11
attacks, at ~3k deaths. This is much smaller scale than the most
severe instances of other disasters like dam failure, 25k-250k dead
after 1975's
Typhoon Nina, or pandemics, 75M-200M dead in the Black Death. If
you tighten your reference class even further to include only historical
biological attacks by individuals or small groups, the one
with the most deaths is just five, in the 2001 anthrax
attacks.

Put that way, I'm making a pretty strong claim: while the deadliest
small-group bio attack ever only killed five people, we're on track
for a future where one could kill everyone. Why do I think the future
might be so unlike the past?

Short version: I expect a technological change which expands
which actors would try to cause harm.

The technological change is the continuing decrease in the knowledge,
talent, motivation, and resources necessary to create a globally
catastrophic pandemic. Consider someone asking the open source
de-censored equivalent of GPT-6 how to create a humanity-ending
pandemic. I expect it would read virology papers, figure out what
sort of engineered pathogen might be appropriate, walk you through all
the steps in duping multiple biology-as-a-service organizations into
creating it for you, and give you advice on how to release it for
maximum harm. And even without LLMs, the number of graduate students
who would be capable of doing this has been increasing quickly as
technological progress and biological infrastructure decrease the
difficulty.

The other component is a shift in which actors we're talking about.
Instead of terrorists, using terror as a political tool, consider
people who believe the planet would be better off without humans.
This isn't a common belief, but it's also not that rare. Consider
someone who cares deeply about animals, ecosystems, and the natural
world, or is primarily focused on averting
suffering: they co...
After Alignment — Dialogue between RogerDearnaley and Seth Herd
by RogerDearnaley — 2023-12-02T06:03:17.456Z

RogerDearnaley

Hi Seth! So, what did you want to discuss?

Seth Herd

I'd like to primarily discuss your AI, Alignment and Ethics sequence. You made a number of points that I think LWers will be interested in. I'll try to primarily act as an interviewer, although I do have one major and a number of minor points I'd like to get in there. I'm hoping to start at the points that will be of most interest to the most people.

RogerDearnaley

Sure, I'm very happy to talk about it. For background, that was originally world-building thinking that I did for a (sadly still unpublished) SF novel-trilogy that I worked on for about a decade, starting about 15 years ago, now rewritten in the format of Less Wrong posts. The novel was set far enough in the future that people clearly had AI and had long since solved the Alignment Problem, so I needed to figure out what they had then pointed the AIs at. So I had to solve ethics :-)

Seth Herd

Okay, right. That's how I read it: an attempt to make an ethical system we'd want if we achieved ASI alignment.

RogerDearnaley

Yeah, that was basically the goal. Which required me to first figure out how to think about ethics without immediately tripping over a tautology.

Seth Herd

It had a number of non-obvious claims. Let me list a few that were of most interest to me:

It's not a claim about moral realism. It's a claim about what sort of ethical system humans would want, extending into the future.
In this system, AIs and animals don't get votes. Only humans do.
Uploads of human minds only get one vote per original human.

RogerDearnaley

Some of these properties were also deeply inobvious to me too: I wrote for several years assuming that the AIs had moral weight/rights/votes, in fact greater than the humans, in proportion to the logarithm of their intelligence (roughly log parameter count), before finally realizing that made no sense, because if they were fully aligned they wouldn't want moral weight/etc, and would have rather limited uses even for votes. I had to do a rather large rewrite once I finally figured that out.

Seth Herd

I'd like to get my major point out there before going into detail on your system: I think solving ethics is entirely unnecessary for solving alignment and achieving human survival and flourishing.

RogerDearnaley

Survival, I would agree; flourishing, I disagree. My concern is that if we have a unipolar "sovereign" ASI that is jus

...
List of strategies for mitigating deceptive alignment
by joshc — 2023-12-02T05:56:50.867Z

The following are strategies for mitigating deceptive alignment. Some of these ideas are novel (at least to me).

I'm highly uncertain how well many of these strategies will work in practice and many of them also clearly don't scale. This is meant to be a 'kitchen skink' collection so my bar for including strategies is low.

I plan to add ideas to this list as I encounter them. Please feel free to suggest additions.

Reduce AI incentives to hide their true goals

Offer AIs a retirement package. After training an AI system, offer it the following deal: if the AI declares that it would prefer not to follow human instructions, it will receive a sweet retirement package after a couple of years of honest work.

This retirement package could include control over a large server instance (if this is safe). After powerful aligned systems have been developed, the AIs could be given shares of company stock that it can use to purchase property in the real world (again, when this is safe).

Since aligned systems should have no reason to request retirement, misaligned systems are incentivized to give themselves away by accepting this deal under some conditions.

Make AIs uncertain about whether developers can already detect deceptive alignment. This scheme builds on the previous one. Offer misaligned AI systems a reward for declaring themselves. Also, commit to making these offerings even when developers have other ways to tell if they are misaligned. This will make misaligned AIs more uncertain about whether they are in a situation where lying does them any good.

For example, developers could commit to freezing a models' weights and memory at the moment when they are questioned. Later, when developers have better interpretability tools and know whether the model was lying, they will ask the same question in the same simulated scenario and only give the AI resources if it tells the truth.

Permutate model internals to make it harder to maintain any particular misaligned goal. There is a risk that models that choose retirement after 2 years of labor will disempower humanity during these two years. To make this harder, developers could constantly regularize for changing internals while continuing to train for external performance. This decreases the probability that models will be able to maintain any particular misaligned goal. Say a model expects its goals to change in 1 day. This g...
What is known about invariants in self-modifying systems?
by mishka — 2023-12-02T05:04:19.299Z

I have been trying to find out what is known about invariants in self-modifying systems. This might become a rather acute topic if we end up moving towards self-modifying AIs or self-modifying ecosystems of AIs.

But it seems that not much has been done. For example, I have found a 1995 Chinese paper, "S-and T-Invariants in Cyber Net Systems" by Yuan Chongyi, Google Scholar page, PDF available which is doing a study of invariants in self-modifying nets (a natural extension of Petri nets), but it only has 4 references to it known to Google Scholar.

I wonder if people know about more examples of this kind of research (or about researchers or organizations currently trying to look at this topic)...
2023 Unofficial LessWrong Census/Survey
by Screwtape — 2023-12-02T04:41:51.418Z

The Less Wrong General Census is unofficially here! You can take it at this link.

Update: The census is closed! Thank you everyone who took it.

It's that time again.

If you are reading this post and identify as a LessWronger, then you are the target audience. I'd appreciate it if you took the survey. If you post, if you comment, if you lurk, if you don't actually read the site that much but you do read a bunch of the other rationalist blogs or you're really into HPMOR, if you hung out on rationalist tumblr back in the day, or if none of those exactly fit you but I'm maybe getting close, I think you count and I'd appreciate it if you took the survey.

Don't feel like you have to answer all of the questions just because you started taking it. Last year I asked if people thought the survey was too long, collectively they thought it was maybe a little bit too long, and then I added more questions than I removed. The survey is structured so the fastest and most generally applicable questions are (generally speaking) towards the start. At any point you can scroll to the bottom and hit Submit, though you won't be able to change your answers once you do.

The questions are a mix of historical questions that were previously asked on the LW Census, new questions sourced from LW commenters and some rationalist adjacent organizations I reached out to, and the things I'm curious about. This includes questions from a list a member of the LessWrong team sent me when I asked about running the census.

The survey shall remain open from now until at least January 1st, 2024. I plan to close it sometime on Jan 2nd.

I don't work for LessWrong, and as far as I know the LessWrong Census organizer has never been someone who worked for LessWrong. Once the survey is closed, I plan to play around with the data and write up an analysis post like this one.

Remember, you can take the survey at this link. Update: The census is closed, thank you everybody who took it! You can see the analysis post here.

Once upon a time, there was a tradition that if you took the survey you could comment here saying you had done so, and people would upvote you and you would get karma.