trentmkelly/LessWrong-43k huggingface dataset:
Web4/Heaven - The Simulation
by Dunning K. — 2022-09-09T22:58:56.284Z

[This is a thought experiment / ranty idea. I don’t necessarily agree with everything I say hear but try to think the idea through to its end. Inspired by the hypothetical practice of heaven banning, hence why I call it Heaven.]

Web4 is the internet that is completely (or at least mostly) replaced by AI generated content. Any content you consume or interactions you have are replaced by AI: GPT-30 generates responses to you comments on Web4-Reddit and posts you favorite tweets on Web4-Twitter. Dalle-10 generates any image you might choose to look at in the internet and VideoDalle creates your favorite Web4-YouTube videos. Every internet user has their own version of the internet, completely tailored to them, perfectly adjusted to maximize everyone’s enjoyment of time spent on the internet, learned from the users’ individual feedback. Why would you need real Reddit posters or Instagram influencers, when Web4 Instagram contains only images that are perfectly generated specifically you?

You might say, posting a comment under a YouTube can provide helpful feedback to the content creator, but your comments under Web4-YouTube videos can also influence what videos VideoDalle will show you in the future.

Some people think we already live in a simulation — this is that but for the internet. Even if you know that your whole online experience is simulated, how many of your online interactions ever make it into the real world? Does it matter whether there are actually real people behind those?

Sure, there is still space to get posts from your real life friends on Web4-Facebook. Maybe you even choose to have generated content be interspersed with real people, this way you can still make real life friends on the web. Maybe fake YouTube influencers will tell you about real-world news stories, courtesy of MultiModalGPT-20.

No more negative interactions on the internet (unless you want some). Presumably content could even be optimized to fulfill some kind of specific goal you set: Learning, intellectual growth, limiting time spent on the internet.

Having access to the entire internet and sufficiently capable content generation AI systems should be sufficient to create a simulated version of it.

Is this a dystopian vision? Do you take the red pill or the blue pill?
Evaluations project @ ARC is hiring a researcher and a webdev/engineer
by Beth Barnes — 2022-09-09T22:46:47.569Z

​The evaluations project at the Alignment Research Center is looking to hire a generalist technical researcher and a webdev-focused engineer. We're a new team at ARC building capability evaluations (and in the future, alignment evaluations) for advanced ML models. The goals of the project are to improve our understanding of what alignment danger is going to look like, understand how far away we are from dangerous AI, and create metrics that labs can make commitments around (e.g. 'If you hit capability threshold X, don't train a larger model until you've hit alignment threshold Y'). We're also taking SERI MATS fellows.

About the project

Big picture

Once AI systems are close to being capable enough to be an x-risk, there should be procedures that labs follow to ensure they don’t build or deploy existentially dangerous models. We should have thought through the plausible paths to models gaining power, and ensured that models are not capable of these. There should be commitments that if model capabilities reach a certain level, labs will not scale up or deploy models until they have reached a certain level of alignment. In order to make this happen, we need good ways to measure both dangerous capabilities and alignment.

We can start preparing for this now by:

evaluating capabilities of current models
predicting how dangerous the next generation of models is likely to be
assessing the quality of our prediction methods by using them to predict the properties of generation n+1 given generation n, for historical generation models
working with labs to evaluate their models before they are deployed, as a warm-up to stronger commitments in the future
designing evaluations for alignment, deception etc to be run on future models

Doing this now has some additional benefits:

Evaluating capabilities of current models is helpful for predicting timelines, by giving us a more precise sense of what needs to change for models to be dangerous
Specifying what we’d actually measure to determine the alignment of some model is helpful for clarifying our thoughts about how we expect misalignment to emerge, and what exactly the alignment problem is
This work may produce demonstrations of models doing competent and scary behavior. This can be useful if we think people are underestimating the capabilities of current or near-future models and making bad decisions because of that

...
Swap and Scale
by Stephen Fowler — 2022-09-09T22:41:49.682Z

Produced as part of the SERI MATS program under John Wentworth
Thanks to Garrett Barker for pointing out you could think of Swap Symmetry as relabeling and to Matthias G. Mayer for catching my mistake in not multiplying the bias in Scale Symmetry.

Here are two ways you can change the weights of any network using ReLU's without changing the behaviour of the network. The first is a discrete symmetry and involves swapping nodes around, the second is a continuous symmetry and involves scaling the weights. (Astute readers will notice that this is where the title has come from.)

I’m currently investigating the behaviour of non-trivial symmetries in networks but that is going to be a much longer post and probably a month or two away. I thought describing the trivial ones would make for a fun bite-sized post to write on a Friday afternoon.

Trivial Symmetries

There are two types of symmetries that occur for any point in parameter space. I refer to these as the trivial symmetries. I’m unsure who the first person to point these out for ReLU neural networks was, but equivalent symmetries for networks using tanh activation functions were mentioned in Bishop’s 2006 “Pattern Recognition and Machine Learning” and cites “Functionally Equivalent Feedforward Neural Networks” (Kurkova and Kainen, 1994) as proving the case for networks in general. In terms of Machine Learning literature this is ancient, and I suspect the authors haven't even heard of Lesswrong.

Swap Symmetry

For any hidden layer, we can imagine permuting the weights and biases in such a way that the output to the next layer remains unchanged. Specifically, we apply one permutation to the weights in the “incoming weights” and the biases of the current layer, then apply the inverse permutation to the weights in the next layer. This has the effect of completely undoing the first permutation.
In the simplest example where our layer only has two nodes, we can imagine swapping two neurons around whilst retaining their original connections.

It may also be helpful to think of this symmetry as simply relabelling the nodes in a single layer. For every hidden layer, there are N! such symmetries, where N is the number of nodes in the layer.

Scaling Symmetry

The second symmetry occurs when we multiply the weights leading into a node by some non-zero real number α, multiply the bias by...
My emotional reaction to the current funding situation
by Sam F. Brown — 2022-09-09T22:02:46.301Z
Link: https://sambrown.eu/writing/trajan

I’m allowed to spend two days a week at Trajan House, a building in Oxford which houses the Center for Effective Altruism (CEA), along with a few EA-related bodies. Two days is what I asked for, and what I received. The rest of the time I spend in the Bodleian Library of the University of Oxford (about £30/year, if you can demonstrate an acceptable “research need”), a desk at a coworking space in Ethical Property (which houses Refugee Welcome, among other non-EA bodies, for £200/month), Common Ground (a cafe/co-working space which I’ve recommended to people as a place where the staff explicitly explain, if you ask, that you don’t need to order anything to stay as long as you like), a large family house I’m friends with, and various cafes and restaurants where I can sit for hours while only drinking mint tea.

I’m allowed to use the hot-desk space at Trajan House because I’m a recipient of an EA Long Term Future Fund grant, to research Alignment. (I call this “AI safety” to most people, and sometimes have to explain that AI stands for Artificial Intelligence.) I judged that 6 months of salary at the level of my previous startup job, with a small expenses budget, came to about £40,000. This is what I asked for, and what I received.

At my previous job I thought I was having a measurable, meaningful impact on climate change. When I started there, I imagined that I’d go on to found my own startup. I promised myself it would be the last time I’d be employed.

When I quit that startup job, I spent around a year doing nothing-much. I applied to Oxford’s Philosophy BPhil, unsuccessfully. I looked at startup incubators and accelerators. But mostly, I researched Alignment groups. I visited Conjecture, and talked to people from Deep Mind, and the Future of Humanity Institute. What I was trying to do, was to discern whether Alignment was “real” or not. Certainly, I decided, some of these people were cleverer than me, more hard-working than me, better-informed. Some seem deluded, but not all. At the very least, it’s not just a bunch of netizens from a particular online community, whose friend earned a crypto fortune.

During the year I was unemployed, I lived very cheaply. I’m familiar with the lifestyle, and – if I’m honest – I like it. Whereas for my holidays while employed I’d hire or buy a motorbike, and go travelling abroad, or scuba dive, instead my holidays would be...
AlexaTM - 20 Billion Parameter Model With Impressive Performance
by MrThink — 2022-09-09T21:46:30.151Z

Amazon trained a seq2seq model that outperforms GPT-3 on SuperGLUE, SQuADv2, and is not (that) far behind PaLM with 540 billion parameters.

Article: https://www.amazon.science/publications/alexatm-20b-few-shot-learning-using-a-large-scale-multilingual-seq2seq-model

Benchmarks:
[Fun][Link] Alignment SMBC Comic
by Gunnar_Zarncke — 2022-09-09T21:38:54.400Z
Link: https://www.smbc-comics.com/comic/align

Alt text: "We just have to create new bigotries faster than they can learn to live in harmony."

Bonus panel:

The author seems to allow sharing (there are multiple sharing buttons at the bottom). Here is the full link to the comic again: https://www.smbc-comics.com/comic/align

Here is Zach Weinersmith's Patreon: https://www.patreon.com/ZachWeinersmith

Note that there are AI/robot-related comics every now and then and you might want to follow him.
Gatekeeper Victory: AI Box Reflection
by Double — 2022-09-09T21:38:39.218Z

A few weeks ago, I made a post – [AI Box Experiment Are People Still Interested](https://www.lesswrong.com/posts/HrZer4yhegweXJ8RH/ai-box-experiment-are-people-still-interested) – about the AI Box Experiment. I had read about it in [Rationality: From AI to Zombies](https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible) and became fascinated. I wanted to win the challenge not to make any grand discoveries about AI Safety but simply because that's what you do when you meet an interesting challenge. I pushed myself to find a new strategy and use it to win the game. Sadly, the strategy I created was not a successful one (or maybe just not successful against DaemonicSigil, my Gatekeeper partner!). Despite this failure, I still learned many things about this game that can help future AI players to victory! (Actual AI researchers are not likely to get anything out of this.)

The start of my thinking process was (the following is pasted from my idea-generation notes):

"One emotion that humans cannot think themselves out of is disgust. Remembering that it is just a game, thinking irrationally, or any other technique I can think of will not work for the gatekeeper."

I looked for uses of this strategy but found none. I treated RationalWiki's omission of this strategy as evidence it hadn't been tried. (Notably, the similar strategy of boredom is listed and has been tried.)

I followed up by refining this idea of using disgust.

I considered the Nocebo Effect[1], ASCII character images (technically allowed in a text-only game)[2], and using the Gatekeeper's imagination against them[3].

But of course, there was a big issue with this plan: I'd have to suffer as much as the Gatekeeper in order to pull it off.[4] As a coder, I, of course, reached for code as the solution.[5] I then realized that I could cut out the code and simplify by bluffing. I could use only the warning of the horrible things the code would do to scare away the Gatekeeper and not need to code anything.

The warning I composed to maximize the probability that the Gatekeeper would quit is the following:

"I realized that phony promises and threats have no power, but the emotion of disgust retains its power.

I did not put myself through the same disgust that you will experience.

I found the most disgusting text on the internet, and I created a computer program to send ...
Interpreting Affordable Housing
by jefftk — 2022-09-09T19:40:03.565Z

In describing affordable housing you'll often see a project described
as "60% AMI", which can be confusing:

how is 60% of median income
affordable?—NUMTOT
comment

If you think that "60% AMI" housing charges 60% of the area median
income, I agree that doesn't sound affordable! The problem is,
"affordable housing" (and "60% of area median income") is an improper noun,
a technical term with a defined meaning that differs from how we
normally use the words.

When housing is described as "affordable", that means it costs 30% of
some specific income. If it's a "60% AMI" unit, that means it's
affordable to someone making 60% of the area median income. For
example, a 60% AMI unit in an area with a median income of $100k would
be 60% * 30% * $100k/y or $18k/y, which is $1500/month. A 30% AMI
unit would be half that, at $750/month. If you don't know about the
factor of 30% thing you end up thinking all this housing is ~3x more
expensive!

That said, this is a mildly silly way to do it: percentages of a
median rarely make sense, and the right way to do this is probably
with percentiles. Instead of targeting affordable housing at, say,
the 60% of median income, we should target a 30th-percentile income.
This is much more robust to changes in the shape of the income
distribution, especially in cases where areas get into some sort of
bimodal situation where a bit more than half the residents are making
a lot and a smaller but significant number are making much less.
London Rationalish Meetup 2022-09-11
by calmiguana — 2022-09-09T18:39:38.645Z

Usually, we start around 3pm, with people showing up from 2pm. This time we’ll be in an office building at 201 Borough High Street, SE1 1JA, which is a two-minute walk from Borough Underground Station. We are going to be let into the building at 3pm, so we will meet at Starbucks, directly opposite Borough Station, from 2pm until a few minutes before 3pm when we will walk across to the office.

As we are relying on being let into the building please note that latecomers may not be able to gain entry after 3.15pm. If you have any concerns or need to call on the day, please phone Phil on his mobile number 07792 009646.

The format of this event will be free discussion in Starbucks and discussing the readings when we get to the office.

Readings:

1. Popular education in Sweden: much more than you wanted to know

https://www.lesswrong.com/posts/tjxgbovwc5Ft7wrtc/popular-education-in-sweden-much-more-than-you-wanted-to-1

2. Cultivating and destroying agency

https://www.lesswrong.com/posts/rXX6vd2RrKHfePPfE/cultivating-and-destroying-agency

3. I don't know how to count that low

https://www.lesswrong.com/posts/koGbEwgbfst2wCbzG/i-don-t-know-how-to-count-that-low

We're a monthly London-based meetup for members of the rationalist diaspora. The diaspora includes, but is not limited to, LessWrong, Astral Codex Ten, rationalist tumblrsphere, and parts of the Effective Altruism movement.

You don't have to identify as a rationalist to attend: basically, if you think we seem like interesting people you'd like to hang out with, welcome! You are invited. You do not need to think you are clever enough, or interesting enough, or similar enough to the rest of us, to attend. You are invited.

If we're in a public space, there will usually be some way to identify us. These days, it's usually a piece of paper saying "rationalish meetup", and/or a cuddly eagle toy.
AI alignment with humans... but with which humans?
by geoffreymiller — 2022-09-09T18:21:50.627Z

Note: I'm cross-posting this from EA Forum (where I posted it on Sept 8, 2022), in case anybody on LessWrong or the AI Alignment Forum is interested in commenting; note that there were some very helpful suggested readings in the replies to this: https://forum.effectivealtruism.org/posts/DXuwsXsqGq5GtmsB3/ai-alignment-with-humans-but-with-which-humans

Updated tldr: If human aren't aligned with each other (and we aren't, at any level of social organization above the individual), then it'll be very hard for any AI systems to be aligned with 'humans in general'.

Caveat: This post probably raises a naive question; I assume there's at least a 70% chance it's been considered (if not answered) exhaustively elsewhere already; please provide links if so. I've studied evolutionary psych & human nature for 30 years, but am a relative newbie to AI safety research. Anyway....

When AI alignment researchers talk about 'alignment', they often seem to have a mental model where either (1) there's a single relevant human user whose latent preferences the AI system should become aligned with (e.g. a self-driving car with a single passenger); or (2) there's all 7.8 billion humans that the AI system should be aligned with, so it doesn't impose global catastrophic risks. In those relatively simple cases, I could imagine various current alignment strategies, such as cooperative inverse reinforcement learning (CIRL) being useful, or at least a vector in a useful direction.

However, there are large numbers of intermediate-level cases where an AI system that serves multiple humans would need to become aligned with diverse groups of users or subsets of humanity. And within each such group, the humans will have partly-overlapping but partly-conflicting interests.

Example 1: a smart home/domestic robot AI might be serving a family consisting of a mom, a dad, an impulsive teenage kid, a curious toddler, and an elder grandparent with Alzheimer's. Among these five humans, whose preferences should the AI try to align with? It can't please all of them all the time. They may have genuinely diverging interests and incommensurate preferences. So it may find itself in much the same position as a traditional human domestic servant (maid, nanny, butler) trying to navigate through the household's minefield of conflicting interests, hidden agendas, family dramas, seething resentments,...
Should you refrain from having children because of the risk posed by artificial intelligence?
by Mientras — 2022-09-09T17:39:55.709Z

Eli Lifland discusses AI risk probabilities here.

Scott Alexander talks about how everything will change completely in this post, and then says "There's some chance I'm wrong about a singularity, there's some chance we make it through the singularity, and if I'm wrong about both those things I'd rather give my kid 30 years of life than none at all. Nobody gets more than about 100 anyway and 30 and 100 aren't that different in the grand scheme of things. I'd feel an obligation not to bring kids into a world that would have too much suffering but I think if we die from technological singularity it will be pretty quick. I don't plan on committing suicide to escape and I don't see why I should be not bringing life into the world either.". I have never seen any convincing argument why "if we die from technological singularity it will" have to "be pretty quick".

Will MacAskill says that "conditional on misaligned takeover, I think like 50/50 chance that involves literally killing human beings, rather than just disempowering them", but "just" being disempowered does not seem like a great alternative, and I do not know why the AI would care for disempowered humans in a good way.

It seems to me that the world into which children are born today has a high likelihood of being really bad. Is it still a good idea to have children, taking their perspective into account and not just treating them as fulfilling the somehow hard-wired preferences of the parents?

I am currently not only confused, but quite gloomy, and would be grateful for your opinions. Optimistic ones are welcome, but being realistic is more important.
Notes on Resolve
by David Gross — 2022-09-09T16:42:00.721Z

This post examines resolve and decisiveness (and things in that bailiwick like determination, commitment, dedication, steadfastness, and firmness), as part of a sequence of posts about virtues. I mostly explore what other people have learned about these virtues, rather than sharing my own research or opinions about them, though I’ve been selective about what I found interesting or credible. I wrote this not as an expert on the topic, but as someone who wants to learn more about it. I hope it will help people who want to know more about these virtues and how to practice them skillfully.

What are these virtues?

In short, a person with these virtues makes conscious, deliberate commitments when it would be advantageous to do so, such that these commitments reliably govern their future actions.

These virtues concern a crucial pivot point in the process of deliberately taking a course of action. For the purposes of this discussion, I divide this process into the following stages:

Prepare for making decisions:
What sorts of decisions do you want to make? What do you want to do with your life? Do you have a “calling” or a sense of purpose? Answers to broad questions like those allow you to weigh your possible decisions against your big-picture goals and your ethos. You need wisdom to understand what sorts of decisions you want to make, temperance so that your desires prompt you to make decisions that align with your wisdom, know-how so that you have the skills you need to act on your decisions, ambition so that you have high-impact decisions to choose from, devotion to help motivate your follow-through, and courage so that you won’t flinch from hard decisions.
Discern that a decision needs to be made:
You need attention to notice that the opportunity for a decision has arisen, while virtues like duty, social responsibility, honor, care, valor, helpfulness, and concern help you understand which decisions are your responsibility.
Weigh the cases for different options:
Virtues like phrónēsis (practical wisdom), rationality, curiosity, and imagination help you discover and consider different possible courses of action and to anticipate their outcomes.
Make a decision:
The point where you stop weighing the case and finally commit to a course of action is where virtues like decisiveness, resolve, and so forth, come into play. (In addition, optimism, confidence, earnestness, and enth

...