trentmkelly/LessWrong-43k huggingface dataset:
I only believe in the paranormal
by Malmesbury — 2022-05-24T12:15:16.619Z

(Cross-posted from Telescopic Turnip)

Recommended soundtrack for this post

The Wiktionary defines paranormal as “that (ostensibly) cannot be explained by what scientists know”. This is to be distinguished from Real Science, which is about – wait, this definition of paranormal corresponds exactly to what scientific researchers spend their time investigating in their labs.

Put this way, every researcher is a paranormal researcher, and science labs are places where an unusual level of paranormal activity occurs. I’m talking about really weird spooky paranormal phenomena like B-form eDNA flipping into Z-form to make lattices within bacterial biofilms. And don’t even ask me about The Vault. Over centuries, many macroscopic magical phenomena have been explained, leading to the impression that magic has disappeared. But look at life in a microscope, and everything is magical and mysterious again.

And not only the Universe is full of magical phenomena; human wizards are also surprisingly common. They are walking among us. Magic comes in roughly two forms: the one where wizards can predict the future, and the one where wizards can do impossible things.

Compare climate change and homeopathy. Most climate experts agree that climate change is human-made, so if you Believe in Science, you should believe in climate change. This is a terrible argument. I would reply that homeopathy experts also agree that homeopathy is effective. They have diplomas, they publish in scientific journals, and they even follow the scientific method. They may fall for p-hacking and publication bias, but so do people in every other field.

However, only one of these fields involves actual magic. In 2013, powerful wizards from the IPCC predicted how sea levels would change over the next few years. Now that the next few years have passed, we can compare the predictions to the most recent measurements, and it appears that their predictions were correct[1]. This is divination. They are literally oracles. I have no understanding of the underlying climatology, but I saw them perform magic, so I’m inclined to believe their reports.

On the other hand, I don’t believe in homeopathy – precisely because I’ve never seen homeopaths perform anything even remotely paranormal or mysterious. Like, you know, miraculously curing diseases. Homeopathy is boringly normal.

Likewise, if you just told me about the equations of q...
Why I Define My Experience At the Monastic Academy As Sexual Assault
by HS2021 — 2022-05-24T08:17:42.686Z

Trigger Warning: This post includes highly triggering details of sexual assault.

***The views expressed here are my own. Hereafter any italics used will indicate that I am quoting another person or source. ***

In December 2021 I published an

OPEN LETTER TO THE MONASTIC ACADEMY

detailing experiences I had at the

Monastic Academy’s

(MA) California branch OAK. One of these experiences was a sexual assault, though this was not the sole or even the primary focus of my letter — which was to focus on the MA’s response to the incident; to offer my critiques and feedback on the MA’s programs, structures, practices, and policies; and to express my concern for the safety and wellbeing of current and prospective participants in the MA’s training program. I’d like to follow up on this aspect of my experience, and in particular why I do consider it to have been sexual assault, which is the purpose of this post. This post here will highlight aspects of relevant state and federal laws as well as aspects of my own personal experience so that readers can better understand both the definitions and laws that exist regarding sexual assault and my view of the incident in November of 2020.

This post was written in response to Alex Flint’s post

: “Gracefully Correcting Uncalibrated Shame”

which I believe to be relevant since it is written in response to my open letter. Alex writes that after speaking to various “spiritual teachers” [none of whom have expertise in understanding sexual violence] he “has concluded that sexual assault is a completely inappropriate way to describe it” and that he believes what I have written about the MA to be “straightforward mischaracterizations.” Notably, Soryu Forall seems to be Alex’s primary source of guidance on this matter. Alex describes Soryu as “shockingly clear in his discernment” [regarding the matter of whether or not Alex is guilty of sexual assault]. In Alex’s post, it seems like Soryu is actively supporting Alex’s denial of responsibility for his actions and his impact. Soryu has direct and obvious conflicts of interest such as himself being directly involved in the covering up of this incident whether intentionally or as a result of negligence — and therefore could directly benefit from Alex’s denial and defense of the MA. It is therefore inappropriate for Soryu to advise Alex on these matters and if Alex were engaged in truth-seeking and ...
The No Free Lunch theorems and their Razor
by Adrià Garriga-alonso — 2022-05-24T06:40:32.748Z

The No Free Lunch (NFL) family of theorems contains some of the most
misunderstood theorems of machine learning. They apply to learning[1] and
optimization[2] and, in rough terms, they state:

All algorithms for learning [respectively, optimization] do equally well at
generalization performance [cost of the found solution] when averaged over all
possible problems.

This has some counterintuitive consequences. For example, consider a learning
algorithm that chooses a hypothesis with the highest accuracy on a training set.
This algorithm generalizes to a test set just as well as the algorithm which
chooses the hypothesis with the lowest accuracy on training! Randomly guessing
for every new instance is also just as good as these two.

The NFL theorems thus seem to show that designing an algorithm that learns from
experience is impossible. And yet, there exist processes (e.g. you, the reader
of this sentence) which successfully learn
from their environment and extrapolate to circumstances they never experienced
before. How is this possible?

The answer is that problems encountered in reality are not uniformly sampled
from the set of all possible problems: the world is highly regular in specific
ways. Successful learning and optimization processes (e.g. the scientific
method, debate, evolution, gradient descent) exploit these regularities to
generalize.

If NFL theorems don't usually apply in reality, why should you care about them?
My central claim is that they are essential to to think about why and when learning
processes work. Notably, when analyzing some process, it is important to state
the assumptions under which it can learn or optimize.
Sometimes it is possible to test some of these assumptions, but
ultimately unjustifiable assumptions will always remain.

NFL theorems also allow us to quickly discard explanations for learning as
incorrect or incomplete, if they make no reference to the conditions under which
they apply. I call this the No Free Lunch
Razor for theories of
learning.

In this post, I will:

State a simple NFL theorem in detail and informally prove it.
I describe some variants of it and
the link between NFL theorems and Hume's problem of induction.

Discuss when the NFL theorems are and aren't applicable in
practice, and what they tell us about
analyzing processes for generating knowledge that extrapolate (or
interpolate).

Introduce the NFL
razor
and

...
Synthetic Media and The Future of Film
by ifalpha — 2022-05-24T05:54:22.073Z

It was long believed that the first jobs to be obsoleted by AI would be lawyers and accountants, as those seemed the prime targets. After all, creativity has hardly been the forte of computers for the past half-century, being almost exclusively the product of human effort. However, in recent years, something has begun to change significantly. Widely introduced to the public via OpenAI's original DALL·E model, text-to-image has captured the imaginations of countless individuals who were under the impression that such advancements were still decades away. As even more advanced models rear their heads, such as DALL·E 2 and the (as of writing) brand new Imagen, we can clearly note that the quality of images is increasing at an incredibly rapid pace. While there have been many posts already written on the limitations of DALL·E 2, it is worth highlighting that newly released Imagen has already solved several of the listed issues, such as text generation and object colorization. All this is to say that text-to-image models are already remarkable, and they've hardly been around for half a decade.

Beyond current models

While existing solutions to the text-to-image problem are incredible, there is clearly a lot of room left to grow. The images are almost photorealistic, and some are even close enough to fool the uninformed, but close inspection always reveals minor flaws. Let's assume that the current rate of progress continues. This realistically leaves us with only a handful of years before images can be generated that are entirely indistinguishable from real photographs. (If you believe that this point will never be reached, I would love to hear your reasoning.) Once that mountaintop is summitted, where is there left to go? Higher resolution? Quicker generation? The answer is obvious: video.

There are already models that are beginning to tackle text-to-video generation, such as NÜWA and the more recent (but not-as-appealingly named) Video Diffusion Models.

"Play golf at swimming pool" from NÜWA

"Shinjuku Time Lapse" from Video Diffusion Models

While the results are low quality and nowhere near the text-to-images models of today, they do bear a striking resemblance to the image models of just a few years ago.

If we are to be bold and assume a rate of progress identical to the text-to-image models, we should expect to see near-photorealistic video generation wit...
Complex Systems for AI Safety [Pragmatic AI Safety #3]
by Dan H — 2022-05-24T00:00:58.873Z

This is the third post in

a sequence of posts

that describe our models for Pragmatic AI Safety.

It is critical to steer the AI research field in a safer direction. However, it’s difficult to understand how it can be shaped, because it is very complex and there is often a high level of uncertainty about future developments. As a result, it may be daunting to even begin to think about how to shape the field. We cannot afford to make too many simplifying assumptions that hide the complexity of the field, but we also cannot afford to make too few and be unable to generate any tractable insights.

Fortunately, the field of complex systems provides a solution. The field has identified commonalities between many kinds of systems and has identified ways that they can be modeled and changed. In this post, we will explain some of the foundational ideas behind complex systems and how they can be applied to shaping the AI research ecosystem.

Along the way, we will also demonstrate that deep learning systems exhibit many of the fundamental properties of complex systems, and we show how complex systems are also useful for deep learning AI safety research.

A systems view of AI safety

Background: Complex Systems

When considering methods to alter the trajectory of empirical fields such as deep learning, as well as preventing catastrophe from higher risk systems, it is necessary to have some understanding of complex systems. Complex systems is an entire field of study, so we cannot possibly describe every relevant detail here. In this section, we will try merely to give a very high level overview of the field. At the end of this post we present some resources for learning more.

Complex systems are systems consisting of many interacting components that exhibit emergent collective behavior. Complex systems are highly interconnected, making decomposition and reductive analysis less effective: breaking the system down into parts and analyzing the parts cannot give a good explanation of the whole. However, complex systems are also too organized for statistics, since the interdependencies in the system break fundamental independence assumptions in much of statistics. Complex systems are ubiquitous: financial systems, power grids, social insects, the internet, weather systems, biological cells, human societies, deep learning models, the brain, and other systems are all complex systems.

It ca...
Contra Astral Codex Ten's "On Sexy In-Laws"
by Dalton Mabery — 2022-05-23T22:53:54.620Z

I.

Trying to change the behavior of someone who acts off instinct by persuading them with logic and reason is a formula destined for failure. You cannot combat intuition or "feeling" through data, logical conclusions, or experimental studies. The two are in direct opposition to one another.

Scott Alexander, on his

Astral Codex Ten blog

, has described these two driving forces as "built in micro-managing instincts" and "human reason." In an effort to explain the two, he illustrates the all too common story of a daughter wanting to marry someone she's "in love with" while her parents wish for her to marry someone else, usually someone who has a prestigious profession. The daughter is acting on instinct. For whatever reason, she believes this guy is a good life partner for her. He could be funny, handsome, or smart, but whatever it is, she's basing her decision on an instinct. The parents, who want their daughter to have a good husband that will support her, be a good father, and give them incredible grandchildren, are acting on reason. They know this other guy has a more prestigious profession than the current love affair, which means more money and a stable life, which leads to healthy offspring. It makes the most logical sense for the daughter to pick the guy behind door number two. Often, though, she goes with her instinct and not rationality. Is this the better decision? Or just how most people decide and it seems to be working out, so people keep doing it?

That is a perfect example of the dichotomy between human intuition and human reason, which makes me ask, which is better? Or, is there not a better way and the best possible route is to combine the two–using instincts to guide your initial options and logic to do your due diligence?

II.

Intuition, or "gut-feeling" as it is casually called, is part of the automatic part of your brain that processes information.

Psychology Today

gives the example of "highway hypnosis," which is what happens when you leave your house in the morning and arrive at work fifteen minutes later but you don't have a conscious thought about the drive there. The automatic part of your brain took over and did what it does best, rely on past situations (your drive every single day to work) to formulate and create the best outcome. In this case, it's your arriving to work safe and sound.

A paper on

intuition and insight

explains that i...
Why I'm Worried About AI
by peterbarnett — 2022-05-23T21:13:59.976Z

Preamble

In this sequence of posts, I want to lay out why I am worried about risks from powerful AI and where I think the specific dangers come from. In general, I think it's good for people to be able to form their own inside views of what’s going on, and not just defer to people. There are surprisingly few descriptions of actual risk models written down.

I think writing down your own version of the AI risk story is good for a few reasons

It makes you critically examine the risk models of other people, and work out what you actually believe.
It may be helpful for finding research directions which seem good. Solving a problem seems much more doable if you can actually point at it.
It seems virtuous to attempt to form your own views rather than just the consensus view (whatever that is).
People can comment on where your story may be weak or inconsistent, which will hopefully push towards the truth.

Additionally, I have some friends and family who are not in EA/AI-safety/Longtermism/Rationality, and it would be nice to be able to point them at something describing why I’m doing what I’m doing (technical AI-safety). Although, admittedly my views are more complicated than I initially thought, so this isn’t a great first introduction to AI-risk.

I don’t expect much or any of these posts to be original and there will be many missing links and references. This can maybe be viewed as a more quick and dirty

AGI safety from first principles

, with less big picture justification and more focus on my specific risk models. In general, I am concerned most about

deceptively aligned

AI systems, as discussed in

Risks from Learned Optimization

.

How do we train neural networks?

In the current paradigm of AI, we train neural networks to be good at tasks, and then we deploy them in the real world to perform those tasks.

We train neural networks on a training distribution

For image classifiers, this is a set of labeled images. E.g. a set of images of cats and dogs with corresponding labels.
For language models like GPT-3, this is a very large amount of text from the internet.
For reinforcement learning, this is some training environment where the AI agent can ‘learn’ to take actions. For example, balancing a pole on a cart or playing the Atari game Breakout.

We start with an untrained AI system and modify it to perform better on the training distr...
Bits of Optimization Can Only Be Lost Over A Distance
by johnswentworth — 2022-05-23T18:55:17.396Z

When we think of “optimization” as

compressing some part of the universe into a relatively small number of possible states

,

it’s very natural to quantify that compression in terms of “bits of optimization”

. Example: we have a marble which could be in any of 16 different slots in a box (assume/approximate uniform probability on each). We turn the box on its side, shake it, and set it down so the marble will end up in one of the 4 slots on the downward side (again, assume/approximate uniform probability on each). Then we’ve compressed the marble-state from 16 possibilities to 4, cut the state-space in half twice, and therefore performed two bits of optimization.

In the language of information theory, this quantity is the

KL-divergence

between the initial and final distributions.

In this post, we’ll prove our first simple theorem about

optimization at a distance

: the number of bits of optimization applied can only decrease over a distance. In particular, in our optimization at a distance picture:

… the number of bits of optimization applied to the far-away optimization target cannot be any larger than the number of bits of optimization applied to the optimizer’s direct outputs.

The setup: first, we’ll need two distributions to compare over the optimizer’s direct outputs. You might compare the actual output-distribution to uniform randomness, or independent outputs. If the optimizer is e.g. a trained neural net, you might compare its output-distribution to the output-distribution of a randomly initialized net. If the optimizer has some sort of explicit optimization loop (like e.g. gradient descent), then you might compare its outputs to the initial outputs tested in that loop. These all have different interpretations and applications; the math here will apply to all of them.

Let’s name some variables in this setup:

Optimizer’s direct outputs: A (for “actions”)
i’th intermediating layer: Mi (with M0=A)
Reference distribution over everything: P[A,M|ref]
Actual distribution over everything: P[A,M|opt]

By assumption, only the optimizer differs between the reference and actual distributions; the rest of the Bayes net is the same. Mathematically, that means P[Mi+1|Mi,opt]=P[Mi+1|Mi,ref]=:P[Mi+1|Mi] (and of course both distributions factor over the same underlying graph).

Once we have two distributions over th...
[Alignment] Is there a census on who's working on what?
by Cedar — 2022-05-23T15:33:28.267Z

I'm new to the Rationality / EA community, and I've been getting the sense that the best use of my skills and time is to try to contribute to alignment.

Currently, my focus has been guided by top posts / most often mentions of names, here on LW. E.g. "I see John Wentworth post a lot. I'll spend my time investigating him and his claims".

The problem with that is, I have some instincts developed in large communities that gives me a sense that everybody is already working on this, and that if I let my attention roam the natural way I am going to end up in a big pool doing duplicate work as everybody else.

Is there anything here on what demographics are working on what problems?
[fiction]A Question of Perspective
by Elias — 2022-05-23T15:06:55.650Z

Disclaimer: This text was written to reflect a state of mind with no other audience than myself in mind. Only later did I get the impulse to publish this. I would therefore like to emphasize, that this is not intended as a guide, a recommendation for behaviour, or anything like that.

Instead, the text is intended to evoke a specific frame of mind in the reader, which I found hard to communicate (even to myself) when I was not in this state of mind. But I have since heard, that some people were identifying strongly with this, so, this is to them then.

I’m falling when I wake. Soon after, I hit the ground. As I look up, far above me, I see the sky, though only a small window of it shines down to me without being blocked by the sides of the abyss I find myself in.
The walls are not of earth or rock, but of tons of pages and various items that seem to have been ordered at some point, but now just form a mountainous plateau around me. I must have fallen down a crevasse. Looking around, it is clear to see that if I can impose some order on these walls, I should be able to climb out and walk the surface. So, I set to work. This paper goes here, that is trash, I can burn it to make more space, this is doesn’t need doing, it can wait… After a long day, there are the beginnings of a stairway upwards. I go to sleep.
Upon awakening on the next day, I look at the mountain of work, I still have to deal with. I can’t find a place to start, so I don’t.
On the third day I just choose a starting point, almost at random, fearing the decay of my previous work, which has already set in. I can see the staircase leaning away from the wall a bit and I need to spend a few hours to fix it. Then, I look for a new thing I can sort, but the walls are just made of stuff, and everything I can see is tangled up in a mess of other things. As it starts getting darker, my frustration turns to anger. Why is there so much stuff in the first place, I should just burn it all and be done with it. Thankfully, I realize that I would burn up with it, just in time, before I can start the flames. Defeated, I go to sleep.
The next morning, I start early, trying to avoid the mistakes of the previous days, trying to not think too much, so I don’t lose myself in the monumental task.
This proves to be a mistake.
As I pick up the first object, a chair that is too small, it catches...
Finding Your Voice
by  — 2022-05-23T14:43:19.842Z
Link: https://guildoftherose.org/articles/finding-your-voice

Most people hate to hear recordings of their own speech. "Oh God, do I really sound like that?" Many people think that their voice is "bad", which should be surprising because almost nobody has an objectively bad voice. But many people use their voices unskillfully, which can be improved.