trentmkelly/LessWrong-43k huggingface dataset:
[Sketch] Validity Criterion for Logical Counterfactuals
by DragonGod — 2022-10-11T13:31:22.918Z

Epistemic Status

Originally written in a 47-minute sprint without any sort of proof checking or review[1]. I've edited it hence, but it's still just a rough sketch.

This should be understood as a brief high-level summary of the idea. I would hopefully refine the thesis of this piece and present it in a clearer and more coherent form at some later point.

Preamble

I claim that there's a single criterion for the validity of a logical counterfactual. That logical counterfactuals are valid if and only if they meet this criterion, and that counterfactuals that satisfy this criterion suffice for reflective reasoning in e.g. logical decision theories.

I'll state the criterion, provide motivations/justification for the criterion, and explain how it might be used in practice.

The Epistemic Criterion

A logical counterfactual is valid with respect to a given agent if and only if the counterfactual is consistent with that agent's epistemic state.

I.e., given the agent's current knowledge/belief pool, the agent considers it possible that the counterfactual is true, or the agent does not explicitly know[2] the counterfactual to be false.

Motivations for the Epistemic Criterion

Desiderata for Logical Counterfactuals

Any compelling account of logical counterfactuals should satisfy two basic properties to be useful for any kind of reasoning.

Local Counterfactual Surgery

Changes to the truth value of a logical proposition should have "local changes". It should affect only propositions that are in some intuitive sense directly dependent on the counterfactual, and not arbitrary other propositions.

Explosion Resistant

It shouldn't be vulnerable to the principle of explosion. We don't want a situation where considering a logical counterfactual which has a false truth value in fact allows us to derive arbitrary false propositions.

The two desiderata are similar, but distinct. Locality isn't entailed by explosion resistance. Locality is not about not deriving false propositions, but the effect of a counterfactual being "confined" in some sense. Affecting only some beliefs, and not others.

Simple Model of Logical Systems for Counterfactual Surgery

One could naively model a logical system as a directed graph[3]:

Each node is a valid sentence of the language of the system
The earliest children of the root node are the axioms of the system

To make the graph a tree[4]
You can imag

...
How much does the risk of dying from nuclear war differ within and between countries? 
by amarai — 2022-10-11T11:55:34.789Z

It seems that being in a major city such as London or New York makes it much more likely that one dies in case of nuclear war than in the countryside or in a smaller town.
Also, some people seem to think that certain countries are much safer from nuclear war than others (e.g., compare New Zealand to the US), which also makes sense.
What seems much less clear to me is how large the difference in risk really is.

How would you model the probability of dying from nuclear war as a function of the distance from a major city in Europe or the US?
How would you estimate the a priori difference in risk of dying from a nuclear war between the highest and lowest-risk countries? Which are those countries?
I very much appreciate being linked to any resources that are useful for answering this question!
Did you enjoy Ramez Naam's "Nexus" trilogy? Check out this interview on neurotech and the law.
by fowlertm — 2022-10-11T11:10:57.964Z

I recently had the great pleasure of interviewing Dr. Allan McCay on the Futurati Podcast. He's a legal scholar and philosopher who focuses on questions at the interface of emerging neurotechnologies and the law, such as how privacy works when it's possible to decode mental images and mental speech from brain readings.

It was a great conversation, one covering issues that are becoming more timely by the day.

Major topics included:

He brought up free will early on, so we spent some time kicking around different definitions of free will and how they interact with legal questions on culpability.
How should we define and think about neurotech, i.e. how is a brain-to-brain interface different from just being really good at verbal communication?
Certain important legal concepts make a strong distinction between mental and physical acts, i.e. between thinking about robbing a person and actually doing it. But as more of the world is mediated by neural interfaces, this distinction will require further clarification.
We'll need to carefully distinguish between using neurotechnologies to help a person control their impulses and using them to effectively neuter people such that they're not even capable of becoming incensed over the gravest outrages. What sort of legal frameworks would help in this?

I hope you like the episode, and if you do, consider sharing it around to help us grow!
What "The Message" Was For Me
by Alex Beyman — 2022-10-11T08:08:11.673Z

Warning: If you have low tolerance for analogies or poetic language, skip to the numbered chain of dependencies for a more straightforward rundown of the reasoning presented herein.

It's been said that when you get the message, hang up the phone. What is "the message" though? I can't dictate what it is for everybody. This was just my biggest "aha moment" in recent years.
The most commonly expressed version of this idea I have seen, by analogy, is that living beings are like the countless little eyes which make up a compound eye. Each seeing a narrow, limited perspective of the world. The "big picture" which only the overall eye can see is all of those combined.

You could also express it as similar to the way in which a picture on a monitor is made up of many little differently colored pixels. There doesn't appear to be any rhyme or reason to it until you zoom out far enough.

In the same way, a human being viewed up close is actually trillions(!) of tiny individual organisms, none of which know they are part of a much larger creature. All expressions of the same basic idea that we're all part of the same thing, connected in ways that are not immediately obvious until viewed on a grand scale, and that separation is to some extent an illusion.

I experienced the same apparent self-evident nature of this idea but my brain is wired such that it is unsatisfied with that sort of just-so answer. I want complete explanations, diagrams with every little part clearly labeled. I want to know how it works. So I kept chugging away at it until this occurred to me:

In biological evolution, simple chemical self-replicators gave rise to intelligent organisms such as ourselves. We know this to be true with quite a lot of certainty.
Humans will eventually develop self-replicating machines. This is a plausible assumption given present technological trends toward automation, and probably occurs on any planet where intelligent life evolves. (Providing they don't destroy themselves first)
Since self-replicating machines will reproduce the conditions that started biological evolution (simple replicators), we have reason to believe the same thing will happen - resulting in the evolution of machine intelligence (Even if we do not make them intelligent to begin with).
These machine intelligences will assume dominion over their environment (which at their technological level will likely be anythi

...
Updates and Clarifications
by SD Marlow — 2022-10-11T05:34:57.851Z

Fighting a decade of Machine Learning momentum.

It seems the well received posts follow many of the same arguments, not about viability or milestones, but compute hardware trajectories or the pace of arXiv submissions reaching a point where even those in the field can't keep up anymore. I'm not suggesting people haven't made well reasoned arguments, but will say they don't break new ground. If everyone is just reading thru the posts, nodding their head, and saying "yep," then where is the challenge to existing views?

ML is moving in one direction, and it has built its own cult following. Miles Brundage posted a thread on Twitter earlier today about the importance of having different views (specific example was within OpenAI, but assume he meant it to apply broadly to the AI community and other research groups). Nothing in that thread gave any indication of having a different view about the core methods involved. Along the lines of "scale is all you need," there is a sense within ML that SOTA has proven (to everyone moving in the same direction) xNN's are the only valid solution.

The clarification required is that ML is not AI, and AI is not AGI. Results are not the same as abilities. Brute force on a narrow task (where a DL method happens to work) is not on a path toward a range of methods, in architecture, that can match or exceed human level cognition.

Risk assessment is bifurcated into where current systems fail (and then just guessing at how much worse it can be when they have 20x more compute), and into a philosophical realm where an AGI system has abilities that are 20x more than any one human (which is also based on incorrect assumptions about architecture and scale).

I don't see anyone here drifting away from the hardware aspect, and what the latest headline grabbing trend might suggest. I do ask that an effort be made to isolate future thinking from all current examples. There will be a few start-ups, perhaps in less than 10 years, that set the new standard on what can be achieved. Work from that assumption.
What if human reasoning is anti-inductive?
by Q Home — 2022-10-11T05:15:49.373Z

Can you help me with generalization of probabilistic reasoning?

I need a generalization of probabilistic inference for "anti-inductive" reasoning.

This post is going to discuss anti-inductive belief updating, perception, learning and reasoning (about hypotheses). Plus evidence of "anti-induction" in psychology. I think anti-induction may help to explain how humans reason.

The overall point/conclusion of the post is described here.

Weird worlds

To see the difference between inductive and anti-inductive probabilistic reasoning, let's visit "weird worlds".

In weird worlds important properties are "shared" between objects. I want to give an example. Let's say you're a tall person. In the normal world, your height doesn't tell much about my height. Our heights are not related.

However, in the weird world you being tall may imply that:

I'm short. Because you already "occupied" the property of being tall.
I'm tall too. Because "height" is shared between people. Your height is my height.

I want to give more examples to show the difference between reasoning methods in both worlds.

1.

Imagine two worlds. In the weird world, the property of intelligence is "shared" among beings. In the normal world, it is not "shared".

You have a question "Are there many beings smarter than humans?".

Then you encounter some beings much smarter than humans.

In the normal world, you update towards answering "yes" using the Bayes' rule.

However, in the weird world something else may happen:

You update towards "no" because if those beings are so smart, then humans are smarter than it seems. (intelligence is shared among beings)
You update towards "no" because if those beings are so smart, it means there's not a lot of intelligence left for other beings. (intelligence is limited)
You hypothesize two categories of beings, A (normal) and B (super-smart). For category A you don't make any updates. For category B you make an update towards "yes".

In the weird world, if you encounter "too much" positive evidence, the evidence may flip into negative evidence.

2.

Imagine two worlds. In the weird world, the wealth is "shared" among beautiful people. In the normal world, it is not "shared".

You have a question "Is beauty related to big wealth?". You don't have any information about anyone yet.

Then you encounter someone very beautiful and very wealthy.

In the normal world, you weakly update towards answering "yes" usi...
Fullness to Indicate Cleanliness
by jefftk — 2022-10-11T00:40:02.150Z

Our house has a simple system for the dishwasher: when you're emptying
a rack, empty it all the way and push it in. This means that if you come across a
dishwasher with a partially-full rack you know it's dirty and can put
things in without checking. It's a lot more reliable than clean/dirty
toggle magnets, because it can't be "out of sync".

Clean:

Dirty:

This system works especially well for us because of how many people we
have in the house. It's not like someone living alone who will
remember whether they already unloaded the dishwasher, and we're
almost never running partial loads.
What happened to the annual LW demographic surveys? 
by ROM — 2022-10-11T00:19:10.742Z

Scott Alexander once ran an annual survey of the LessWrong community. After 2014 he seems to have stopped and the surveys happened intermittently. User namespace picked up the torch in 2016 and 2017. Bob Jacobs also did one in 2020.

Since then it doesn't look like any have been done.

Given that the early surveys were popular, I'm surprised they no longer happen. Is there a simple reason for this I might have missed?
EA & LW Forums Weekly Summary (26 Sep - 9 Oct 22')
by Zoe Williams — 2022-10-10T23:58:22.991Z

Supported by Rethink Priorities

Author's note: this post covers the past two weeks, with a karma bar of 80+ instead of the usual 40+. Covid knocked me out a bit so playing catch up! Future posts will be back to the usual schedule and karma requirement.

This is part of a weekly series - you can see the full collection here. The first post includes some details on purpose and methodology.

If you'd like to receive these summaries via email, you can subscribe here.

Podcast version: prefer your summaries in podcast form? A big thanks to Coleman Snell for producing these! Subscribe on your favorite podcast app by searching for 'Effective Altruism Forum Podcast'.

Top Readings / Curated

Designed for those without the time to read all the summaries. Everything here is also within the relevant sections later on so feel free to skip if you’re planning to read it all.

Does Economic Growth Meaningfully Improve Well-being? An Optimistic Re-Analysis of Easterlin’s Research: Founders Pledge

by Vadim Albinsky

The Easterlin Paradox states that happiness varies with income across countries and between individuals, but not significantly with a country’s income as it changes over time. A 2022 paper attempts to verify this with recent data, and finds it holds.
The post author challenges the research behind that paper on two counts:
1. The effect on happiness from increasing a country’s GDP over time is disregarded as small, but is big enough to be meaningful (similar size to effects seen from GiveDirectly’s cash transfers). If we run the same analysis for health, pollution, or welfare measures, GDP has the largest effect.

2. The paradox largely disappears with only minor changes in methodology (eg. marking one more country as a ‘transition’ country or using 2020 data instead of 2019). Both bring the effect size closer to that of variance between countries / individuals than to zero.

Winners of the EA Criticism and Red Teaming Contest

by Lizka, Owen Cotton-Barratt

31 winners were announced out of 341 entries. The top prizes went to ‘A critical review of GiveWell's 2022 cost-effectiveness model’ and ‘Methods for improving uncertainty analysis in EA cost-effectiveness models’ by Alex Bates (Froolow), ‘Biological Anchors external review’ by Jennifer Lin and ‘Population Ethics without Axiology: A Framework’ by Lukas Gloor.

The post summarizes each of the winning entries, as well as ...
QAPR 4: Inductive biases
by Quintin Pope — 2022-10-10T22:08:52.069Z

Introduction

This is week 4 of Quintin's Alignment Papers Roundup. The current focus is the inductive biases of stochastic gradient descent.

For most datasets and labels, there are many possible models that reach good performance. "Inductive biases" refers to the various factors that incline a particular training process to find some types of models over others. When the data under-specify the learned model, a training process's inductive biases determine what sort of decision making process the model implements, and how the model generalizes beyond its training data.

I'd intended to publish this last week, but it turns out that there's a lot of work on SGD's inductive biases, and it's very technical. I kept finding new papers that seemed relevant. That's why this roundup has 16 papers, in place of the usual ~9 or so.

Papers

Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks

Understanding the fundamental principles behind the massive success of neural networks is one of the most important open questions in deep learning. However, due to the highly complex nature of the problem, progress has been relatively slow. In this note, through the lens of infinite-width networks, a.k.a. neural kernels, we present one such principle resulting from hierarchical localities. It is well-known that the eigenstructure of infinite-width multilayer perceptrons (MLPs) depends solely on the concept frequency, which measures the order of interactions. We show that the topologies from deep convolutional networks (CNNs) restructure the associated eigenspaces into finer subspaces. In addition to frequency, the new structure also depends on the concept space, which measures the spatial distance among nonlinear interaction terms. The resulting fine-grained eigenstructure dramatically improves the network's learnability, empowering them to simultaneously model a much richer class of interactions, including Long-Range-Low-Frequency interactions, Short-Range-High-Frequency interactions, and various interpolations and extrapolations in-between. Additionally, model scaling can improve the resolutions of interpolations and extrapolations and, therefore, the network's learnability. Finally, we prove a sharp characterization of the generalization error for infinite-width CNNs of any depth in the high-dimensional setting. Two corollaries follow: (1) infinite-width deep C

...
Apollo
by Jarred Filmer — 2022-10-10T21:30:44.496Z

Life is a response to being

Part II

Over the course of my twenties, I'd accumulated thousands of hastily jotted notes. Thoughts that had grown in the gaps between work and play I'd managed to save before lost off the cliff of forgetting.

Thoughts like, "I'm starting to suspect it's possible to be robustly happy on purpose..." or "Inadequate tools for consensus keeps coming up as a kind of problem behind all problems I care about".

Upon arrival, I didn't fully understand why I'd felt so strongly I had to go to a cabin in the woods to be alone for a month. But I knew it had something to do with these notes. They were all the errant Frisbees my mind had wanted to chase when it should have been working, or shopping, or trying to sleep. Thought that couldn't be fully explored for lack of space.

Fruits of a wayward youth

I spent my entire first week in the cabin working through each text file A to Z. From amsterdam-trip-reflections.txt all the way through to xmas-gifts-2021.txt.

Many felt irrelevant to who I was now.

Others felt revelatory.

I began a fresh note, (threads.txt) to tease out the myriad themes, inquiries, and projects tangled across these vast tracts of text. From ~500 text files I pulled ~200 "threads", and wrote the title of each down in pen on an index card.

Everything I'd ever been passionate about in the last 7 years. Anytime a book, or a podcast, or someone's tipsy rant landed in my brain and flowered into violent spirals of thought. It was all here, finally down in one place.

Personal and collective flourishing, how we could better make sense of our shared condition, why is community so scarce if we all crave it, escaping the meaning crisis, x-risk, governance, prediction markets, crypto, effective altruism, dating, what's up with the free energy principle, memetics, sense-making, dungeons and dragons, gender, origins of life, suffering, living in new york, progress studies, dance, game b, virtue, vocation, factory farming, intentional community, the qualia research institute, charter cities apollo longevity vTaiwan attachment the meta crisis powerlifting axiology circling functional programming awakening biotech aesthetics metamorphosis *mind explodes*

For days I couldn't bear to look at it. I played video games, went for walks, and sat in the woods filling my copy of Walden with underlines (feeling enormously grateful to the EAs I met passing through Ber...
Does biology reliably find the global maximum, or at least get close?
by Noosphere89 — 2022-10-10T20:55:35.175Z

Jacob Cannell has claimed that biological systems at least get within 1 OOM of not a local, but global maximum in abilities.

His comment about biology nearing various limits are reproduced here:

The paper you linked seems quite old and out of date. The modern view is that the inverted retina, if anything, is a superior design vs the everted retina, but the tradeoffs are complex.

This is all unfortunately caught up in some silly historical "evolution vs creationism" debate, where the inverted retina was key evidence for imperfect design and thus inefficiency of evolution. But we now know that evolution reliably finds pareto optimal designs:

biological cells operate close to the critical Landauer Limit, and thus are pareto-optimal practical nanobots.

eyes operate at optical and quantum limits, down to single photon detection.

the brain operates near various physical limits, and is probably also near pareto-optimal in its design space.

Link to comment here:

https://www.lesswrong.com/posts/GCRm9ysNYuGF9mhKd/?commentId=aGq36saoWgwposRHy

I am confused about the Landauer limit for biological cells other than nerve cells, as it only applies to computation, but I want to ask, is this notion actually true?

And if this view was true, what would the implications be for technology?