trentmkelly/LessWrong-43k huggingface dataset:
Is there something fundamentally wrong with the Universe?
by Caerulea-Lawrence — 2023-09-12T12:02:44.445Z

Edit: added a paragraph nearing the end
Usually the question is: "What is wrong with our society?", but I guess my gripe is that if you look at things with a System-lense, usually what you focus on is what has the most influence.
In other words, if our Planet would veer off course, it isn't really the fault of the Ants - even when they could have done more! The reason is bigger gravitational forces, or our Sun wanting some more space.
So, why would you blame a human being for something that seems to be governed by laws, forces and powers we A) don't fundamentally understand, B) can't fundamentally influence C) can't fundamentally change ?
I mean, from a human perspective, there are a lot of answers - Psychology, sociology, anthropology, evolutionary disciplines, religion - even transdisciplinary groups and projects. But again, you only answer the question from inside the box. If the box gives a certain output, and has specific constraints, why blame the Outputs for how the box works?

It makes more sense to me to assign fault at the level where the problem lies. If we dislike and abhor something, at which level does it arise?
A lot of expressions we use are filled with assumptions that are focused on our personal actions. Take murder, for example. Killing is bad, I agree, but is it the personal choice of the killer to enable murder, to make possible the possibility of dying? The pain, suffering, dread etc.? It is the Universe's Laws and principles that are allowing it and actively engaging in it (There are endless ways this Universe can and will kill us), is it not?
That is just one of numerous examples I can think of where I am questioning if by looking closely at an issue, it seems to be the direct consequence of a superintended principle we have 0 direct control over.
I mean, personal responsibility sounds nice, but if you really look at it - does it make any sense? Take genes, for example. They are based on millions of years of biological, and then even cultural evolution. That you have the 'ability' to see a different way than eating your fellow friend, and can see more gains in keeping them alive - is it really "Your" achievement? How many choices are really "Yours", and not the inexplicable results of processes that started millions of years before you were even consciously aware that you had a face and a body separate from the rest of the wo...
Stupidity is also hard
by walkthroughwalls — 2023-09-12T02:45:49.294Z

Everyone today talks about Intelligence, but no one talks about the inverse problem: Stupidity. How stupid can you make a system? And what are the (mis)alignment properties of such a system?

If intelligence is the ability to efficiently achieve goals in a wide range of domains, then stupidity is the lack of such ability. Give a stupid being a goal and it will with great probability fail to achieve it.

Let's look at specific examples. What about a random agent? How stupid is it? Not as stupid as you would think. Give it a multiple choice test and it will average 50%. Let it try out the stock market and watch it beat troves of traders. In a chess game, with some small percentage it will play a perfect game.

What about a superintelligence who likes to fail? Given any task, it will try to fail it. It will always get 0% on multiple choice tests and lose all its money in the stock market. It will lose each chess game in record time. Does true stupidity require intelligence?

What about the average human? It will get above 50% on many multiple choice tests. It will lose money in the market. And it will lose most chess games.

What about a rock? It does literally nothing. Give it a multiple choice test and it will average 0% and it will not lose a cent in the stock market. It will lose every chess game on time. The strategy a rock implements is nontrivial. For instance, most humans would rather make things worse than do nothing.

Who is stupider? Certainly, the one who performs worst is the failure-seeking superintelligence. But most people would not consider it "stupid", because it has the "potential" to achieve all the goals it failed given a relatively minor modification to its utility function.

So what if I take GPT-5 and replace the last layer with a zero matrix? It now outputs a zero vector every time, so it is the zero function, but it is within a small edit-distance from GPT-5 proper. Is this over-engineered zero function intelligent or stupid?

What if I take a very large sequence of matrix computations, such that probabilistically it contains a section isomorphic to GPT-5, is this intelligent or stupid? Here, the key is that although it contains sufficient complexity that seems "intelligence", intelligence is also required to prune the non-intelligent parts out.

The hallmark of intelligence is the capacity to direct the world toward certain states. (Is a really large b...
Apple Cider Baklava
by jefftk — 2023-09-12T02:10:04.251Z

Since coming across

apple cider
syrup last year I've been trying to figure out what sort of things
to make with it. Cider donuts are traditional, but I'm not that much
of a fan. Mixing it into whipped cream works very well, and makes a
good sweet and tart filling or topping for baked goods. I've
experimented with muffins, using it for a combination of the liquid,
the sugar, the flavoring, and the acid for the leavening, but I
haven't ended up with anything I'm happy with yet. But a few
days ago I made some apple baklava that's my favorite yet!

Baklava is layers
of phyllo dough (a
very thin dough) separated by a fat, with nuts every few layers.
After baking you pour a sweet syrup over the top. In the US people
are probably most familiar with a Greek style which uses butter for
the fat, walnuts for the nuts, and honey for the syrup, but there are
an enormous number of regional variations.

I made two pans, one vegan (earth balance) and one not (butter). The
butter one tasted a lot better; possibly there are better choices than
earth balance? I used almonds for the nuts because that's what I had.
For the syrup I used New England boiled cider. Very happy with how it
turned out, and I plan to make it again!

(Each loaf pan used ~$1.80 of cider
syrup, though if you really like it you could buy
in bulk and get it down to $1.42.)
How useful is Corrigibility?
by martinkunev — 2023-09-12T00:05:41.995Z

Epistemic Status: My best guess (I'm new to AI safety)

We don't know how to formally define corrigibility and this is part of the reason why we haven't solved it so far. Corrigibility is defined in Arbital as:

[the agent] doesn't interfere with what we would intuitively see as attempts to 'correct' the agent, or 'correct' our mistakes in building it

The precise meaning of words like "interfere", "attempt", or "correct" is unclear from this informal definition. Even with the most generous definition of those concepts, a corrigible agent can still be dangerous. A first intuition is that if we have a corrigible agent, we can always stop it if something goes wrong. I suppose most people here understand the flaws in this reasoning but I have never seen them stated them explicitly so here is my attempt:

By the time we figure out an agent needs to be corrected, it may have done too much damage.
Correcting the agent may require significant amount of research.
Being able to correct the agent is insufficient if we cannot coordinate on doing it.

In my opinion, solving the hard corrigibility problem is a more meaningful intermediate goal on the path to the terminal goal of solving alignment.

Disaster Scenario

What follows is a scenario that illustrates how a corrigible agent may cause serious harm. Use your imagination to fill in the blanks and make the example more realistic.

Suppose we develop a corrigible agent which is not aligned with our values. It doesn't matter exactly what the agent is designed to do but for concreteness you can imagine the agent is designed to optimize resource usage - it is good at task scheduling, load balancing, rate limiting, planning, etc. People can use the agent in their server infrastructure, for logistics, in factory production or many other places. No matter its terminal goal, the agent is likely to value resource acquisition instrumentally - it may use persuasion or deception in order to get access and control of more resources. At some point, people would leverage it for something like managing the electric grid or internet traffic. This process of integrating the agent into various systems continues for some years until at some point the agent starts paying paperclip factories to increase their production.

At this point, in the best case we realize something's wrong and we consider turning the agent off. However, the agent controls the energ...
Contra Heighn Contra Me Contra Functional Decision Theory

by Bentham's Bulldog — 2023-09-11T19:49:43.216Z

In my article about why

Eliezer was often confidently and egregiously wrong

, I gave his decision theory as a solid example of this. Eliezer thinks that the correct theory of rationality—of what it is wise to do in a particular situation—is FDT which says roughly that you should act in ways such that agents acting in those ways get the most utility. Heighn and I are having a debate about this that will go back and forth for a bit—so here’s my response to

Heighn’s first post

.

I think basically all of Heighn’s replies rest on the same crucial error, so I won’t go through line by line. Instead, I’ll briefly explain the core argument against FDT, which has various manifestations, and then why the basic reply doesn’t work. I’ll then clean up a few other extraneous points.

The most important argument against FDT is that, while it’s a fine account of what type of agent you want to be, at least, in many circumstances, it’s a completely terrible account of rationality—of what it’s actually wise to do when you’re in one situation. Suppose that there’s an agent who has a very high probability of creating people who once they exist will cut off their legs in ways that don’t benefit them. In this case, cutting off one’s legs is clearly irrational—one doesn’t benefit at all and yet is harmed greatly. But despite that, FDT instructs one to cut off their legs because those agents get more utility, on average.

Heighn’s response to this argument is that this is a perfectly fine prescription. After all, agents who follow their advice get more utility on average than agents who follow EDT or CDT.

But the CDTists have a perfectly adequate description of this. Sometimes, it pays to be irrational. If there is a demon who will pay only those who are irrational, then it obviously pays to be irrational. But this doesn’t make it rational to be irrational. Cutting off your legs in ways that you know will never benefit you or anyone else is flagrantly irrational—it is bad for everyone—and this is so even if such agents win more.

Heighn would presumably agree with this. After all, if there’s a demon who pays a billion dollars to everyone who follows CDT or EDT then FDTists will lose out. The fact you can imagine a scenario where people following one decision theory are worse off is totally irrelevant—the question is whether a decision theory provides a correct account of rationality. But if a ...
Machine Evolution
by Justin Bullock — 2023-09-11T19:29:53.989Z

Epistemic Status: This post explores the relationship between biological evolution and the evolution of our machines. We examine parallels between the evolutionary pressures shaping organisms and machines. We are still exploring this analogy and our findings are tentative. We hope to explore these relationships further in future posts. Feedback and criticism are very much appreciated.

Introduction

After all then it comes to this, that the difference between the life of a man and that of a machine is one rather of degree than of kind, though differences in kind are not wanting. An animal has more provision for emergency than a machine. The machine is less versatile; its range of action is narrow; its strength and accuracy in its own sphere are superhuman, but it shows badly in a dilemma; sometime when its normal action is disturbed, it will lose its head, and go from bad to worse like a lunatic in a rain frenzy: but here, again, we are met by the same consideration as before, namely, that the machines are still in their infancy; they are mere skeletons without muscles and flesh.

— “Erewhon” by Samuel Butler, 1872

Machines change over time. They evolve. Our machines of today are advancements and refinements of the machines of yesterday, with each new generation exposed to selective pressures. Some changes promote the proliferation of some machines over other types and are more likely to persist and undergo further refinement and transformation. Innovations in machine capabilities exploit new niches and proliferate, diversify, and rapidly spread throughout their ecosystem. At the same time, just as with biological evolution, previous generations of machines go extinct, while others persist comfortably in their niche, undergoing only minor changes for hundreds or thousands of years. For example, Samuel Butler pointed to

the tobacco pipe

, and other pre-machine tools such as hand axes and needles lasted for millenia.

The hand axe is the

longest-used tool in human history

, employed with little modification for over a million years, and sewing needles that resemble those we use today, made from bone, have been dated to over 60 millenia ago.

A 450 million-year-old fossil of a horseshoe crab beside a modern specimen. This basic design has persisted for nearly half a billion years.

Similar processes of change over time occur in biological evolution, in particu...
Amazon KDP AI content guidelines
by ChristianKl — 2023-09-11T18:36:08.464Z

Amazon updated the content guidelines for Kindle Direct Publishing to address AI and in the process attempted to define terms AI-generated and AI-assisted. Guidelines like this are how our society tries to engage with AI in the near-term.

Artificial intelligence (AI) content (text, images, or translations)
We require you to inform us of AI-generated content (text, images, or translations) when you publish a new book or make edits to and republish an existing book through KDP. AI-generated images include cover and interior images and artwork. You are not required to disclose AI-assisted content. We distinguish between AI-generated and AI-assisted content as follows:

AI-generated: We define AI-generated content as text, images, or translations created by an AI-based tool. If you used an AI-based tool to create the actual content (whether text, images, or translations), it is considered "AI-generated," even if you applied substantial edits afterwards.
AI-assisted: If you created the content yourself, and used AI-based tools to edit, refine, error-check, or otherwise improve that content (whether text or images), then it is considered "AI-assisted" and not “AI-generated.” Similarly, if you used an AI-based tool to brainstorm and generate ideas, but ultimately created the text or images yourself, this is also considered "AI-assisted" and not “AI-generated.” It is not necessary to inform us of the use of such tools or processes.

You are responsible for verifying that all AI-generated and/or AI-assisted content adheres to all content guidelines. For example, to confirm an AI-based tool did not create content based on copyrighted works, you're required to review and edit any AI tool outputs.

What do you think about how Amazon defined the terms and makes those distinctions? Do you think they should change the definitions? Do you think there are other distinctions they should make?
A Case for AI Safety via Law
by JWJohnston — 2023-09-11T18:26:46.534Z

This post is to make the subject case more available and open to comment. A paper with the above title is currently languishing in arXiv limbo but available in Google Docs. I was surprised to see my last and only postings to LessWrong were made on this very subject in 2010 as comments to the thread "Why not just write failsafe rules into the superintelligent machine?"

Unfortunately (IMO) this approach to AI alignment doesn't seem to have gained much traction in the past 13 years. The paper does cite, however, 15 precedents that show there is some support. (Anthropic's Constitutional AI and John Nay's Law Informs Code are two good recent examples.)

The following reproduces the Summary of Argument and Conclusion sections from the paper.

The claim being argued is "Effective legal systems are the best way to address AI safety."

4 Summary of Argument

4.1 Law is the standard, time-tested, best practice for maintaining
order in societies of intelligent agents.

Law has been the primary way of maintaining functional, cohesive societies for thousands of years. It is how humans establish, communicate, and understand what actions are required, permissible, and prohibited in social spheres. Substantial experience exists in drafting, enacting, enforcing, litigating, and maintaining rules in contexts that include public law, private contracts, and the many others noted in this brief. Law will naturally apply to new species of intelligent systems and facilitate safety and value alignment for all.

4.2 Law is scrutable to humans and other intelligent agents.

Unlike AI safety proposals where rules are learned via examples and encoded in artificial (or biological) neural networks, laws are intended to be understood by humans and machines. Although laws can be quite complex, such codified rules are significantly more scrutable than rules learned through induction. The transparent (white box) nature of law provides a critical advantage over opaque (black box) neural network alternatives.

4.3 Law reflects consensus values.

Democratically developed law is intimately linked and essentially equivalent to consensus ethics. Both are human inventions intended to facilitate the wellbeing of individuals and the collective. They represent shared values culturally determined through rational consideration and negotiation. They reflect the wisdom of crowds accumulated over time—not preferences th...
Erdős Problems in Algorithmic Probability
by Aidan Rocke — 2023-09-11T16:44:06.692Z

Motivation:

Before the Pandemic of 2020, I had a number of constructive discussions with several neuroscientists including Konrad Körding, a computational neuroscientist and pioneer exploring connections between deep learning and human learning, and Alex Gomez-Marin, an expert on behavioural neuroscience, on the hypothetical boundary between Human and Machine Learning. One promising avenue for making progress along these lines appeared to be Kolmogorov’s theory of Algorithmic Probability [4] which Alexander Kolpakov(Sasha) and myself recently used to probe this boundary, motivated in large part by open problems in Machine Learning for Cryptography [1].

However, more progress appears to be possible based on the analyses of Marcus Hutter and Shane Legg who established that Algorithmic Probability defines the epistemic limit of Machine Learning [3]. In the hopes of developing more powerful tools in Algorithmic Probability for probing the epistemic limits of Machine Learning, Sasha and myself propose cash prizes for deriving two mathematical laws using Algorithmic Probability.

Erdős Problems in Algorithmic Probability:

Benford's Law, which Terrence Tao himself admitted lacks a satisfactory explanation: https://terrytao.wordpress.com/2009/07/03/benfords-law-zipfs-law-and-the-pareto-distribution/
Atle Selberg's Identity, which proved crucial in Paul Erdős' and Atle Selberg's elementary proof of the Prime Number Theorem:

ψ(x)ln(x)+∑p≤xln(p)ψ(xp)=2xln(x)+O(x)

which is derived using elementary number theory in [5] where ψ(x)=∑p≤xlnp, though its true meaning remains a mathematical mystery [6,7].

Submission guidelines and Monetary Prizes:

The challenge is named in honour of Paul Erdős whose probabilistic method has provided Sasha and myself with a number of important insights, and the cash prize is 1000.00 USD for the best submission under 30 pages in each category. The main evaluation criteria are clarity, correctness and concision.

Submissions typeset in LaTeX must be sent to the tournament-specific email of Alexander Kolpakov, Managing Editor of the Journal of Experimental Mathematics, before September 17, 2024: wignerweyl@proton.me

Prizes shall be awarded on January 1st, 2025.

A model for submissions:

As a standard model for submissions, we recommend Yuri Manin's derivation of Zipf's Law [9] as a corollary of Levin's Univers...
PSA: The community is in Berkeley/Oakland, not "the Bay Area"
by maia — 2023-09-11T15:59:47.132Z

Posting this because I recently had a conversation that went like this:

Friend: Hey, you used to live in SF. Is there any rationalist stuff actually happening in San Francisco? There don't seem to be many events, or even that many aspiring rationalists living here. What's up with that? [Paraphrased. I've had similar versions of this conversation more than once.]

Me: Something we realized living there is that SF actually suffers the same brain drain as most other cities, because everyone just goes to Berkeley/Oakland.

The same way people move from the East Coast or elsewhere to Berkeley, they move from the rest of the Bay Area to Berkeley. Actually, they do it even more, because moving to Berkeley is easier when you already live pretty close by.

And you don't figure this out until you move there, because people who live outside the Bay Area think of it as being all the same place. But the 45 minute train ride really matters when it comes to events and socializing, as it turns out.

Friend: That sounds so inconvenient for people who have jobs in the city or South Bay!

Me: Sure is! I don't have a super-solid answer for this, except that 1) Lots of people actually just do awful, awful commutes, because having a real, in-person community is that valuable to them, as bad as commuting is. 2) A surprising fraction of the community works at rationalist/rationalist-adjacent nonprofits, most of which are actually located in the East Bay. Plus, 3) in a post-COVID world, more people can work remote or partly remote. So you can choose to live where your community is... which is Berkeley... even though it is crazy expensive.

I don't actually live in the Bay Area anymore, so I don't have the most up-to-date information on where events are happening and things. But it seems from what I hear from folks still there that it's still broadly true that East Bay is where things are happening, and other parts of the area have much less of the community.

If you're thinking about moving to the Bay in part for the rationality community, take this into account!
A Bat and Ball made me Sad
by Darren McKee — 2023-09-11T13:48:24.686Z

'A bat and a ball cost $110 in total.

The bat costs $100 more than the ball.

How much does the ball cost?

The answer is $5.

Please enter the number 5 in the blank below.

$_____'
That question is from one of 59 new studies (N = 72,310) which focus primarily on the “bat and ball problem," found in The formation and revision of intuitions - ScienceDirect by Andrew Meyer and Shane Frederic.
23% of respondents failed to give the correct answer.
I don't have anything insightful to say but wanted to share because it seems worth knowing.
Additionally, I'm not exactly sure how (and how much) I should update about this.
(Edit: I initially posted the wrong question)