trentmkelly/LessWrong-43k huggingface dataset:
On Frequentism and Bayesian Dogma
by DanielFilan — 2023-10-15T22:23:10.747Z

I've heard that you believe that frequentism is correct. But that's obviously wrong, so what gives?

DanielFilan

I guess first of all I should ask, what do you mean by "frequentism"?

DanielFilan

I mean classical statistical frequentism. Though somewhat tongue-in-cheek, as I don't think it's fully correct, I think it's much more correct than orthodox Jaynesian Bayesianism.
Some scattered thoughts:

Bayes' theorem derives from conditional probability so it's also included in frequentism.
Bayesian epistemology only applies to situations when your beliefs are a probability distribution, and is thus incomplete.

It doesn't account for e.g. limited computation.

Frequentism solves these things by framing the problem in a different way. Rather than 'how should I think?' it's "this algorithm seems like a sensible way to think, let's figure out what epistemic guarantees it has".

In particular, it makes it OK to believe things that are not expressible as probability distributions.

Adrià Garriga-alonso

I'm still sort of unsure what you mean by "classical statistical frequentism". Like, I'm pretty sure I agree that the purported theorems of Fisher are in fact theorems. Do you mean something like "the way we should think about thinking is to ask 'what cognitive algorithms perform well with high probability in the long run'"?

DanielFilan

(and regarding Bayesianism, I think that's a separate question that's more productively talked about once I understand why you think frequentism is good)

DanielFilan

Sure. Thank you for the clarification -- I agree there are many fine theorems on both sides.
Statistics is the problem of learning from data, frequentism is saying "Here's an algorithm that takes the data and computes something (that's an estimator). Let's study the properties of it."

"the way we should think about thinking is to ask 'what cognitive algorithms perform well with high probability in the long run'"?

Yeah, I agree with this. Though I recognize in practice it's pretty hard, frequentist theory attempts to do so (for performance usually being 'have correct beliefs').

Adrià Garriga-alonso

Here's an algorithm that takes the data and computes something (that's an estimator). Let's study the properties of it.

I first want to note that this is an exhortation and not a proposition. Regarding the implicit exhortation of "it's good to understand estimators", I guess I agree? I think my crux is somet

...
More or Fewer Fights over Principles and Values?
by Ben Pace — 2023-10-15T21:35:31.834Z

I wrote:

I wish that people would generally get into more conflicts over the values and principles that they hold

Vaniver wrote:

I think we are unusually fractious and don't value peace treaties and fences as much as we should

I wrote:

Fight me :)

Start of Dialogue

People seem to me to be very conformist and most of the values that people want to build around get overwhelmed by the aggressively conventional-minded people (of Paul Graham's essay). My sense is that people who fight for their values in honorable ways just make the world so much better and we should all aspire to be more like that.

Ben Pace

How much of this is a difference between "people" and "we"? I was imagining, like, LessWrongers, instead of people that I went to college with. Who specifically are you talking about?

Vaniver

I mean, many LWers are people with ordinary-enough lives who have enough sanity to see that the world could be better, but still most of them just fade away with time. So I'm including LWers in this too.

Ben Pace

I think now maybe I'm also confused about what specifically you mean by values and principles and so on. I like for things to be high-quality and notice flaws; also I have developed an attitude of tolerance about this to be a bit less bitchy. It's not yet clear to me if you're trying to argue that I should be more critical of things in general.
Similarly, one of the things that I'm more in touch with now than before is animal suffering things; I could fight with housemates who buy chicken because it's cheaper about how I think they're doing something hideous but this goes against my sense of good neighborliness.
(I think the thing where I most feel like you're right that I could be more fractious is, like, talking with people about how much time I think we have left and how much I think various political changes are necessary; I think here I have learned helplessness from being a rare libertarian instead of actually endorsing how fighty I am.)

Vaniver

I have historically found myself with few allies in the comment section of the EA Forum for most of the history of EA, when I tried to stand up for letting people speak their mind and not be subject to crippling PR constraints.

Also I have found myself supportive of people who I thought were in the EA ecosystem for the values and for doing what's right, and then when I pointed out times that the ecosystem as a whole was not doing th

...
Mapping ChatGPT’s ontological landscape, gradients and choices [interpretability] 
by Bill Benzon — 2023-10-15T20:12:36.361Z

Cross-posted from New Savanna.

By ontological landscape I mean the repertoire of different things it knows about. The parlor game, Twenty questions, was invented in the United States in the 19th century as a way of playing around on this landscape. One player thinks of some phenomenon and the other has to guess it by asking a series of no more than 20 questions that have a “yes” or “no” answer. One popular variant of the game begins with a simple question: “Animal, vegetable, or mineral?” Once the question has been answered the questioner must continue with binary questions.

In this post I take a look at how ChatGPT deals with this structure. I start simply, with concrete and abstract object, and move on from there.

* * * * *

All of the interactions reported below took place on the morning of October 15 using the September 25 version of ChatGPT.

Physical things

I decided to start with the simplest prompt I could imagine, “thing.” That’s the most general category we’ve got, no? Anything whatever is going to be some kind of thing. But how would ChatGPT respond? It responded as I thought it would, so I gave it a more explicit prompt, to which it responded.

Thing

It seems like you mentioned "thing," but you didn't provide a specific question or context. Could you please clarify or provide more information so I can assist you better?

Give me a list of 20 things, anything.

Certainly! Here's a list of 20 random things:

1. A beach ball
2. A rainbow
3. A bicycle
4. A bookshelf
5. A coffee mug
6. A mountain
7. A laptop computer
8. A painting
9. A pineapple
10. A pair of sunglasses
11. A telescope
12. A running shoe
13. A wristwatch
14. A musical instrument
15. A flower vase
16. A hammock
17. A traffic light
18. A hot air balloon
19. A chef's knife
20. A Rubik's Cube

These are just a few examples of things, and the list could go on endlessly!

I had no particular expectations about how ChatGPT would respond. But I couldn’t help but notice that each item in its response was some kind of physical thing. To be sure, rainbows (the second item in the list) are rather diaphonous, and you couldn’t actually touch one (I should have asked ChatGPT about that; later...), but it is still a physical thing.

I didn’t ask for physical things, but that’s what ChatGPT gave me. That’s worth noting.

Let’s think a bit before continuing on. I wonder if that first list is “rigid” in the sense that ChatGPT would respond the sa...
The Hidden Perils of Hydrogen
by Yudhister Kumar — 2023-10-15T19:51:07.418Z
Link: https://ykumar.org/the-hidden-perils-of-hydrogen/

Hydrogen is the fuel of the future. It’s the most abundant element in the universe, is incredibly mass-efficient, and can be produced without much fuss. It can be used for both large and small scale energy production (think fusion and fuel cells respectively), and has virtually no emissions, carbon or otherwise. However, there’s just one problem.

It’s absurdly volume-inefficient.

One liter of hydrogen can produce 0.01 MJ of energy at STP (standard temperature and pressure, 273K and 1 atm) compared to 34 MJ/L for gasoline. That’s 0.02% of the energy output for liter. Granted, things improve drastically for liquid hydrogen, where the comparison is 8 MJ/L vs 34, but this requires maintenance of temperatures below −252.8°C, only a few degrees above absolute zero.

Gaseous hydrogen isn’t that easy to store either: it requires containers pressurizable up to 700 bar (700x atmospheric pressure). That’s 10,000 PSI. And even then, it takes 5L of hydrogen to match up to 1L of gasoline. The Department of Energy estimates that to meet most lightweight vehicular driving ranges, between 5-13 kg of hydrogen need to be carried onboard the vehicle. If we do some quick calculations, that means that vehicles need to carry between 85 - 294 L of liquid hydrogen (some multiple of this for gaseous hydrogen) to go anywhere. For all you Americans, this is roughly between 22-77 gallons.

(For reference, 5-13kg of hydrogen converts to 20-70 liters of gasoline, in terms of energy efficiency. I also make no claim as to the specific accuracy of the numbers, these are napkin calculations, and this doesn’t take into account energy efficiencies created by recycling hydrogen or fuel cells being more efficient than combustion engines, etc.)

It would not be feasible to have stereotypical sedans with fifty gallon gas tanks in a world of solely hydrogen fuel. Additionally, the extreme conditions liquid hydrogen must be stored as bottlenecks its production and transportation.

The Department of Energy set standards for hydrogen storage to be met by 2020 in order for hydrogen fuel to become feasible for portable power and light-duty vehicular applications. These were:

1.5 kWh/kg (overall system performance)

1.0 kWh/L (overall system performance)

$10/kWh (translates to $333/kg for stored hydrogen)

So far, none of these goals have been met, and I have doubts about the scalability of present research (althou...
Arguments for optimism on AI Alignment (I don't endorse this version, will reupload a new version soon.)
by Noosphere89 — 2023-10-15T14:51:24.594Z

Or, why we probably don't need to worry about AI.

So this post is partially a response to Amalthea's comment on how I simply claimed that my side is right, and I responded by stating that I was going for a short comment rather than having to make another very long comment on the issue.

https://www.lesswrong.com/posts/aW288uWABwTruBmgF/?commentId=r7s9JwqP5gt4sg4HZ#r7s9JwqP5gt4sg4HZ

This is the post where I won't try to claim that my side is right, and instead give evidence so I can properly collect my thoughts here. This will be a link-heavy post, and I'll reference a lot of concepts and conversations, so it will help if you have some light background on these ideas, but I will try to make everything intelligible to the lay/non-technical person.

This will be a long post, so get a drink and a snack.

The Sharp Left Turn probably won't happen, because AI training is very different from evolution

Nate Soares suggests that a critical problem in AI safety is the sharp left turn, and the sharp left turn essentially is that capabilities generalize much more than the goals, ie it is basically goal misgeneralization plus fast takeoff:

My guess for how AI progress goes is that at some point, some team gets an AI that starts generalizing sufficiently well, sufficiently far outside of its training distribution, that it can gain mastery of fields like physics, bioengineering, and psychology, to a high enough degree that it more-or-less singlehandedly threatens the entire world. Probably without needing explicit training for its most skilled feats, any more than humans needed many generations of killing off the least-successful rocket engineers to refine our brains towards rocket-engineering before humanity managed to achieve a moon landing.

And in the same stroke that its capabilities leap forward, its alignment properties are revealed to be shallow, and to fail to generalize. The central analogy here is that optimizing apes for inclusive genetic fitness (IGF) doesn't make the resulting humans optimize mentally for IGF. Like, sure, the apes are eating because they have a hunger instinct and having sex because it feels good—but it's not like they could be eating/fornicating due to explicit reasoning about how those activities lead to more IGF. They can't yet perform the sort of abstract reasoning that would correctly justify those actions in terms of IGF. And then, when they

...
Hyperreals in a Nutshell
by Yudhister Kumar — 2023-10-15T14:23:58.027Z
Link: https://ykumar.org/hyperreals-in-a-nutshell/

Epistemic status: Vaguely confused and probably lacking a sufficient technical background to get all the terms right. Is very cool though, so I figured I'd write this.

And what are these Fluxions? The Velocities of evanescent Increments? And what are these same evanescent Increments? They are neither finite Quantities nor Quantities infinitely small, nor yet nothing. May we not call them the ghosts of departed quantities?

George Berkeley, The Analyst

When calculus was invented, it didn't make sense. Newton and Leibniz played fast and dirty with mathematical rigor to develop methods that arrived at the correct answers, but no one knew why. It took another one and a half centuries for Cauchy and Weierstrass develop analysis, and in the meantime people like Berkeley refused to accept the methods utilizing these "ghosts of departed quantities."

Cauchy's and Weierstrass's solution to the crisis of calculus was to define infinitesimals in terms of limits. In other words, to not describe the behavior of functions directly acting on infinitesimals, but rather to frame the the entire endeavour as studying the behaviors of certain operations in the limit, in that weird superposition of being arbitrarily close to something yet not it.

(And here I realize that math is better shown, not told)

The limit of a function f(x)at x=a is L if for any ϵ>0 there exists some δ>0 such that if

|x−a|<δ,

then

|f(x)−L|<ϵ.

Essentially, the limit exists if there's some value δ that forces f(x) to be within ϵ of L if x is within δ of a. Note that this has to hold true for all ϵ, and you choose ϵ first!

From this we get the well-known definition of the derivative:

f′(x)=limh→0f(x+h)−f(x)h

and you can define the integral similarly.

The limit solved calculus's rigor problem. From the limit the entire field of analysis was invented and placed on solid ground, and this foundation has stood to this day.

Yet, it seems like we lose something important when we replace the idea of the "infinitesimally small" with the "arbitrarily close to." Could we actually make numbers that were infinitely small?

The Sequence Construction

Imagine some mathematical object that had all the relevant properties of the real numbers (addition, multiplication are assoc...
Discovering Latent Knowledge in the Human Brain: Part 1 – Clarifying the concepts of belief and knowledge
by Joseph Emerson — 2023-10-15T09:02:19.509Z

Acknowledgements: Many thanks to Milan Cvitkovic and Collin Burns for their help in workshopping this project proposal. Many of the ideas presented here regarding the use of neuroscience and neurotechnology were originally proposed by Milan

here

. This post represents my current thinking on the topic and does not necessarily represent the views of the individuals I have just mentioned.

tl;dr: Methods for promoting honest AI hinge on our ability to identify beliefs/knowledge in models. These concepts are derived from human cognition. In order to use them in AI, we need to provide definitions at the appropriate level of resolution. In this post, I describe why I think it would be a useful starting point to recover truth-like latent features in human brain data with contrast-consistent search (CCS). In an upcoming post, I will outline why I think this is possible with current fMRI technology.

Summary

Methods for discovering latent knowledge in language models, such as contrast-consistent search (CCS), hold the potential to enable humans to directly infer a model’s “beliefs” from its internal activations. This can be useful for promoting honesty in AI systems by ensuring that their outputs are consistent with their “beliefs”.

However, it is currently unclear how the truth-like features recovered by CCS relate to the concepts of belief and knowledge, whose definitions are derived from human intelligence. Since these concepts underlie behaviors like deception and honesty, is essential that we understand them at the appropriate level of resolution to enable us to translate the concepts to AI systems.

A starting point could be to determine whether CCS identifies analogous features in human brains. My proposal is to use ultra-high-field functional magnetic resonance imaging (fMRI) to replicate discovering latent knowledge experiments in humans. This approach not only promises to shed light on the representations identified by CCS but also offers a crucial testbed for evaluating techniques aimed at detecting deception in AI.

About this post

I am posting a proposal to perform a human version of "Discovering Latent Knowledge in Language Models" in order to elicit feedback and help clarify my thinking on the motivation and utility of the project. I have separated my proposal into two posts. In this post, I’ll cover why I think it is hard to define honesty and deceptio...
Rationalist horror movies
by Elizabeth — 2023-10-15T07:42:14.509Z

I like horror movies with smart, agentic protagonists, while staying in horror genre rather than thriller or action. These are rather thin on the ground. In the spirit of Scott Garabant's Puzzle Game post, here is my list of favorites and an invitation to add yours.

My favorites:

Oculus.
Hush
Green Room

Honorable mentions. These aren't quite the total package, but are close and extremely good such that they still seemed worth including:

The Ring's protagonist is not dumb, but it's more "solving a mystery" than "well does this weapon work?"
Tremors would need to change very little to be a horror movie, but it is in fact an action movie.
It Follows spends a little too much time denying the problem
Unity Gridworlds
by WillPetillo — 2023-10-15T04:36:31.548Z

Summary:

Unity Gridworlds

is an open-source recreation of DeepMind’s 2017 paper, AI Safety Gridworlds in the Unity game engine, including a level editor that decreases the barrier to entry for non-specialists to design their own Gridworlds-style experiments.

(See this blog for a longer form of this post with more background explanation)

Background:

Gridworlds is a series of experiments published by DeepMind in 2017 involving toy environments that analogize to dangerous situations for future AI, exploring questions like “will an AI allow itself to be shut down?” and “will it act differently if it knows it is being supervised?”

Unity is an extremely popular game engine used by millions of hobbyist and professional game developers around the world. It can be downloaded for free and has a

Machine Learning Agents

plugin that abstracts away the algorithms and lets you focus on designing environments and reward systems.

Project Demonstration:

To demonstrate some of the core features of Unity Gridworlds, I have created Risk Aversion, a novel environment that explores AI willingness to search for high risk/reward strategies during training. See

this video

for a step-by-step walkthrough of the creation process, including level design, testing, training, and deployment.

Some highlights:

Environment design is done through an intuitive point-and-click graphical user interface.
Core features, such as agent movement, basic object interactions, saving/loading layouts, and result visualization are built-in.
You can watch training happen visually and in accelerated-time, gathering qualitative feedback that would be easy to miss in reports and charts (but those are also available). In my video demonstration, for example, the agent finds a high-reward strategy during training, then discards it for some unknown reason.
Training for simple environments can take as little as 5-15 minutes on a consumer-grade PC, even without GPU acceleration.

What’s Next:

My hopes with Unity Gridworlds are to provide a resource for established AI safety researchers and also to introduce AI alignment concepts to a broad audience of hobbyist and professional game developers. I am currently looking for collaborators to provide feedback regarding UI and workflow improvements, suggest (or help implement) useful features, and design interesting experiments. &...
In memory of Louise Glück
by Joe Carlsmith — 2023-10-15T02:59:42.687Z

(Cross-posted from my website. Podcast version here, or search "Joe Carlsmith Audio" on your podcast app.)

Louise Glück, circa 1977. Image source here.

Louise Glück, one of my favorite poets, died yesterday.

I took a writing workshop with her back in 2009. I remember the force and precision of her words as she spoke, slowly, in class. There was a feeling like stones falling, one by one, into place. Like something large was being built, steadily, in simple movements, and you could see it forming.

I remember, too, a meeting in her office. It was early in the semester, and she hadn’t liked my poem. She wanted me to write from some place closer to where dreams come from – something deeper and more inchoate, some raw edge. She wanted what she had called, in my memory, a “real poem.”

I read various of her poems in that class, but the book of hers that has most stayed with me, and mattered most to me, came out later, in 2014: “Faithful and Virtuous Night.” I associate this book, in particular, with the dream-like quality of many of the poems — the way they shift underneath you as you read, full of quiet mystery and surprise. Here’s one of my favorites:

A Foreshortened Journey

I found the stairs somewhat more difficult than I had expected and so I sat down, so to speak, in the middle of the journey. Because there was a large window opposite the railing, I was able to entertain myself with the little dramas and comedies of the street outside, though no one I knew passed by, no one, certainly, who could have assisted me. Nor were the stairs themselves in use, as far as I could see. You must get up, my lad, I told myself. Since this seemed suddenly impossible, I did the next best thing: I prepared to sleep, my head and arm on the stair above, my body crouched below. Sometime after this, a little girl appeared on the top of the staircase, holding the hand of an elderly woman. Grandmother, cried the little girl, there is a dead man on the staircase! We must let him sleep, said the grandmother. We must walk quietly by. He is at that point in life at which neither returning to the beginning nor advancing to the end seem bearable; therefore, he has decided to stop, here, in the midst of things, though this makes him an obstacle to others, such as ourselves. But we must not give up hope; in my own life, she continued, there was such a time, though that was long ago. And here, she l

...