trentmkelly/LessWrong-43k huggingface dataset:
rabbit (a new AI company) and Large Action Model (LAM) 
by MiguelDev — 2024-01-10T13:57:48.371Z

Just sharing it here, curious of what the LW community would think of rabbit's neuro-symbolic approach and their new device "R1"?

This is one of the two interactive videos in their research page, their training involved observing realtime data as I understood it.

This is probably the meat of their research page:
"We believe that in the long run, LAM exhibits its own version of "scaling laws," [3] where the actions it learns can generalize to applications of all kinds, even generative ones.Over time, LAM could become increasingly helpful in solving complex problems spanningmultiple apps that require professional skills to operate. By utilizing neuro-symbolic techniques in the loop, LAM sits on the very frontier ofinterdisciplinary scientific research in language modeling (LM), programming languages (PL), and formal methods (FM). Traditionally, the PL/FM community has focused on symbolic techniques — solver technologies that rely on logical principles of induction, deduction, and heuristic search. While these symbolic techniques can be highly explainable and come with strong guarantees, they suffer from a scalability limit. By contrast, recent innovations in the LM community are grounded in machine learning and neural techniques: while highly scalable, they suffer from a lack of explainability and come with no guarantees of the output produced. Inspired by the success of machine learning and neural techniques, the PL/FM community hasrecently made waves of progress on neuro-symbolic methods: by putting together neural techniques (such as LLM) and symbolic ones, one ends up combining the best parts of both worlds, making the task of creating scalable and explainable learning agents a feasibleone. Yet to date, no one has put cutting-edge neuro-symbolic techniques into production — LAM seeks to pioneer this direction."

R1 device

Rabbit made strong claims that LAM is better than an LLM in doing tasks. A demo video presented their device called "R1" that uses LAM as it's operating system.

The conclusion in their research page:

embodiment in AI-native devices

We believe that intelligence in the hands of the end-user is achievable without heavy client-side computing power. By carefully and securely offloading the majority of computation to data centers, we open up opportunities for ample performance and cost optimizations, making cutting-edge interactive AI experie

...
Saving the world sucks
by Defective Altruism — 2024-01-10T05:55:46.504Z

I don’t want to save the world. I don’t want to tile the universe with hedonium. I don’t want to be cuckolded by someone else’s pretty network-TV values. I don’t want to do anything I don’t want to do, and I think that’s what (bad) EAs, mother Teresa, and proselytizing Christians all get wrong. Doing things because they sound nice and pretty and someone else says they’re morally good suuucks. Who even decided that warm fuzzies, QALYs, or shrimp lives saved are even good axes to optimize? Because surely everyone doesn’t arrive at that conclusion independently. Optimizing such universally acceptable, bland metrics makes me feel like one of those blobby, soulless corporate automata in bad tech advertisements.

I don’t see why people obsess over the idea of universal ethics and doing the prosocial thing. There’s no such thing as the Universal Best Thing, and professing the high virtue of maximizing happiness smacks of an over-RLHFed chatbot. Altruism might be a “virtue”, as in most people’s evolved and social environments cause them to value it, but it doesn’t have to be. The cosmos doesn’t care what values you have. Which totally frees you from the weight of “moral imperatives” and social pressures to do the right thing.

There comes a time in most conscientious, top-of-distribution kids’ lives when they decide to Save the World. This is very bad. Unless they really do get a deep, intrinsic satisfaction from maximizing expected global happiness, they’ll be in for a world of pain later on. After years of spinning their wheels, not getting anywhere, they’ll realize that they hate the whole principle they’ve built their life around. That, deep down, their truest passion doesn’t (and doesn’t have to) involve the number of people suffering malaria, the quantity of sentient shrimps being factory farmed, or how many trillion people could be happy in a way they aren’t 1000 years from now. I claim that scope insensitivity isn’t a bug. That there are no bugs when it comes to values. That you should care about exactly what you want to care about. That if you want to team up and save the world from AI or poverty or mortality, you can, but you don’t have to. You have the freedom to care about whatever you want and shouldn’t feel social guilt for not liking the same values everyone else does. Their values are just as meaningful (or meaningless) as yours. Peer pressure is an evol...
Questions about Solomonoff induction
by mukashi — 2024-01-10T01:16:58.595Z

I am confused about Somolonoff induction and some other related concepts. I appreciate some help from anyone who understands this.

What I understand so far is:

I understand Bayesian statistics and the problem of choosing the priors
I understand that Kolmogorov complexity can be used to measure the complexity of an hypothesis
I think to understand that Somolonoff induction consists of evaluating all the possible hypotheses generating some observed data, and weighting them according to their complextiy

For example, the sequence HTHTT could be generated by different programs:

A program that flips a fair coin
A program that flips a coin with p=0.4
A program that generates "exactly" the sequence HTHTT
... etc. Infinite rules could be written here

I do understand that a program that flips a fair coin is simpler than a program that generates the exact output

What I don't understand is:

How do you weight exactly complexity against likelihood?

In the previous example, we could say that the first program has complexity X bits while the the third program has complexity Y, bein Y > X (but only a small difference, because we don't have that many observations). Here is where things get a bit murky for me. The "predictive" power of the first program to explain that specific output, being probabilistic, is 1/2^5, while for the second program the likelihood is 1. To prefer the first hypothesis we have to assign a prior probability at least 2^5 times higher to the first program than to the second one. Where does this 2^5 factor comes from?

I know that it is a theoretical thing but if you can use coin flips in the explanation, that helps a lot. If you identify other places where I am confused but mistakenly thought I wasn't, please point that out.
AI as a natural disaster
by Neil  — 2024-01-10T00:42:39.616Z

[This post is not meant for the rationalist audience directly, because it has already internalized this lesson, but is to be used as an analogy in discourse with an unfamiliar crowd. The end goal is to make people intuitively understand why AI is extremely risky whether or not it has much agency, by burning through a lot of inferential steps quickly. I take it as a given that p(doom) is high. I also assume that it doesn't take that much more intelligence than that of a very intelligent human in order to significantly increase the probability of doom. Note: by “AI” I often mean “ASI”.]

I believe one of the fundamental misunderstandings in AI discourse has to do with anthropomorphizing AI too much. We give a sense of agency to AI when we refer to it as “malignant AI”, or when we talk of “deceptive alignment”. It’s not that giving a sense of agency is bad; it is necessary and helpful in many ways. But in other ways, I think it actively inhibits people from understanding the sheer aura of threat something more intelligent than they would exhibit (regardless of its agency/intention). If you linger on agency, you point people in the direction of wondering why AI would want to kill everyone; and then they stray in the direction of Skynet, and conclude that there is no reason in particular why AI would hate them. Uh oh, now the burden of proof is suddenly on you to explain orthogonality and instrumental convergence, which costs you precious and finite inferential steps.

Before you even get to agency—before you let people think of reasons why AI might want them dead— you must underline why capabilities, in and of themselves, put the burden of proof on those who claim AI is safe by default. You cannot jump the gun on this: it must be made clear why any claims arguing a superintelligence is not inherently dangerous are to be immediately investigated and treated with high suspicion because it is a very strong assumption.

Hopefully this analogy will serve as a useful intuition pump for this.

tl;dr: There is a certain string of words, in the possibility space of all strings, which flawlessly explains how to kill all humans. This string, if it were printed out onto paper, would be a major infohazard to humanity. It would have to be kept away from as many agents as possible, because if a single person uncovere...
Stop being surprised by the passage of time
by duck_master — 2024-01-10T00:36:51.598Z

(Note: Yes, I[1] know this post is nine days late. It's still the early into the first month of the year, so I guess it still counts! Also, I'm not very good at wordsmithing.)

Why are you often surprised at the passage of time?

First of all, time is always passing. You should know that already. Since everything changes when time passes, everything is always changing.[2]

That said, there is a common phenomenon, where you are often surprised by the passage of time. This is because the way you think about the world is often static - or, in more rationalist-esque terminology, you are caching your mental models. More precisely: when you see, hear, or read of something, you usually don't think, "This is what [thing] was like at [time]". More commonly, you just think: "This is what [thing] is like". Then, when you revisit this particular thing, you notice that it's different from what you remember, and feel a pang of surprise. But you predictably felt surprised because you forgot to think about how that thing may have evolved since you last visited it. I call this "being surprised by the passage of time".[3]

Here are some examples of the thing I'm trying to get at the spirit of:

The new year! This is probably the most quintessential example. When January 1 of a year arrives, you still feel like it was the previous year - you often don't really internalize the year you're in as being the current year until February or so.[4]
You might visit a landmark one day (e.g. a street, a park, a school, etc.) as a tourist or visitor. But the next time you stop by, it might have been renovated or removed to your surprise.
You might have spoken with an old friend many years ago. But when you speak to the friend next, you're surprised that they're so old, even though everyone is always getting older.
You might get an interesting gadget (like a computer, a smartphone, or a robot), which was state-of-the-art when you bought it. But later on it might feel woefully obsolete, and you might start coveting the newer gadgets your friends all got instead.

The new year, and how to not be surprised by the passage of time

New Year's Day is a celebration of the new year, and we all celebrate it.[5] But I propose that it is really a celebration of the passage of time - and, by extension, a celebration of the virtue of not being surprised by the passage of time.

How should you celebrate Ne...
A discussion of normative ethics
by Gordon Seidoh Worley — 2024-01-09T23:29:11.467Z

Adam Zerner

Hey Gordon. Thanks for being willing to chat with me about this idea I have of Reflective Consequentialism. I've been wanting to do so for a while now.

To start, let me provide some context regarding what lead me to think about it.

Consequentialism makes a lot of sense to me. I vibe pretty strongly with Scott Alexander's Consequentialism FAQ. And sometimes I read things that make me think that people in the rationality community all basically agree with it as well.

But then I read things a bunch of things praising virtue ethics that make me feel confused. The idea with virtue ethics, from what I understand, is that yes, consequentialism is what makes the most sense. But, if humans try to actually make decisions based on which action they think will lead to the best consequences, they'll kinda do a bad job at it.

On the other hand, if they pick some virtues (for example, honesty and humility) and make decisions according to what these virtues would advise, it will lead to better consequences. So then, utilizing virtue ethics leads to better consequences than utilizing consequentialism, which means that humans should utilize virtue ethics when making decisions.

Ok. Now here is where I'm confused. This virtue stuff kinda just feels like a bunch of heuristics to me. But why limit your decision making to those particular heuristics? What about other heuristics? And what about, as opposed to heuristics, more "bottom-up", "inside view" calculus about what you expect the consequences to be? I feel like there is a place for all of those things.

But I also feel like virtue ethics is saying "no, forget about all of that other stuff; your decisions should be based on those heuristics-virtues alone". Which sounds very unreasonable to me. Which makes me surprised at how popular it appears to be on LessWrong.

(This is a little rambly and open-ended. Sorry. Feel free to take this where you want to go, or to ask me to propose a more targeted path forward.)

Gordon Seidoh Worley

I'm excited to talk about this because I'm pretty dedicated to virtue ethics at this point as my personal way of reasoning about how to act in my own life. To give context to readers who might not have it, I received lay ordination (jukai) within a Soto Zen lineage, and that required taking the 16 Bodhisattva Precepts. These precepts are explicit virtues I'm expected to uphold, and are, at least with

...
On the Contrary, Steelmanning Is Normal; ITT-Passing Is Niche
by Zack_M_Davis — 2024-01-09T23:12:20.349Z

Rob Bensinger argues that "ITT-passing and civility are good; 'charity' is bad; steelmanning is niche".

The ITT—Ideological Turing Test—is an exercise in which one attempts to present one's interlocutor's views as persuasively as the interlocutor themselves can, coined by Bryan Caplan in analogy to the Turing Test for distinguishing between humans and intelligent machines. (An AI that can pass as human must presumably possess human-like understanding; an opponent of an idea that can pass as an advocate for it presumably must possess an advocate's understanding.) "Steelmanning" refers to the practice of addressing a stronger version of an interlocutor's argument, coined in disanalogy to "strawmanning", the crime of addressing a weaker version of an interlocutor's argument in the hopes of fooling an audience (or oneself) that the original argument has been rebutted.

Bensinger describes steelmanning as "a useful niche skill", but thinks it isn't "a standard thing you bring out in most arguments." Instead, he writes, discussions should be structured around object-level learning, trying to pass each other's Ideological Turing Test, or trying resolve cruxes.

I think Bensinger has it backwards: the Ideological Turing Test is a useful niche skill, but it doesn't belong on a list of things to organize a discussion around, whereas something like steelmanning naturally falls out of object-level learning. Let me explain.

The ITT is a test of your ability to model someone else's models of some real-world phenomena of interest. But usually, I'm much more interested in modeling the real-world phenomena of interest directly, rather than modeling someone else's models of it.

I couldn't pass an ITT for advocates of Islam or extrasensory perception. On the one hand, this does represent a distinct deficit in my ability to model what the advocates of these ideas are thinking, a tragic gap in my comprehension of reality, which I would hope to remedy in the Glorious Transhumanist Future if that were a real thing. On the other hand, facing the constraints of our world, my inability to pass an ITT for Islam or ESP seems ... basically fine? I already have strong reasons to doubt the existence of ontologically fundamental mental entities. I accept my ignorance of the reasons someone might postulate otherwise, not out of contempt, but because I just don't have the time.

Or think of it this w...
What's the protocol for if a novice has ML ideas that are unlikely to work, but might improve capabilities if they do work?
by drocta — 2024-01-09T22:51:23.291Z

This may be a silly question.
Suppose (I) someone who doesn't have practical experience with machine learning research (but is generally aware of the structure of a number of different NN architectures, has read some papers, has a decent level of mathematical maturity, etc.) has an idea for a (as far as they/I know) novel NN architecture. And, suppose (I) they understand that, given (my) their lack of experience in the topic, and how most ideas from outsiders are unlikely to work, the idea probably will almost certainly not work.
Suppose also that implementing and testing the idea well would either take (me) this person much longer than it would people who are versed in the field, or just straight up be beyond their (my) current skill-set.
But, suppose that the idea seems, in the unlikely case that it would work, more likely to improve capabilities than to improve interpretability or safety.
(For concreteness, assume that the improvement in capabilities wouldn't be particularly fundamental. Like, "an idea for potentially increasing the context window length cheaply in LLMs" or something like that.)

What is the appropriate protocol for me/such-a-person to follow?
Some ideas that come to mind are:
1) don't bother anyone with the idea, as it is unlikely to work, and is irrelevant (this option seems safe of course, but less satisfying than getting confirmation that it wouldn't/doesn't actually work)
2) spend a bunch of time learning how, and then try testing it privately (without making the idea public), and in the unlikely even that it seems to maybe-kinda-work in the small-scale that one(I) is/am able to try, approach someone who is more experienced and is appropriately concerned about AI safety issues and ask them to evaluate whether it is likely to actually work when scaled up (or, whether it has perhaps already been done before), and if it is novel and works, what to do about it
3) consult some kind of list or something to determine whether capability improvements in that direction are sufficient safe that, if the idea works, it would be better for it to be publicly available, than to keep it private, or visa versa
4) rather than trying to test it oneself(myself), (after spending time searching the literature checking if it's already been tried), ask someone knowledgeable whether it sounds likely to work
5) perhaps, if the idea would work, and such an inexperien...
Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer – a New Metaphor
by RogerDearnaley — 2024-01-09T20:42:28.349Z

Thanks to

Quentin FEUILLADE--MONTIXI

for the discussion in which we came up with this idea together, and for feedback on drafts.

TL;DR A better metaphor for how LLMs behave, how they are trained, and particularly for how to think about the alignment strengths and challenges of LLM-powered agents. This is informed by simulator theory — hopefully people will find it more detailed, specific, and helpful than the old shoggoth metaphor.

Humans often think in metaphors. A good metaphor can provide a valuable guide to intuition, or a bad one can mislead it. Personally I've found the shoggoth metaphor for LLMs rather useful, and it has repeatedly helped guide my thinking (as long as one remembers that the shoggoth is a shapeshifter, and thus a very contextual beast).

However, as posts like

Why do we assume there is a "real" shoggoth behind the LLM? Why not masks all the way down?

make clear, not everyone finds this metaphor very helpful. (My reaction was "Of course it's masks all the way down — that's what the eyes symbolize! It's made of living masks: masks of people.") Which admittedly doesn't match

H.P. Lovecraft's description

; perhaps it helps to have spent time playing around with base models in order to get to know the shoggoth a little better (if you haven't, I recommend it).

So, I thought I'd try to devise a more useful and detailed metaphor, one that was a better guide for intuition, especially for alignment issues. During

a conversation

with

Quentin FEUILLADE--MONTIXI

we came up with one together (the stage and its animatronics were my suggestions, the puppeteer was his, and we tweaked it together). I'd like to describe this, in the hope that other people find it useful (or else that they rewrite it until they find one that works better for them).

Along the way, I'll show how this metaphor can help illuminate a number of LLM behaviors and alignment issues, some well known, and others that seem to be less widely-understood.

A Base Model: The Stage and its Animatronics

A base-model LLM is like a magic stage. You construct it, then you read it or show it (at enormous length) a large proportion of the Internet, and if you wish also books, scientific papers, images, movies, or whatever else you want. The stage is inanimate: it's not agentic, it's

goal agnostic

(well, unless you want consider 'contextually guess the next t...
Bent or Blunt Hoods?
by jefftk — 2024-01-09T20:10:11.545Z

One of the many new EV companies,

Canoo, is making a van:

Looking at the shape of it, one of the reactions I saw was:

This vehicle looks like a pedestrian killing machine ... the high
blunt shape of the vehicle is going to rocket pedestrians to their
death in an impact.

Let's compare to a few different vans:

Mercedes-Benz
Sprinter modern full-size van.

Ford
E-Series (Econoline) traditional full-size van.

Toyota Sienna minivan

These are all more angled than the Canoo, but let's compare their
profiles:

All three have an approximately vertical section, followed by a bend
inwards. On the Sienna that bend is at ~40", while on the other three
it's ~48".

Unfortunately all of these are too high to help much. The
Institute for Highway Safety found
that for hoods 40"+, a sloped vs blunt profile doesn't matter: they're
both ~45% more dangerous than a traditional car's low and sloped
front. Which makes sense intuitively: the bend would need to be below
your center of gravity, and for that to be above 40" you generally
need to be >6ft.

One thing this design does have going for it relative
to the many new vehicles with tall blunt front ends is good forward
visibility. Because there's no engine or frunk they can make the
bottom front panel transparent:

While I'm not enthusiastic about the general shift towards taller and
blunter front ends, delivery vans do seem like the sort of place where
they make the most sense. A long sloping front means you either need
a longer van or a lower capacity, which is why old vans like the
original VW Bus
or Ford
Taunus Transit were flat-fronted.