trentmkelly/LessWrong-43k huggingface dataset:
Linear infra-Bayesian Bandits
by Vanessa Kosoy — 2024-05-10T06:41:09.206Z
Link: https://arxiv.org/abs/2405.05673

Linked is my MSc thesis, where I do regret analysis for an infra-Bayesian[1] generalization of stochastic linear bandits.

The main significance that I see in this work is:

Expanding our understanding of infra-Bayesian regret bounds, and solidifying our confidence that infra-Bayesianism is a viable approach. Previously, the most interesting IB regret analysis we had was Tian et al which deals (essentially) with episodic infra-MDPs. My work here doesn't supersede Tian et al because it only talks about bandits (i.e. stateless infra-Bayesian laws), but it complements it because it deals with a parameteric hypothesis space (i.e. fits into the general theme in learning-theory that generalization bounds should scale with the dimension of the hypothesis class).
Discovering some surprising features of infra-Bayesian learning that have no analogues in classical theory. In particular, it turns out that affine credal sets (i.e. such that are closed w.r.t. arbitrary affine combinations of distributions and not just convex combinations) have better learning-theoretic properties, and the regret bound depends on additional parameters that don't appear in classical theory (the "generalized sine" S and the "generalized condition number" R). Credal sets defined using conditional probabilities (related to Armstrong's "model splinters") turn out to be well-behaved in terms of these parameters.

In addition to the open questions in the "summary" section, there is also a natural open question of extending these results to non-crisp infradistributions. (I didn't mention it in the thesis because it requires too much additional context to motivate.)

^

I use the word "imprecise" rather than "infra-Bayesian" in the title, because the proposed algorithms achieves a regret bound which is worst-case over the hypothesis class, so it's not "Bayesian" in any non-trivial sense.

^

In particular, I suspect that there's a flavor of homogeneous ultradistributions for which the parameter S becomes unnecessary. Specifically, an affine ultradistribution can be thought of as the result of "take an affine subspace of the affine space of signed distributions, intersect it with the space of actual (positive) distributions, then take downwards closure into contributions to make it into a homogeneous ultradistribution". But we can also consider the alternative "take an affine subspace

...
Why Care About Natural Latents?
by johnswentworth — 2024-05-09T23:14:30.626Z

Suppose Alice and Bob are two Bayesian agents in the same environment. They both basically understand how their environment works, so they generally agree on predictions about any specific directly-observable thing in the world - e.g. whenever they try to operationalize a bet, they find that their odds are roughly the same. However, their two world models might have totally different internal structure, different “latent” structures which Alice and Bob model as generating the observable world around them. As a simple toy example: maybe Alice models a bunch of numbers as having been generated by independent rolls of the same biased die, and Bob models the same numbers using some big complicated neural net.

Now suppose Alice goes poking around inside of her world model, and somewhere in there she finds a latent variable ΛA with two properties (the

Natural Latent

properties):

ΛA approximately mediates between two different observable parts of the world X1,X2
ΛA can be estimated to reasonable precision from either one of the two parts

In the die/net case, the die’s bias (ΛA) approximately mediates between e.g. the first 100 numbers (X1) and the next 100 numbers (X2), so the first condition is satisfied. The die’s bias can be estimated to reasonable precision from either the first 100 numbers or the second 100 numbers, so the second condition is also satisfied.

This allows Alice to say some interesting things about the internals of Bob’s model.

First

: if there is any latent variable (or set of latent variables, or function of latent variables) ΛB which mediates between X1 and X2 in Bob’s model, then Bob’s ΛB encodes Alice’s ΛA (and potentially other stuff too). In the die/net case: during training, the net converges to approximately match whatever predictions Alice makes(by assumption), but the internals are a mess. An interpretability researcher pokes around in there, and finds some activation vectors which approximately mediate between X1 and X2. Then Alice knows that those activation vectors must approximately encode the bias ΛA. (The activation vectors could also encode additional information, but at a bare minimum they must encode the bias.)

Second

: if there is any latent variable (or set of latent variables, or function of latent variables) ...
What I learned from doing Quiz Bowl
by Jacob G-W — 2024-05-09T21:05:38.299Z
Link: https://jacobgw.com/blog/quizbowl/2024/05/09/qb.html

I did Quiz Bowl throughout my time in high school, and looking back on it, it was a pretty positive thing to do! In this blog post, I want to make a list of some of the life lessons I have taken from Quiz Bowl.

If you know about Quiz Bowl, just skip to the next paragraph. For the uninitiated, Quiz Bowl is a team activity where students compete, typically on teams of four, to answer questions about the world. There are questions on physics, philosophy, mathematics, chemistry, literature, history, economics, videogames, and pretty much every subject. The format of the game is simple: the moderator reads a tossup and then players can “buzz in” (no talking during tossups). If they get it correct, they earn 10 or 15 points and secure a three question bonus set for their team, with each bonus question being worth 10 points. Talking between players is allowed during the bonus questions. If a player gets a tossup wrong, their whole team is locked out from buzzing during that tossup and they can lose 5 points. If you’re interested, you can find the full rules for the NAQT style here.

Team chemistry matters more than you’d think

I was kind of surprised to learn this because Quiz Bowl is an activity based on facts. You either know the answer or you don’t. But it turns out that for some reason, encouraging your teammates and having what we call “good team chemistry” helps a lot. My team has played against some teams that were much higher ranked than us (e.g. Hunter from New York City, Thomas Jefferson from Virginia, Mira Loma from California), and in many cases, if our spirit was higher, we could beat them even if we knew less.

I don’t know the exact mechanism by which this happens but I think it has to do with how emotions affect your calibration with the buzzer (which I’ll talk about next). If everyone on the team is happy and in a good mood, it seems that people hit the sweet spot in buzzer timing. They don’t wait to long to buzz when they know the answer, but they also don’t buzz to early and get the wrong answer. Any deviation from being in a good mood causes erratic buzzing which adds to a negative feedback loop of the players feeling worse and thus playing worse, ultimately leading to a loss. This is commonly known as “tilting.” Not being happy also probably makes your thoughts and reflexes slower, which could additionally explain the necessity of the team being ...
My thesis (Algorithmic Bayesian Epistemology) explained in more depth
by Eric Neyman — 2024-05-09T19:43:16.543Z
Link: https://ericneyman.wordpress.com/2024/05/09/algorithmic-bayesian-epistemology/

In March I posted a very short description of my PhD thesis, Algorithmic Bayesian Epistemology, on LessWrong. I've now written a more in-depth summary for my blog, Unexpected Values. Here's the full post:

***

In January, I defended my PhD thesis. My thesis is called Algorithmic Bayesian Epistemology, and it’s about predicting the future.

In many ways, the last five years of my life have been unpredictable. I did not predict that a novel bat virus would ravage the world, causing me to leave New York for a year. I did not predict that, within months of coming back, I would leave for another year — this time of my own free will, to figure out what I wanted to do after graduating. And I did not predict that I would rush to graduate in just seven semesters so I could go work on the AI alignment problem.

But the topic of my thesis? That was the most predictable thing ever.

It was predictable from the fact that, when I was six, I made a list of who I might be when I grow up, and then attached probabilities to each option. Math teacher? 30%. Computer programmer? 25%. Auto mechanic? 2%. (My grandma informed me that she was taking the under on “auto mechanic”.)

It was predictable from my life-long obsession with forecasting all sorts of things, from hurricanes to elections to marble races.

It was predictable from that time in high school when I was deciding whether to tell my friend that I had a crush on her, so I predicted a probability distribution over how she would respond, estimated how good each outcome would be, and calculated the expected utility.

And it was predictable from the fact that like half of my blog posts are about predicting the future or reasoning about uncertainty using probabilities.

So it’s no surprise that, after a year of trying some other things (mainly auction theory), I decided to write my thesis about predicting the future.

If you’re looking for practical advice for predicting the future, you won’t find it in my thesis. I have tremendous respect for groups like Epoch and Samotsvety: expert forecasters with stellar track records whose thorough research lets them make some of the best forecasts about some of the world’s most important questions. But I am a theorist at heart, and my thesis is about the theory of forecasting. This means that I’m interested in questions like:

How do I pay Epoch and Samotsvety for their forecasts in a way that

...
Dyslucksia
by Shoshannah Tekofsky — 2024-05-09T19:21:33.874Z

The curious tale of how I mistook my dyslexia for stupidity - and talked, sang, and drew my way out of it.

Sometimes I tell people I’m dyslexic and they don’t believe me. I love to read, I can mostly write without error, and I’m fluent in more than one language.

Also, I don’t actually technically know if I’m dyslectic cause I was never diagnosed. Instead I thought I was pretty dumb but if I worked really hard no one would notice. Later I felt inordinately angry about why anyone could possibly care about the exact order of letters when the gist is perfectly clear even if if if I right liike tis.

I mean, clear to me anyway.

I was 25 before it dawned on me that all the tricks I was using were not remotely related to how other people process language. One of my friends of six years was specialized in dyslexia, and I contacted her, full excitement about my latest insight.

“Man, guess what? I realized I am dyslectic! This explains so much! I wish someone had told me sooner. It would have saved me so much grief.”

“Oh, yeah, I know.”

“Wait, what?”

“You are very obviously dyslectic.”

“Wait, why didn’t you tell me?”

“You didn’t seem bothered.”

“Oh…”

Turns out my dyslexia was a public secret that dated back all the way to my childhood (and this was obviously unrelated to my constitutional lack of self-awareness).

Anyway.

How come I kind of did fine? I’m fluent in English (not my native language), wrote my PhD thesis of 150 pages in 3 months without much effort, and was a localization tester for Dutch-English video game translation for two years.

I also read out loud till the age of 21, trace every letter like it’s a drawing, and need to sing new word sounds to be able to remember them.

I thought everyone had to but no one sent me the memo.

Dear reader, not everyone has to.

When I recently shared my information processing techniques with old and new friends, they asked if I had ever written them down so maybe other people could use them too.

I hadn’t.

So here is my arsenal of alternative information processing techniques.

Read Out Loud

Honestly, I didn’t realize there was an age where you were supposed to stop doing this. In school you obviously had to whisper to yourself. At home you go to your room and read at normal volume. If it’s a fiction book, you do voices for the different characters. It’s great.

I remember my sister sometimes walking in to my room when I was little cause...
Thoughts on the relative economic benefits of polyamorous relationships?
by OKlogic — 2024-05-09T18:26:03.142Z

I have a limited desire to pursue polyamory, but I have a strong inclination that when properly managed, a stable polycule would be far more economically efficient.

Has any serious literature (books, studies, ect) been published on this question? Any thoughts?*

Off the top of my head:

Advantages of Monogamy:

Less of risk of STIs
Generally More Stable (data is lacking, most likely due to the stigma surrounding polyamorous relationships naturally creating response bias, but it seems intuitively correct that it is more difficult to maintain the relationship).
Sanctioned by society at large (especially relevant for public-facing careers and familial inheritance)
Less Time Intensive/Easier to Arrange

Advantages of Polyamory:

Allows you to easily influence more people (ie: gain social capital)*
More people = more efficiency in general (Buying in bulk, sharing living space, etc).
More opportunities for intercourse

Edits to steer the conversation in a more productive direction + Footnotes:

I know this is far less common in the wild, but I also want to discuss a form of polyamory which is a strict, potentially contractual agreement similar to marriage, in which each party agrees to abstain from extra-polycule relations.

*On point one for the advantages of polyamory, the idea is that it is easier to influence the decisions of romantic partners than friends (on average) is a pretty huge deal in my opinion. It could just be because I am fairly introverted (ie socially challenged), but it seems much easier to change your family's behavior than friends. This should have serious implications for EA folk, as in theory, you can increase effective donations by a great margin by engaging in a polyamorous relationship, assuming there are no other disadvantages to polyamory.

*Okay, I get that relationships are complex and it's difficult to study them due to a large number of confounding variables, but has there really been no systematic analysis (ie a book) on the economics of polyamory?
AI Safety Strategies Landscape
by Charbel-Raphaël — 2024-05-09T17:33:45.853Z

The full draft textbook is available here. This document constitutes the Chapter 3.

Introduction

tldr: Even if we still don't know how to make AI development generally safe, many useful classes of strategies already exist, which are presented in this chapter. You can look at the table of contents and the first figure to see the different classes of strategies presented in this document.

Epistemic Status: I'm pretty satisfied with this document. I wrote it because it doesn't seem like we've made any major breakthroughs in alignment in the last year, and I wanted to consolidate what I know. And beyond alignment, it seems to me that a large class of strategies are quite important and neglected, and will continue to be relevant in the future. Alignment is only one class of strategy to achieve AI safety. And to mitigate misuses and systemic risks, I think we already have a pretty good idea of what could be done. Let me know if you think there is a major blind spot in this document.

Although the field of AI safety is still in its infancy, several measures have already been identified that can significantly improve the safety of AI systems. While it remains to be seen if these measures are sufficient to fully address the risks posed by AI, they represent essential considerations. The diagram below provides a high-level overview of the main approaches to ensuring the safe development of AI.

Tentative diagram summarizing the main high-level approaches to make AI development safe.

This document is far from exhaustive and only scratches the surface of the complex landscape of AI safety. Readers are encouraged to explore this

recent list of agendas

for a more comprehensive review.

AI Safety is Challenging

Specific properties of the AI safety problem make it particularly difficult.

AI risk is an emerging problem that is still poorly understood. We are not yet familiar with all its different aspects, and the technology is constantly evolving. It's hard to devise solutions for a technology that does not yet exist, but these guardrails are also necessary because the outcome can be very negative.

The field is still pre-paradigmatic. AI safety researchers disagree on the core problems, difficulties, and main threat models. For example, some researchers think that takeover risks are more likely [

AGI Ruin

], and some research emphasizes more progressive failure modes with progressive...
Knowledge Base 9: Increasing intelligence
by iwis — 2024-05-09T15:46:07.451Z

The post Knowledge Base 2: The structure and the method of building I proposed to build a database of credible knowledge, understandable both to people and computers, including AI. This database could be used as a source of information in computer applications, and to increase the collective intelligence of people by easing and speeding up the exchange of knowledge and information between them. In my opinion, it could also be used to improve AI.

The current AI, including LLMs, has among others the following problems:

the sources of information available for training LLMs often have low credibility;
LLMs do not have detailed knowledge about the world; e.g., GPT will not tell you where to buy coffee nearby;
LLMs often hallucinate;
there is a risk associated with the creation of artificial general intelligence because large neural networks are black boxes and we do not know what they do internally.

The importance of the first three topics for the development of AI was discussed at the following times: 0:57:10-0:59:10, 1:00:30-1:00:50, 1:25:00-1:28:25 during the Twitter Spaces session about xAI in July 2023.

The proposed database of credible knowledge could be used with LLMs in two ways. First, it could be used in the training process. Second, it could be used with the trained LLM as an external source of credible information on any topic, if the LLM lacks it. Finally, because the knowledge database is human-readable, it could be safer than a black-box LLM. The knowledge database can also be used as a source of knowledge for AI types other than neural networks.

In summary, the proposed knowledge database could enhance the intelligence of people and computers, making the AGI safer at the same time. Additionally, increasing the collective intelligence of people might also be an alternative method for achieving superintelligence.
We might be missing some key feature of AI takeoff; it'll probably seem like "we could've seen this coming"
by Lukas_Gloor — 2024-05-09T15:43:11.490Z

Predicting the future is hard, so it’s no surprise that we occasionally miss important developments.

However, several times recently, in the contexts of Covid forecasting and AI progress, I noticed that I missed some crucial feature of a development I was interested in getting right, and it felt to me like I could’ve seen it coming if only I had tried a little harder. (Some others probably did better, but I could imagine that I wasn't the only one who got things wrong.)

Maybe this is hindsight bias, but if there’s something to it, I want to distill the nature of the mistake.

First, here are the examples that prompted me to take notice:

Predicting the course of the Covid pandemic:

I didn’t foresee the contribution from sociological factors (e.g., “people not wanting to get hospitalized” – Zvi called it “

the control system

”).

As a result, I overpredicted the difference between countries with a lockdown policy vs ones without. (Note that this isn’t necessarily an update against the cost-effectiveness of lockdowns because the update goes both ways: lockdowns saved fewer lives than I would’ve predicted naively, but costs to the economy were also lower compared to the counterfactual where people already social-distanced more than expected of their own accord since they were reading the news about crowded hospitals and knew close contacts who were sick with the virus.)

Predicting AI progress:

Not foreseeing that we’d get an Overton window shift in AI risk awareness.

Many EAs were arguably un(der)prepared for the possibility of a “chat-gpt moment,” where people who weren’t paying attention to AI progress previously got to experience a visceral sense of where AI capabilities progress is rapidly heading. As a result, it is now significantly easier to make significant policy asks to combat AI risks.

Not foreseeing wide deployment of early-stage “general” AI and the possible irrelevance of AI boxing.

Early discussions of AI risk used to involve this whole step about whether a superhuman AI system could escape and gain access to the internet. No one (to my knowledge?) highlighted that the future might well go as follows:
“There’ll be gradual progress on increasingly helpful AI tools. Companies will roll these out for profit and connect them to the internet. There’ll be discussions about how these systems will eventually become dangerous, and safety-concerned groups might e

...
Four Unrelated Is Over
by jefftk — 2024-05-09T14:50:02.389Z

Somerville historically had a zoning ordinance limiting housing units
to at most four unrelated people:

any number of persons related by blood, marriage, adoption, or
foster care agreement and up to three (3) additional unrelated persons
living together as a single housekeeping unit

This is something I'd been
unhappy about for years, and was enthusiastic about the "4 unrelated
is outdated" campaign to change it in in
2018. So I'm very happy that after a
request for a variance the city council instead ended up
removing
the restriction.

The actual change was in November, so I'm a bit late on this!

I also think there was an oversight, where the removal didn't include changing
the text in section
7-153 which says "All schools shall be responsible for publicizing
to their students the limitations of the city's zoning ordinance which
limits occupancy to not more than four unrelated individuals." I've
written to the city council to let them know.

I've also noticed that several school-affiliated sites still list
this limitation:

Tufts: Off-Campus
Housing and Zoning
Ordinances.

Harvard: Harvard Housing Off
Campus and Beckwith
Circle.

I've written to Tufts and Harvard to let them know this has changed.
I wasn't able to find this listed on any MIT or Lesley sites, and
didn't check all the other Boston-area college websites.