trentmkelly/LessWrong-43k huggingface dataset:
Occlusions of Moral Knowledge
by herschel — 2023-12-25T05:55:16.529Z
Link: https://brothernin.substack.com/p/occlusions-of-moral-knowledge

I want to discuss a notion of moral knowledge, and ways in which it can be occluded. I think these sorts of occlusions are upstream of some vital conversations that aren’t getting anywhere.

Moral Knowledge

We have some sort of innate capacity to apprehend goodness, beauty, rightness, etc. Presumably there's some standard term elsewhere for this, but I will call this moral knowledge.

Moral knowledge isn't necessarily about explicit theories of morality. My term might evoke some connotation of formality, moral propositions, etc. etc.--none of that is really what's meant. Moral knowledge comes before "morality", it's experientially basic and common to all people. (There's some diversity, obviously, which is part of what I want to discuss.)

This is in many respects different from episteme, and from the kinds of knowledge that we are normally used to including under 'epistemology'. I'm not necessarily attached to the term I've chosen, but I think it's fairly apt. At some point later I want to discuss some attempts towards moral or spiritual epistemics.

Occlusions

There are quite common reports in my circles of people doing psychedelics, meditation, therapy etc. and having experiences of "openings", after which they are aware of or sensitive to some aspect of experience which was previously invisible, or unavailable, or whatever. Similar reports also come from a cluster of contemporary contemplative traditions, and also align with some variously older spiritual and mystical traditions.

These reports often go along the lines of "wow, this was always happening but I could never feel it," or "suddenly I realized that the person in front of me was actually real," "I've been disowning this part of myself my whole life," etc., including even "I could see that we're all just love loving love." Note, btw, these ones are all (caricatured) reports of sudden changes, which are pithier, but many reported openings are gradual.

At least for some chunk of these, people often refer to experience without, or before, such openings as 'dissociated'. This is often a sloppy term, and the meanings used for it vary widely, but it's still helpful. Also, note that access to these not-dissociated states often fluctuates with various patterns.

Fine, so obviously some of these are going to be ontologically or metaphysically problematic to somebody, probably even all of them. People will sometimes den...
Would you have a baby in 2024?
by martinkunev — 2023-12-25T01:52:04.358Z

Given how fast AI is advancing and all the uncertainty associated with that (unemployment, potential international conflict, x-risk, etc.), do you think it's a good idea to have a baby now? What factors would you take into account (e.g. age)?

Today I saw a tweet by Eliezer Yudkowski that made me think about this:

"When was the last human being born who'd ever grow into being employable at intellectual labor? 2016? 2020?"

https://twitter.com/ESYudkowsky/status/1738591522830889275

Any advice for how to approach such a discussion with somebody who is not at all familiar with the topics discussed on lesswrong?

What if the option "wait for several years and then decide" is not available?
align your latent spaces
by bhauth — 2023-12-24T16:30:09.138Z
Link: https://www.bhauth.com/blog/thinking/align%20latents.html

I learned very early the difference between knowing the name of something and knowing something.

— Richard Feynman

Here is a simplified diagram of the mind of someone reading English text and writing English text in response:

Here is a simplified diagram of the mind of someone fluent in English who is beginning to learn French and studying using flashcards with word pairs:

Red arrows indicate the main path of information from flashcards. Maybe you can see the reason I don't consider flashcards an optimal approach to learning languages. That approach can load word associations into a vector lookup system, but what's needed is encoder/decoder training. For that to happen with flashcards, the best-case scenario is training across the entire chain of two encoders to increase similarity at the concept latent space, and flashcards are inefficient for that because they provide a small amount of data without context. Anyone who understands neural networks knows that large amounts of in-context real data is better than repeatedly using small numbers of small fragments.

So, instead of using flashcards, I think it's better to read pairs of complete paragraphs with the same meaning, without repetition. For example, reading a book together with its translation. The foreign text not being understood as it's read doesn't mean that reading it is ineffective as an approach.

One reason flashcards are overrated is that a lot of language tests are for "vocabulary" because association between word pairs is easy to test.

Align your spine.

— Malcolm in the Middle

Once someone has become fluent in a 2nd language, they typically gain the ability to write and understand sentences using words from both languages. That implies integration of the encoders and decoders, which presumably happens via distillation, as in this diagram:

Considering that this happens with more practice and is seen in more-fluent people, it's probably better overall.

This integration of multiple systems for different languages seems to me to be fundamentally the same process as integration of understanding of multiple scientific fields. The equivalent of reading a book together with its translation for that would be reading a book that considers questions and problems from the perspective of one field and then from the perspective of a 2nd field. However, that's less available than translated literature, because the...
Viral Guessing Game
by jefftk — 2023-12-24T13:10:11.917Z

Julia introduced me to

Metazooa, a
taxonomic guessing game. You're trying to figure out what animal the
computer has chosen. Each time you guess the computer will reveal the
lowest taxonomic clade that they both belong to. For example, if you
guessed platypus and the answer was humans then it would tell you they
were both mammals, since they have nothing else in common
taxonomically.

We played a bit and it was fun, and then I wanted to play a version
with human-infecting viruses. They have one for plants (Metaflora) but not for viruses,
so I made my own: Metahomoviria. The
code is on
github.

The first version had ~200
human-infecting viruses and wasn't very fun because you probably
haven't heard of most of them:

...
80941 Guaroa virus
273341 Ilesha virus
35514 Keystone virus
11580 Snowshoe hare virus
11577 La Crosse virus
273352 Macaua virus
348013 Madrid virus
35515 Melao virus
...

But I've trimmed it down to

42
reasonably well known viruses and it's better now:

11234 Measles
2560602 Mumps
11250 Respiratory syncytial virus (RSV)
1570291 Ebola
11292 Rabies
1980413 Hantavirus
11320 Flu A
...

It still isn't all that fun—what's the difference between
Picornavirales and Picornaviridae and is there really so far from
being a property Picornavirales have in common that it needed to be an acronym—but I like it. And I do
feel like I'm getting a better feel for which viruses are genetically
closer to each other.

If this sounds interesting, play with it here.

Comment via: facebook, mastodon
The Sugar Alignment Problem
by Adam Zerner — 2023-12-24T01:35:20.226Z

In a parallel world that has just recently discovered sugar.

Alice: Hey, so have you guys heard about that new thing they discovered to make food taste better?

Bob: Yeah I think so. It's powdered and white, right?

Alice: Yeah.

Carol: It's actually granulated, like tiny little pebbles. And it's called sugar.

Bob: Ah, ok.

Carol: It's similar to the discoveries made however many years back about umami. Y'know, that aspect of taste where we perceive things as savory and "meaty".

Alice: Oh, I remember that. I love it. I'm all about that nutritional yeast now. And the other "umami bombs" like soy sauce and miso paste.

Bob: Yeah, me too. And I'm excited to do the same thing with sugar. So far I've been adding it to my coffee every morning and it's been great. It gives it this fun little kick. It mellows out the bitterness of the coffee too, but in a nice way that complements it. And in a different way from milk. It's subtle. It's really nice.

Alice: Oh yeah, for sure. I like it in my tea.

Bob: It's also great on top of things that might be a little counterintuitive. Like on top of popcorn. It might seem weird next to that butter and salt, but I dunno, I think it's pretty good.

Carol: That makes sense. I hear that in some Asian countries they're sprinkling some on top of fried rice and other stir fries.

Alice: Huh, that sounds a little weird actually. That sugary flavor with fried rice?

Carol: Yeah maybe. I was watching a YouTube video about it and the guy was saying how it's intended to be more of a subtle addition. Like, you don't really taste the sugariness itself, but there's just something different about the fried rice that is hard to put your finger on, but that tastes really good.

Alice: Ah, I gotcha.

Bob: But I've also heard about how they're experimenting with adding lots of sugar to certain things. Like there's this thing called ice cream. It's basically yogurt but with tons of sugar in it, and much colder, almost frozen. I got to try it the other day and it's sooooooo good. I'm never having yogurt again.

Dan: Is that a good thing?

Alice: Oh. Hey Dan. Nice to see you too.

Dan: Sorry, I just walked in, but I heard about this sugar and ice cream thing, and I'm a little worried.

Alice: Worried? What do you mean, worried? I haven't heard of any negative effects. It's supposed to just make food taste better.

Dan: Have you done any research?

Alice: Research?

Bob: I think what Dan is...
A Crisper Explanation of Simulacrum Levels
by Thane Ruthenis — 2023-12-23T22:13:52.286Z

I've read the previous work on Simulacrum Levels, and I've seen people express some confusion regarding how they work. I'd had some of those confusions myself when I first encountered the concept, and I think they were caused by insufficiently crisp definitions.

The extant explanations didn't seem like they offered a proper bottom-up/fundamentals-first mechanism for how simulacrum levels come to exist. Why do they have the specific features and quirks that they have, and not any others? Why is the form that's being ascribed to them the inevitable form that they take, rather than arbitrary? Why can't Level 4 agents help but act psychopathic? Why is there no Level 5?

I'd eventually formed a novel-seeming model of how they work, and it now occurs to me that it may be useful for others as well (though I'd formed it years ago).

It aims to preserve all the important features of @Zvi's definitions while explicating them by fitting a proper gears-level mechanistic explanation to them. I think there are some marginal differences regarding where I draw the boundaries, but it should still essentially agree with Zvi's.

Groundwork

In some contexts, recursion levels become effectively indistinguishable past recursion level 3. Not exactly a new idea, but it's central to my model, so I'll include an example for completeness' sake.

Consider the case of cognition.

Cognition is thinking about external objects and processes. "This restaurant is too cramped."
Metacognition is building your model of your own thinking. What biases it might have, how to reason about object-level topics better. "I feel that this restaurant is too cramped because I dislike large groups of people."
Meta-metacognittion is analysing your model of yourself: whether you're inclined to embellish or cover up certain parts of your personality, etc. "I'm telling myself the story about disliking large groups of people because it feels like a more glamorous explanation for disliking this restaurant than the real one. I dislike it out of contrariness: there are many people here because it's popular, and I instinctively dislike things that are mainstream."
Meta-meta-metacognition would, then, be "thinking about your analyses of your self-centered biases". But that's just meta-metacognition again: analysing how you're inclined to see yourself. "I'm engaging in complicated thinking about the way I think about myself be

...
Hyperbolic Discounting and Pascal’s Mugging
by Andrew Keenan Richardson — 2023-12-23T21:55:27.091Z

Crossposted from my personal blog: https://mechanisticmind.com/hyperbolic-discounting-and-pascals-mugging/

TL;DR

Hyperbolic discounting, shown in 🟥, is an imperfect approximation for exponential discounting, shown in 🟦. It's commonly pointed out that this causes humans to overvalue nearterm rewards, but it's less commonly appreciated that this causes us to overvalue distant rewards as well. There's a lot of evidence that humans use hyperbolic discounting, and it helps explain why humans pursue unrealistic longterm goals.

What is Hyperbolic Discounting?

If you search the words “hyperbolic discounting”, Google tells you that it is “a psychological bias where people to prioritize immediate rewards and satisfaction over future rewards”. In fact, this is only half right. Hyperbolic discounting also causes people to overvalue distant rewards in addition to overvaluing near-term rewards.

Temporal discounting is basically a way of thinking about how to answer questions like this one:

Would you rather have $100 today, or $120 one year from now?

This is relatively easy to think about in monetary terms, and you probably know that the answer to this question has to do with interest rates, and the idea that taking $120 one year from now is equal to a 20% annual return, which is better than the 4-10% you might expect in the stock market, and also better than the 4.88% rate you can currently get for a 1-year treasury bond.

This model of calculating the interest rate is known as exponential discounting, and it’s basically represented by this equation:

value at time t=adt

This says that if something would be worth $a$ at time $t=0$, such as a $100 bill now, then its value drops off exponentially the more you need to wait for it.

So it turns out that humans and other animals don’t appear to be using this equation to estimate the value of future rewards. Instead, they use something called hyperbolic discounting, which can be represented by this equation:

value at time t=11+dt

This says that as a reward gets more distant, it’s value falls off according to an inverse.

How we Know

Scientists did experiments with undergraduates and $100 bills. Realistically, grant funding is hard to come by, so most of them used $10 bills, and researchers from lesser-known universities presumably had to get by with $1 bills and some change.

They also did experiments with monkeys and juice. Monke...
AISN #28: Center for AI Safety 2023 Year in Review
by Dan H — 2023-12-23T21:31:40.767Z
Link: https://newsletter.safe.ai/p/aisn-28-center-for-ai-safety-2023

Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.

Subscribe here to receive future versions.

As 2023 comes to a close, we want to thank you for your continued support for AI safety. This has been a big year for AI and for the Center for AI Safety. In this special-edition newsletter, we highlight some of our most important projects from the year. Thank you for being part of our community and our work.

Center for AI Safety’s 2023 Year in Review

The Center for AI Safety (CAIS) is on a mission to reduce societal-scale risks from AI. We believe this requires research and regulation. These both need to happen quickly (due to unknown timelines on AI progress) and in tandem (because either one is insufficient on its own). To achieve this, we pursue three pillars of work: research, field-building, and advocacy.

Research

CAIS conducts both technical and conceptual research on AI safety. We pursue multiple overlapping strategies which can be layered together to mitigate risk (“defense in depth”). Though no individual technique brings risk to zero, we hope to build layered defenses which reduce risks to a negligible level.

Here are a few highlights from our technical research in 2023:

LLM Attacks: bypassed the safety guardrails on GPT-4, Claude, Bard, and Llama 2, causing models to behave dangerously such as by outputting instructions for building a bomb. This work created the field of automatic adversarial attacks for LLMs. It was covered by the New York Times.
Representation Engineering: the first paper controlling the internals of models to cause models to lie or be honest. Experiments show these techniques can make AIs more truthful, power-averse, and ethical.
MACHIAVELLI Benchmark: evaluated the ability of AI agents to make ethical decisions. The benchmark provides 13 measures of ethical behavior on deception, rule-following, power-seeking, and utility. Published as an oral paper in ICML 2023.
DecodingTrust: showed that GPT-4 is more vulnerable to misleading targeted system prompts than other models. It won the outstanding paper award at NeurIPS 2023.
We also published research on rule-following for LLMs and unrestricted adversarial attacks.
Our researchers are helping multiple labs, such as OpenAI and Meta, red-team their models.

We also conducted conceptual research about AI safety:&...
"Inftoxicity" and other new words to describe malicious information and communication thereof
by Jáchym Fibír — 2023-12-23T18:15:50.369Z

Epistemic status: Proposal for a new terminology, to make an important concept easier and faster to communicate.

TL;DR: This is a proposition to start using new words such as "inftoxic" to describe information or data intentionally created or spread to cause harm. The post is not meant to present the best way to do this but to see if other people would consider this a useful endeavor.

Background

We are becoming more and more surrounded by synthetic data (such as text or images generated by LLMs) and as these mix together with real-world, human-generated data, there is great need for ways how to distinguish between the two, as well as generally make sense of all the informational mess.

While writing about the risks arising from generating synthetic data with potential malicious uses, I realized that I was lacking the proper vocabulary to effectively describe "a dataset that was intentionally constructed to mislead". I have tried looking for a suitable adjective to describe such dataset, and having failed, I tried to ask GPT4 to help me find and coin a new word for it.

The conversation I had with the LLM was quite engaging at the time, and with some reflection there still seemed to be some valuable takeaways, so I decided to do this little writeup about it. (Read the original conversation only at your own risk, haha!)

Inftoxicity and related terms

After some brainstorming from the GPT, I asked it to elaborate on the word "inftoxic" as an adjective that can be used to describe data, information sources, systems, actors, or actions that had been created with, or are acting in, a malicious intent to damage someone or something, or manipulate or mislead in a negative way.

This would be in contrast to "biased", which describes something or someone arriving at incorrect or misleading conclusions, yet not necessarily producing a negative outcome or being a result of malicious intent.

Below is a bunch of derivative words GPT4 was able to flesh out (slightly edited by me). I am intentionally leaving in all ten it generated, both to amuse the reader and inform of this interesting capability for creating novel words and definitions.

Inftoxic - adjective

Definition: Describes data, information sources, systems, actors, or actions that are created with or exhibit malicious intent to harm, manipulate, or mislead negatively. Distinguished from 'biased' by its de

...
AI's impact on biology research: Part I, today
by octopocta — 2023-12-23T16:29:18.056Z

[12/29/23 edited to correct math error]

I'm a biology PhD, and have been working in tech for a number of years. I want to show why I believe that biological research is the most near term, high value application of machine learning. This has profound implications for human health, industrial development, and the fate of the world.

In this article I explain the current discoveries that machine learning has enabled in biology. In the next article I will consider what this implies will happen in the near term without major improvements in AI, along with my speculations about how our expectations that underlie our regulatory and business norms will fail. Finally, my last article will examine the longer term possibilities for machine learning and biology, including crazy but plausible sci-fi speculation.

TL;DR

Biology is complex, and the potential space of biological solutions to chemical, environmental, and other challenges is incredibly large. Biological research generates huge, well labeled datasets at low cost. This is a perfect fit with current machine learning approaches. Humans without computational assistance have very limited ability to understand biological systems enough to simulate, manipulate, and generate them. However, machine learning is giving us tools to do all of the above. This means things that have been constrained by human limits such as drug discovery or protein structure are suddenly unconstrained, turning a paucity of results into a superabundance in one step.

Biology and data

Biological research has been using technology to collect vast datasets since the bioinformatics revolution of the 1990's. DNA sequencing costs have dropped by 5 orders of magnitude in 20 years ($100,000,000 dollars per human genome to $1000 dollars per genome)[1]. Microarrays allowed researchers to measure changes in mRNA expression in response to different experimental conditions across the entire genome of many species. High throughput cell sorting, robotic multi-well assays, proteomics chips, automated microscopy, and many more technologies generate petabytes of data.

As a result, biologists have been using computational tools to analyze and manipulate big datasets for over 30 years. Labs create, use, and share programs. Grad students are quick to adapt open source software, and lead researchers have been investing in powerful computational resources...