trentmkelly/LessWrong-43k huggingface dataset:
Thinking without priors?
by Q Home — 2022-08-02T09:17:45.622Z

My questions is strongly related to this post:

Bayes for Schizophrenics: Reasoning in Delusional Disorders by Scott Alexander

My question: can you balance the absence of priors with something else? Even if it's something irrational, such as wishful thinking or faith.

I'm interested in this because I want to now how I think. I have a suspicion that maybe something's wrong with my thinking. Maybe it's not Bayesian.

Note: I struggle with the simplest math, so sorry for possible mistakes and whatnot. It's part of the reason why I can't be sure if my thinking is Bayesian or not.

Librarian or farmer?

I encountered the following example in this video "Bayes theorem, the geometry of changing beliefs" by 3Blue1Brown.

Steve is very shy and withdrawn, invariably helpful but with very little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.

Which of the following do you find more likely: “Steve is a librarian” or “Steve is a farmer”? (from Kahneman and Tversky)

The trick is that some people don't consider how many there're librarians and farmers in general (the prior knowledge). Base rate fallacy

But I'm having trouble imagining to what irrational choices or behaviour it may lead. For example, if I want to find Steve in some real place, of course I'm going to care how many there're librarians and farmers.

So, I think in some situations a prior may seem missing, but pop right back up when it's really needed.

Note: when such questions or thought experiments get criticized for the lack of details, sometimes the criticism is answered with "you have to be able to think without details". But I don't think that some situation can exist without details (even as a fantasy).

Bank teller or feminist?

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

Which is more probable? "Linda is a bank teller" or "Linda is a bank teller and is active in the feminist movement"?

The infamous example. A more specific idea can't be more likely. Conjunction fallacy

But it's even harder for me to imagine the consequences of this mistake in terms of actions and decisions. Not surprising, since it's a logical impossibility.

And if I want to actually find Linda,...
Turbocharging
by CFAR!Duncan — 2022-08-02T00:01:23.148Z

Epistemic status: Mixed

The concepts underlying the Turbocharging model (such as classical and operant conditioning, neural nets, and distinctions between procedural and declarative knowl- edge) are all well-established and well-understood, and the "further resources" section for this class is one of the largest in the handbook. Before cofounding CFAR, formally synthesizing these and his own insights into a specific theory of learning and practice was Valentine Smith’s main area of research. What is presented below is a combination of early model-building and the results of iterated application; it’s essentially the first and last thirds of a formal theory, with some of the intermediate data collection and study as-yet undone. It has been useful to a large number of participants, and has not yet met with any strong disconfirming evidence.

Consider the following anecdotes:

A student in a mathematics class pays close attention as the teacher lectures, following each example problem and taking detailed notes, only to return home and discover that they aren’t able to make any headway at all on the homework problems.
A police officer disarms a hostile suspect in a tense situation, and then reflexively hands the weapon back to the suspect.
The WWII-era Soviet military trains dogs to seek out tanks and then straps bombs to them, intending to use the dogs to destroy German forces in the field, only to find that they consistently run toward Soviet tanks instead.
A French language student with three semesters of study and a high GPA overhears a native speaker in a supermarket and attempts to strike up a conversation, only to discover that they are unable to generate even simple novel sentences without pausing noticeably to think.

. . . this list could go on and on. There are endless examples in our common cultural narrative of reinforcement-learning-gone-wrong; just think of the pianist who can only play scales, the neural net that was intended to identify images of tanks but instead only distinguished cloudy days from sunny ones, or the fourth grader who reflexively says “I love you” to his classmate over the phone before hanging up in embarrassed silence.

There is a common pattern to these and many other failures, and recognizing it can both prevent you from ingraining the wrong habits and “turbocharge” your efforts to train the right ones.

A closer look: math education

In th...
Letter from leading Soviet Academicians to party and government leaders of the Soviet Union regarding signs of decline and structural problems of the economic-political system (1970)
by M. Y. Zuo — 2022-08-01T22:35:08.750Z

Crossposted from my blog: mzuo.ca

Some of the observations by these Soviet intellectuals may be of interest to LW readers...

"

This obscure letter offers a fascinating glimpse into the main concerns that preoccupied leading Soviet intellectuals in the beginning of the 1970s. It also represents an excellent example of the methods of communication employed by this group to express their concerns to leading party and state authorities of the USSR.

It also provides their perspective towards the changes in Soviet, Western, and World society and the interactions between the constellation of forces causing such changes.

Some interesting parallels can be drawn with modern day society and events.

As I am not a native Russian speaker, this is mostly a machine translation of the original with some minor adjustments and amendments based on limited knowledge of the leading party, states and academic bodies of the period, along with the scientific, technological, economic, and socio-cultural developments of the period. Therefore the translation may sound funny or bizarre in parts.

This is the only full translation into English, that I know of, so I welcome any comments or suggestions to improve the translation via my email: michaelyzuo AT gmail.com

Authors: Sakharov A.D., Turchin V.F., Medvedev R.A.
Title: Letter to party and government leaders

Soviet scientist academician A. D. SAKHAROV, as well as physicist V. F. TURCHIN and historian R. A. MEDVEDEV, wrote this letter and sent to the Central Committee of the CPSU L. I. BREZHNEV, to the Council of Ministers of the USSR A. N. KOSYGIN, and the Presidium of the Supreme Soviet of the USSR N. B. PODGORNY, letter of the following
content:

Dear Leonid Ilyich!

Dear Alexey Nikolaevich!

Dear Nikolai Viktorovich!

This letter discusses and develops a point of view, which can be summarized in the form of the following theses:

I. At present, it is imperative to carry out a number of activities aimed at further democratization of public life in the country. This need stems, in particular, from the close connection of the problem of [organizational] feasibility, scientific management methods with issues of freedom of information, transparency and competition. This need also stems from other domestic and foreign policy problems.

2. Democratization should contribute to the preservation and strengthening of the Soviet socialist system, soci

...
Technical AI Alignment Study Group
by Eric K — 2022-08-01T18:33:36.758Z

Is anyone interested in joining a study group for studying technical AI alignment (specifically the MIRI course list)?

I've been lurking on this forum for quite some time, and I'm trying to get deep into technical Alignment by following the books on the MIRI course list. Following advice from Nate Soares, I'm trying to "Pair Up" with someone to:

Have a Commitment mechanism to help us follow through the course list
Quickly address questions/misunderstandings on the materials
Form a tight feedback loop

Plans

Ideally, we're going to go through one book every month (or longer). We'll first decide which books to study, independently go through them, while in the study group we do:

Twice a day progress check-in (how many pages, number of Anki cards created/reviewed, practice questions solved, etc)
Ask/answer questions whenever they arise
Monthly meeting to discuss the materials in depth

Since people may prefer studying different books first, we can form a new group for each book. Also, because people may have different understanding levels / time commitments, it can be a monthly/bi-monthly thing depending on the group.

How to Join

If you're interested, please contact me through the DM. I'd love to get started this week!
Is there any writing about prompt engineering for humans?
by Alex Hollow — 2022-08-01T12:52:01.930Z

In much the same way that we need to prompt engineer language models to get them to handle our questions correctly, we know that humans respond differently to "This intervention will save 200 lives" and "This intervention will result in 400 deaths" when talking about a population of 600 people.

Is there any pre-existing writing that touches on this?
Meditation course claims 65% enlightenment rate: my review
by KatWoods — 2022-08-01T11:25:37.017Z

TL;DR

I eliminated my impostor syndrome and dramatically reduced my work-related anxiety. I did this in a way that I think can be replicated.
Different techniques work for different people. If you want to get the benefits of meditation, you should experiment widely, then drill down on the methods that work for you. Don’t just try the same technique for months or years and hope you’ll eventually “get it” or give up and say meditation doesn’t work for you. Explore then exploit.
Loving-kindness meditation is underrated and should be the main-course meditation for a lot of people.

This 55-minute video

is the 80/20 of the course. If you like it, you will probably like the rest of the course.
I recommend the Finder’s Course for most people. If you follow the instructions, you will very likely become happier. If you prefer and are good at self-directed learning, you can do your own self-directed course and get similar benefits.

What makes the Finder’s Course different: methodology

Applying science to meditation isn’t unique to Jeffery Martin’s course. Fortunately for the world, there’s a whole movement around this. That being said, I haven’t heard of anything that seems more likely to figure out how to actually achieve enlightenment (or fundamental well-being (FWB) as he calls it, which I prefer).

Most science I know of is doing things like putting meditators in brain scans and seeing if anything is different from regular brains, or running RCTs to see if meditation makes you happier. This is foundational and important to do. However, it’s very black box thinking and doesn’t give you any gears-level understanding of how to achieve fundamental well-being.

Meditation classes usually teach a variety of different techniques. Which techniques are causing the change? Most studies focus on averages, which ignores the thing we’re most interested in - those outliers who don’t just start feeling less stressed but have eliminated suffering. Who are living in states of profound bliss and serenity. How does that show up on a psychological item asking “On a scale of 1 to 10, how satisfied are you with your life?”?

The Finder’s Course on the other hand clearly followed a methodology that was truly trying to solve the problem. The way he did this was to find over 1,000 people saying that they had achieved fundamental well-being, and he went an...
Polaris, Five-Second Versions, and Thought Lengths
by CFAR!Duncan — 2022-08-01T07:14:16.429Z

Author's note: During CFAR's 4.5d workshops, concepts that had been formalized as "techniques," and which could be described as algorithms and practiced in isolation, generally received 60+ minute sessions. Important concepts which did not have direct practical application, or which had not been fully pinned down, were often instead taught as 20-minute "flash classes." The idea was that some things are well worth planting as seeds, even if there was not room in the workshop to water and grow them. There were some 30 or 40 flash classes taught at various workshops over the years; the most important dozen or so make up the next few entries in this sequence.

Polaris

Imagine the following three dichotomies:

A high school student mechanically following the quadratic formula, step by step, versus a mathematician who has a deep and nuanced understanding of what the quadratic formula is doing, and uses it because it’s what obviously makes sense
A novice dancer working on memorizing the specific steps of a particular dance, versus a novice who lets the music flow through them and tries to capture the spirit
A language student working on memorizing the rules of grammar and conjugation, versus one who gesticulates abundantly and patches together lots of little idioms and bits of vocabulary to get their points across

By now, you should have a set of concepts that help you describe the common threads between these three stories. You can point at goal factoring and turbocharging, and recognize ways in which the first person in each example is sort of missing the point. Those first three people, as described, are following the rules sort of just because—they’re doing what they’re supposed to do, because they’re supposed to do it, without ever pausing to ask who’s doing the supposing, and why. The latter three, on the other hand, are moved by the essence of the thing, and to the extent that they’re following a script, it’s because they see it as a useful tool, not that they feel constrained by it.

How does this apply to a rationality workshop?

Imagine you’re tutoring someone in one of the techniques—say, TAPs—and they interrupt to ask “Wait, what was step three? I can’t remember what came next,” and you realize that you don’t remember step three, either. What do you do?

You could give up, and just leave them with an incomplete version of the technique.

You cou...
A Word is Worth 1,000 Pictures
by Kully — 2022-08-01T04:08:28.199Z

Uber or Walk

Something interesting happened last night. It was approaching 12:30am, and a few hours of house-warming chats had petered down to a natural silence. With empty beer cans sprawled across the coffee table and kitchen counter of the brand new apartment, the three of us decided it was time to wrap up the night. William, sunken into the leather couch facing the open concept living room, pulled out his phone. “I’m gonna call an Uber”, he said.

To this I suggested, “Why don’t you just walk it? You live on our street.”

But in sudden reflex Callum, my roommate standing only a few feet away, chimed in: “Nah, grab an Uber. Our apartments are 350 units away.”

Life of a Salesman

Callum is a salesman. Quick to his feet when his work phone rings with work emergencies; he’s a problem solver who applies tactics tactfully. A gregarious guy, and among his several skills is the ability to persuade with his words.

Let’s assume that Callum’s response originated from a knee-jerk emotion that he decided to express with his words: a car ride is gonna be easier and warmer than a cold walk through the streets of downtown Toronto in early March. But this begs the a followup question: What if his brain automatically composed a reason to explain the emotional decision, and he verbalized it afterwards?

When he said 350 units, it felt like he was selling this idea to me, using words and body language to communicate an idea into my mind. This is something we all do by virtue of being human, but surely the persuasion factor is heightened in Callum-like sales folk.

Pictures in our Mind's Eye

I’m a visual person. Pictures come naturally to me, as I tend to think in images.

And even if it wasn’t my roommate’s intention to explicitly trigger any pictures into my head, it happened. The text "350" plopped into my visual field. And since our brain is a neural network of connections, one idea can trigger another, and another, and another. Only a few moments later, as I was looking up and into the top-left quadrant of my visual field, hyper aware at this point of my mind's eye, I noticed a long line of dots, one after the other, in close proximity.

Mental Picture No.1

And as the image faded into the ether, as mental pictures do, my mind naturally went elsewhere. Will said that he was going to call the Uber, and my emotions kicked in.

“It’s healthier to walk than to take an Uber”, I ...
On akrasia: starting at the bottom
by seecrow — 2022-08-01T04:08:12.187Z

[ Why theorizing about your addictions fails to help you combat them: an addict-thinker's version of "just do it." ]

For a few years, I've struggled to overcome several addictions. I'll list them here briefly: (nicotine, nail-biting, overeating, physical inactivity, and sleeping late). I call them addictions because it's the simplest word, but they could also be called akrasia or maladaptive behaviors.

Whatever we call them, they are just behaviors which (a) I would like to stop doing because they negatively impact my body and (b) for whatever reason, are very difficult to stop doing. For years, I've been obsessed with beating these addictions. I've thrown everything I can at them, but they won't budge. Why is that?

I claim that the reason I haven't been able to make any headway is that I've been approaching the problem of addiction (and more broadly, the problem of my life) from the top down. By switching to a 'bottom-up' approach, these addictions (and my life) have become much more manageable.

For as long as I can remember, whenever I've run into a problem, I've always found that there is some way I can mentally or abstractly frame the issue that makes it resolvable. There has always been some near-correct understanding I can achieve that makes solving the problem feel easy and natural. The exception is, of course, anything that resembles an addiction.

For the past four years, I've been trying to apply this 'understanding-first' way of solving problems to my addictions. I take space and allow myself to think creatively and rationally until I arrive at some crystallized strategy. This can be a simple strategy, like "I just need to take it one day at a time!" or something a bit more complex, like "Negative intention (trying not to do something) goes nowhere. Rather than focusing on what I won't do, I'm going to focus on what positive thing I am doing. By doing this, I'll escape my addictions indirectly." And so on... I've probably tried on hundreds of 'understandings' to see if they give me any power against addiction - so far they haven't. I come up with what seems like a good plan, then it fails.

When it comes to addiction/akrasia, I argue that understanding cannot come first. For instance, I always hate knowing I'm about to exercise, and I dislike the feeling of exercise itself. I can't access the felt-sense understanding of why I shoul...
How likely do you think worse-than-extinction type fates to be?
by span1 — 2022-08-01T04:08:06.293Z

The risks of unaligned-AGI are usually couched in terms of existential risk, whereby the outcome is explicitly or implicitly human extinction. However, given the nature of what an AGI would be able to do, it seems though there are plausible scenarios worse than human extinction. Just as on an individual level it is possible to imagine fates we would prefer death to (e.g. prolonged terminal illness or imprisonment), it is also possible to expand this to humanity as a whole, where we end up in hell-type fates. This could arise from some technical error, wilful design by the initial creators (say religious extremists) or some other unforeseen mishap.

How prominently do these concerns feature for anyone? How likely do you think worse-than-extinction scenarios to be?
Abstraction sacrifices causal clarity
by Marv K — 2022-07-31T19:24:01.783Z

In my previous post I listed some considerations for a theory of narratives. The smallest building block of narratives are abstractions over empirically observed things and events; that is the ontology of the language that the narrative uses. In this post I want to start laying out a framework that allows showing how, although initially, one may have observational access to a graph with clear causality between events, by abstracting over its vertices and edges naively (or as best you can?), you lose this causal clarity and are left with correlation. The end goal is to end up with some considerations on how to abstract well while preserving causal clarity optimally.

My observational model liberally affords you observations of the universe in the form of a directed acyclic graph, which consists of perceptions in two embedding spaces: vertices, which we'll call entities, and directed edges, which we'll call actions. Each of these observation sets is strictly partially ordered (non-reflexive, asymmetric, transitive) by time. Both entities and actions are encodings of your sensory pre-processing into some perception space with some topology that allows for grouping/clustering/classification. Note that "entities" in this model do not yet persist across time, but are mere instantaneous observations that may at most correspond to more permanent entity entries that you might keep track of in some separate dynamical model of the universe.

The above observational structure is supposed to directly represent your best possible model of causation between things; the action edges are observed causations. This model is already limited in predicting the world in three major ways:

Entities and actions are already (perceptual preprocessing) abstractions over incredibly complex systems, such as a human, with layers of theoretically analyzable causal interactions going down multiple orders of magnitude and academic branches down to the quantum level. Thus even the perceived entities and actions are already abstracted intelligible patterns in some spatiotemporal substrate, so there is already a loss of causal clarity when going from the true (quantum?) lowest level causal interactions to interactions at any reasonably perceptible level of abstraction/grouping above, both spatially and temporally. I do not directly take this loss into account in this post, and want to show that

...
Time-logging programs and/or spreadsheets (2022)
by mikbp — 2022-07-31T18:18:03.030Z

I am trying to decide how to track the time I spend in different activities/projects and I found this post by Torello. The answers are good but it is a very old post (9y), so I think it would be good to see how the landscape has changed in this time.

If you use any kind of time-logging, please describe the process and link to the software, spreadsheet or other system you use (if possible). It would also be interesting if you comment on whether you time-log only for work or also for non-work related activities.