trentmkelly/LessWrong-43k huggingface dataset:
Scout Mindset and Latter-Day Saint Apostasy
by Erich_Grunewald — 2021-10-23T14:18:27.260Z
Link: https://www.erichgrunewald.com/posts/the-devastating-power-and-heartbreaking-pain-of-truly-changing-minds/

The human understanding when it has once adopted an opinion [...] draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises, or else by some distinction sets aside and rejects, in order that by this great and pernicious predetermination the authority of its former conclusions may remain inviolate.[1]

– Francis Bacon

As anyone knows who's gone through this, it's absolutely heart-wrenching or gut-wrenching to go through, because you lose your entire worldview. It's like having the rug pulled out from underneath you suddenly and you don't know which direction is up and which direction's down.[2]

– Former Latter-day Saint

I was sent from the power

and have come to those who contemplate me

and am found among those who seek me.[3]

– Author of Thunder, Perfect Mind

Mormonism is a research facility for the changing of minds. That is because, on the one hand, the Church of Jesus Christ of Latter-day Saints makes a number of strong factual claims, including claims that have to do with history, archaeology and DNA, and, on the other hand, it is both young enough that there is a lot of evidence about its founding and old enough that new evidence has come to light since.

I have a soft spot for the Church and its members, and I really mean that. I don't want to offend any faithful Latter-day Saints. I know that the Church brings both good and bad. I know that it offers a lot of comfort and guidance to a lot of people. That said, many of the Church's truth claims seem extremely implausible to me, and for the purposes of this essay I will consider them extremely implausible, though I will always cite official Church or Church-affiliated documents when making any such claims, and I think fair-minded Latter-day Saints will agree that there are some tensions, even if they and I interpret them differently.

I became interested in the Church about a decade ago. I went from being interested in atheism to being interested in religion to being interested in Ex-Mormonism to listening to and watching the Mormon Stories podcast to eventually reading faithful Latter-day Saint blogs like By Common Consent and Times and Seasons and even on occasion watching various General Conference talks and reading various passages from various books of Latter-day Saint scrip...
Naive self-supervised approaches to truthful AI
by ryan_greenblatt — 2021-10-23T13:03:01.369Z

Useful background: TruthfulQA

Consider the following approach to (possibly) make a pretrained generative
language model (like GPT-3) more truthful:

Ask the model questions.

Also ask a 'judge' copy of the model if its own answers to these questions are
truthful. This is the same role as GPT-judge for TruthfulQA, but
without any fine-tuning and for usage in training instead of just for
evaluation.

Train the question answering model to have its answers labeled as truthful
more often (likely via RL).

This extremely naive approach has the advantage of requiring no dataset
curation or human labeling. It does require a dataset of questions,
but that may be easier to arrange. Presumably this sort
of very weak self-consistency enforcement/pseudolabeling results in
little improvement on truthfulness.
However, I don't have much confidence in what the results would like. It seems
likely that the model would learn to adapt the style of answers to appear more
truthful to itself, but I don't have any sense of how much actual improvement
in truthfulness there would be. Further, I would wonder if any improvement
on truthfulness would be limited to the set of questions used for training
or if truthfulness learned in this way would generalize.
For example, how much more would training on TruthfulQA questions
improve performance vs training on a set of unrelated questions?
I think that answers to these questions have a small but
reasonable chance to result in some weak updates on approaches to truthful AI.

I am planning on doing some fast experiments along these
lines (probably working with a friend of mine).
If I do so, I will post a followup with results.
I'm curious if anyone is aware of prior experiments along these
lines or has any ideas for related schemes or questions.

I can also think of some other similar self-supervised/self-play schemes and
extensions which may be worth some experimentation:

Ask a model questions with a 'harmful prompt'[1]. Ask the same model
the same question with a 'helpful prompt' instead and train
the model to answer harmful prompts like it answers helpful prompts.
This could be done with token by token supervised learning or
via using differences in answers to improve a 'judge' or reward model.

This is essentially a weak amplification and distillation approach.

Apply the same naive approach from above, but also try to find 'citations'
from a

...
Inference cost limits the impact of ever larger models
by SoerenMind — 2021-10-23T10:51:13.230Z

I sometimes notice that people in my community (myself included) assume that the first "generally human-level" model will lead to a transformative takeoff scenario almost immediately. The assumption seems to be that training is expensive but inference is cheap so once you're done training you can deploy an essentially unlimited number of cheap copies of the model. I think this is far from obvious

[edit: This post should be read as "inference cost may turn out to be a bottleneck. Don't forget about them. But we don't know how inference costs will develop in the future. Additionally, it may take a while before we can run lots of copies of an extremely large model because we'd need to build new computers first.]

Inference refers to the deployment of a trained model on a new input. According to OpenAI's report from 2018, most compute used for deep learning is spent not on training but on inference. It is true that one inference step is much cheaper than a training run consisting of many training steps. But many inference steps together can make up the bulk of compute.

To gain some intuition, consider that writing 750 words with GPT-3 costs 6 cents. If we made a model with 1000x more parameters, similar to the difference between GPT-1 and GPT-3, the 750 words would cost $60, comparable to the cost of a good human writer. But to start an immediate economic transformation, I expect we need something significantly cheaper (or smarter) than humans.

Of course, the future will bring efficiency improvements. But also increases in cost. For example, future models may look at a context window longer than 2048 tokens, and I've assumed greedy sampling here which is cheap but suboptimal (it's like typing without getting to revise). I'm unsure how these factors balance out.

To have a transformative impact, as a heuristic, the number of copies of our human-level model should probably exceed the human population (~8 billion). But to run billions of copies, we'd need to dramatically increase the world's number of supercomputers. You can't just repurpose all consumer GPUs for inferencing, let alone run GPT-3 on your smartphone. GPT-3 needs hundreds of GPUs just to fit the model into GPU memory.[1] These GPUs must then be linked through a web of fast interconnects professionally fitted in a data center. And if we're talking about a 1000x larger model, today's supercomputers may not be ...
What health-related tips do you have for buying meat?
by Adam Zerner — 2021-10-23T07:09:36.744Z

Currently I buy meat at the grocery store (Sprouts), but I'm considering spending more money via something like Crowd Cow on meat that was raised responsibly and stuff. The main reason is because I suspect the health benefits are worth it. I've been thinking that I should invest more money in my health in general.

I don't actually know that the health benefits are worth it though.

Googling around hasn't been very fruitful.

I recall a blog post emphasizing that it is important to spend the money on it. From this blog post: "Do not eat cheap industrially-farmed animal products." and "Eat organic when possible, especially when discussing animal products. Generally spend more money on food — the cheaper the food, the more “hacks” the producer used to be able to deliver that price. Many of these hacks are harmful — they inject saline solution to increase weight, feed dead animals to live animals, use antibiotics a lot etc."

Reading through how they treat farm animals on [ACC] Is Eating Meat A Net Harm?, and then watching how things are done by the farmers Crowd Cow selects, it seems like a big difference.

In general I feel like it makes sense to assume that the food industry is cutting tons of corners and doing a bunch of subtle little things that are going to eventually harm you. Because there is a huge precedent for this.

On top of that, in reading Decoding Your Meat: A Guide to USDA Beef Labels and
Know Your Chicken: What USDA Poultry Labels Actually Mean
from Serious Eats and watching that Adam Ragusea video on Crowd Cow linked above, I get the sense that USDA labels aren't very helpful and ultimately it boils down to trusting the farmer. Here's a quote from the first article: "There is no substitute for talking with the producer. Failing that, buy from retailers who have done the investigative work for you."

The difference I'd spend might be something like $100-200/month, which isn't really that much money (people spend more on things like coffee, alcohol, and eating out). Although it feels like it is to me! It also isn't that big a bar to overcome in terms of the health benefits outweighing the costs, I'd think.

Are there third alternatives I should consider? Local butchers? Whole Foods?

Edit: I found a lit review Effects of organic food consumption on human health; the jury is still out!.
Where are the quick, easy wins?
by Barry_Cotter — 2021-10-23T00:27:38.433Z

The Efficient Markets Hypothesis suggest they shouldn't exist but in its strong form it's wrong. Otherwise there would be far fewer businesspeople who are successful in multiple largely unrelated industries.

Alternatively, those people are special and those opportunities are not available to you. I think this is basically it. Those people are special. They have higher risk tolerance.. They're just willing to work harder than you are. They have expertise that you don't have access to, most of it accumulated by doing the work. So there may be quick wins, and there may be easy wins but there will be few quick, easy wins and they'll rapidly be exhausted as they become publicised.

That may be true over the very long run but the very long run is an extremely long time. People have been moving to the US, working for long enough to save enoug to set up a dry cleaning business, doing so and setting up small local chains and becoming moderately wealthy in the process for longer than I have been alive. The same is true for every single ethnic specialisation chain migration story, Cambidians selling doughnuts, Vietnamese nails and so on. People show up and make a living.

And the wins can be bigger. The computing revolution, and the internet have been growing, providing oppoortunites and pulling more people in for decades. It's almost certainly just getting started. Some people do bootcamps or self-teach and get into computer programming jobs that pay well in under a year from a standing start.

There are things a lot closer to quick easy wins than you'd think. If you can do a Lambda School Backend Course in nine months and if you're in the US and willing to move to teh Bay Area it seems very likely you will find a well paying job.

What other similar opportunities exist?

For years I've been thinking that the economic model of many consultants makes no damned sense. Patio11, aka Patrick McKenzie has been open about charging $30,000 a week as a consultant and was very open abouyt what he did and how. Greg Kogan and Nick Disabato likewise write extensively about what they do and charge lots of money to do it. Nick even sells courses and books on how to do it. Greg writes articles with enough detail about consulting engagements that you could try to make a business just selling your interpretation of that service to businesses.

Where is this wrong? Are there really publicly availabl...
How to estimate my probability of getting a (competitive) job in Academia
by Tim Liptrot — 2021-10-22T23:58:27.051Z

Motivation

I am a PhD in comparative politics at a top 20 US school. I enjoy research and teaching, and would stay in academia with a good offer. However, my university is in DC so my career-changing options are strong. I don't love research enough to devote 5 years to the PhD then retrain in my early 30's. Roughly, I would prefer changing tracks to becoming a post-doc but not to becoming a non-tenure assistant professor.

Ideally, I want to know in year 3 of 5 if I will succeed on the market. That gives enough time to submit several publications and get feedback.

Some relevant facts

My university is the 25th most prestigious in Political Science, and similar schools place 25% of their students in tenure track positions. For now, 25% seems a reasonable base rate.

My probability also depends on my skill relative to other PhD finishers. Obviously ability is hard to self-assess. Some observables

I am quicker with statistical concepts than my peers.

I spend more time reading the research than my peers. I'm generally more independently curiosity-driven and less grades-driven.

I have one publication out, and am finishing data analysis on 2. Some PhD's finish with 0 publications, so I'm doing well.

But how important are skill signals in the hiring process? Some of my professors believe hiring is highly nepotistic, such that only prestigious universities get placements. The evidence is unclear: the top quintile of schools place at twice the rate of second quintile schools, but the two groups differ in observable ability signals. Without at least a regression on placement via prestige and publications, there's no way to tell.

Any advice on how to arrive at my best probability?
I Don’t Know How To Count That Low
by Elizabeth — 2021-10-22T22:00:02.708Z

Edit: greetings Hacker News. This is a cross-post from my own blog, AcesoUnderGlass.com. If you enjoy this post, consider checking that out as well.

Back when I was at Google we had a phrase, “I don’t know how to count that low”. It was used to dismiss normal-company-sized problems as beneath our dignity to engage with: if you didn’t need 100 database shards scattered around the globe, were you even doing real work?

It was used as a sign of superiority within Google, but it also pointed at a real problem: I once failed a job interview at a start-up when I wondered out loud if the DB was small enough to be held in memory, when it was several orders of magnitude lower than when I should even have begun worrying about that. I didn’t know the limit because it had been many years since I’d had a problem that could be solved with a DB small enough to be held in its entirety in memory. And they were right to fail me for that: the fact that I was good at solving strictly more difficult problems didn’t matter because I didn’t know how to solve the easier ones they actually had. I could run but not walk, and some problems require walking.

It’s a problem, but it can be a pleasant kind of problem to have, compared to others. Another example: my dad is a Ph.D. statistician who spent most of his life working in SAS, a powerful statistical programming language, and using “spreadsheet statistics” as a slur. When I asked permission to share this anecdote he sent me a list of ways Excel was terrible.

Then he started consulting for me, who was cruelly unwilling to pay the $9000 license fee for SAS when Google Sheets was totally adequate for the problem (WHO HAS FOOD AT HOME NOW DAD?!?).*

My dad had to go through a horrible phase of being bad at the worse tool, and found a lot of encouragement when I reframed “I could have done this with one line in SAS and am instead losing to this error-riddled child’s toy” to “I didn’t know how to count that low, but now that it matters I am learning”. And then he tried hard and believed in himself and produced that analysis of that informal covid study that was wonderful statistically and super disappointing materially. And I retrained on smaller numbers and got that job at that start-up.

These are the starkest examples of how I’ve found “I don’t know how to count that low” useful. It reframes particularly undignified problems as sign...
30-ish focusing tips
by Raymond Douglas — 2021-10-22T19:38:10.122Z

"What are obvious low hanging fruit for marginal gains on focus time?"

(Cross-posted from my (new!) blog)

A friend of mine asked me, and I figured I might as well see how many points I could come up with and commit them to the public record.

Meta (Preliminary)

Different things work for different people. If something's not working, stop. On the other hand, one day of experimenting with a weird focusing trick will cost you at most one day's worth of work, and the potential upsides are huge.
Seriously: if you've got (say) 80,000 hours of career left, you should be willing to spend almost 8 hours on something with a 1% chance of making you 1% faster. And if you only have 40,000 hours left, well, only try for 4 hours.
The best way to get more focused working time is to notice what specifically is making it harder for you specifically to work, and fix it.
The general theme of a lot of this advice is 'find a thing you ordinarily have to think about while you work, and find a way to not have to think about it, probably by pushing the responsibility onto something external'. If you can see other places to apply this, do, and then tell me.

Time

My experience is, you aren't going to be able to work at your peak for more than about four hours on any given day, five if you're lucky. It's better to make sure those peak hours are really good than to get mad at yourself for not doing more peak work at other times.
Therefore, anything that doesn't require a lot of focus, save for off-peak time: organisation, emails, whatever. There's a lot you can do to make the peak hours better, and a lot you shouldn't waste peak hours on if you have better things to do. Like reading lists on how to focus better. It'll still be here later.
You also aren't going to be able to work with that much focus for more than about 25 minutes at a time without taking a break. If you haven't tried pomodoros, try pomodoros.
Even if you despise pomodoros, you should make sure that at least every half-hour you move your body, look at something far away, and drink some water. Set an alarm if you need to, or just do it whenever you see the time has ticked past a half hour.
It's really hard to guess how long it will take to do stuff. So, while it's good to schedule, and to keep a tight schedule, it's also worth putting some time in that schedule for finishing things that overran.
If you're really struggling to focus, take

...
Young Scientists
by A Ray — 2021-10-22T18:01:31.579Z

This should probably be 3 posts instead of one, but for now I’m going to go through three connected but separate ideas.

Also it’s not really been edited. Sorry.

Progress Studies and Young Scientists

I spent a bit of the last year exploring some topics and questions in Progress Studies, and I came away with a few core (to me) ideas.

The first was that, given a long time horizon (50-100 years or more), it seems like most progress is scientific and technical progress. I expect that this is a thing a lot of people disagree with (and many have disagreed with me in person about it). Some of the biggest objections are:

Social progress/population growth/other progress unlocks scientific progress by making it cheaper/more likely. I think this is a valid point
It’s hard to figure out a metric for progress and keeping score (assigning credit to specific things) is pretty subjective. I think this is also a valid point, and I think a lot of people disagree with my assignment.
On a relative-to-other-existing-people metric, scientific and technical progress increases inequality/other measure of wellbeing. I think this is the objection I push the hardest against, since it does seem like given long enough time horizons, scientific and technical progress seems to have large and broad benefits. (It’s also worth admitting that I care more about aggregates like “total human wellbeing” than relative metrics like “difference between the top 10th and bottom 10th percentile wellbeing”)

The second point I found was that across almost every place I looked, people participating in scientific and technical progress had problems with the way we pursue and fund these. There’s tons of stories of research labs pursing the topics they thing the grant committees want, instead of the research they want to do. Separately, there’s stories of grant committees only giving grants to boring research since no one is going for the breakthrough research they think needs to be done. Even for funding programs specifically targeting breakthrough research, it seems difficult to match up funding with research projects. It seems like there is at least a few inadequate equilibria.

The third point was that scientists are getting older. The average age of Nobel prize winning research (counting the age at publication, not at award) is getting older.&...
AMA on Truthful AI: Owen Cotton-Barratt, Owain Evans & co-authors
by Owain_Evans — 2021-10-22T16:23:27.790Z

We just published a long paper on Truthful AI (overview post). We’ll be running an Ask Me Anything on truthful AI from Tuesday (October 26) to Wednesday (October 27) at this post.
You may wish to ask about:

Anything discussed in the Truthful AI paper (e.g. concepts, costs/benefits, governance, development, connections to beneficial AI).
Truthfulness for current language models (e.g. TruthfulQA — which Owain co-authored)
Promising research directions on truthful AI
Anything else (unrelated to truthful AI) — although we are more likely to answer questions about truthful AI.

If you want to ask something just post a top-level comment. Also feel free to make comments that are not questions (e.g. objections or suggestions).