trentmkelly/LessWrong-43k huggingface dataset:
The Power of Annealing
by meanderingmoose — 2020-12-14T11:02:44.698Z

Crossposted from mybrainsthoughts.com

Metals are a unique form of matter, especially with regards to their behavior under heating and cooling. While metals are described by the material they’re made of (e.g. copper or iron), their properties are determined by their arrangement of atoms, and this arrangement can vary significantly.

The arrangement is semi-permanent, but can be manipulated through certain heating and cooling techniques. One especially interesting technique is “annealing” (if you’re interested in this technique or the many others, this video provides a great introduction), where the metal is first heated and then slowly cooled over time. Through this process, the metal becomes less hard and more ductile, as the heating provides the necessary energy for the form to shift, and the slow cooling provides the time and energy levels needed to form larger grains (see below).

Computer programmers have taken note of these particular properties and ported them into the field, resulting in an algorithm known as “simulated annealing”. The algorithm is used to find an approximate global optimum for a specified problem, and works, like metallurgic annealing, by gradually reducing the “temperature” (for the algorithm, “temperature” determines the likelihood of moving in the direction of a worse solution). We can better understand how the algorithm works by graphically representing the solution space for a problem (the below picture represents a 1-dimensional space, but we can extend the idea to arbitrarily complex N-dimensional spaces).

The algorithm starts from a random input value (a random place on the above line), then at each iteration selects a random nearby point (the definition of “nearby” changes with the temperature; at higher temperatures, further points can be chosen). If the randomly selected point is higher than the current point, the algorithm “moves” to that point; if it is lower, the algorithm may or may not “move”, depending on the temperature (the algorithm is more likely to “move” at higher temperatures). This strategy converges to an approximate global optimum – one way to think about it is that the initial runs (at high temperature) identify the right “hill” to “climb” by searching the entire solution space, and the later runs (at low temperature) “climb” to the top of that “hill”. The below image (from Wikipedia) shows an exampl...
Hermione Granger and Newcomb's Paradox
by lsusr — 2020-12-14T05:27:11.667Z

Omega hovered before Hermione Granger. Between them rested two boxes labelled "A" and "B".

"Do the boxes have any special powers beyond modifying my Muggle bank balance?" Hermione asked.

"No," Omega said.

"Where do you get the money?" she asked.

"What do you mean?" Omega said with mock incomprehension.

"Fiat currency is a zero-sum agreement between human beings. It is not a physical thing. You cannot conjure one half of a mathematical equation out of nothingness. For you to have a million dollars, a human must have given it to you," Hermione said.

Omega opened an orifice. Its tentacle withdrew an infinite deck of transparent envelopes. Its claw extracted an infinite deck of opaque envelopes. The top envelope of each stack was labelled "#1".

"If you open the transparent envelope then one pound will be deducted from your Muggle bank account and the opaque envelope will have contained nothing. If you never open the transparent envelope then the opaque envelope will have always contained a debt of one pound. You can ensure the opaque envelope contains a debt of one pound by burning the transparent envelope. Do you choose these envelopes over either box?" asked Omega.

"You already know my answer," Hermione said.

"I derive pleasure from you saying it anyway," said Omega, "As do you."

"I choose the envelopes," Hermione said.

"Granted," Omega flew away.

"Wait!" Hermione shouted to the stars, "What did my Box B contain?"

Money

Hermione opened transparent envelope #1. It contained a note stating that "one pound has been deducted from your bank account." Then she opened opaque envelope #1 which contained nothing.

Hermione opened opaque envelope #2. It contained a note stating "one pound has been deducted from your bank account". She burned transparent envelope #2 without opening it.

Hermione flipped a coin eight times. She got THTTHTTH. She opened transparent envelopes #3,5,6,8,9. £5 pounds were deducted from her Muggle account. She opened opaque envelopes #4,7,10. £3 pounds were deducted from her Muggle account. Hermione opened the empty opaque envelopes #3,5,6,8,9. Hermione burned transparent envelopes #4,7,10.

Hermione flooed to Gringotts the next morning where she confirmed the transactions with her account manager Chauna. Together they identified the highest-vega security that could be traded on time-horizons of one hour. Hermione gave opaque envelopes #11-#20 to Chauna and inst...
All GPT skills are translation
by p.b. — 2020-12-13T20:06:00.399Z

After having my mind blown by the GPT-2 codes python demo, I realized that all „skills“ that GPT-2 and especially GPT-3 possess fall into a certain category.

Autoregressive language models like the GPT models can write text by recursively predicting the next token. The exciting development of the last two years is the discovery that implicit in this functionality is a variety of skills, if the language model is good enough. So really good language models do not just write really good text. They can be queried for facts, they can be set up to be a conversation partner, they can solve SAT-like analogy tasks, they can do arithmetic, they can translate between languages and they can even generate (often) functioning code from a verbal description.

Why does a model that just predicts the very next token develop the ability to translate between languages? Predicting the next token, generally, is very hard. But there are occasions where it becomes a lot easier. These are basically loss-function sinks, where suddenly the language model can learn to reduce the loss much more than is generally possible. Translation is a very intuitive example, because the original text very strongly constrains the possible continuations of the translation.

But, on second thought, really all the skills mentioned above fall into this category! Except maybe for holding a conversation, they all translate information from one modality into another. Questions into answers, comment strings into code, expression into result, French in English, etc. And even in a conversation each remark constrains the possible replies. So what seems to be happening with more powerful models is that they increasingly detect these loss function sinks and exploit them for what they are worth.

Along these line, we can divide what GPT models can learn to do into two very different camps:

On the one hand they can learn a new data distribution from scratch. This is what the training objective is ostensibly about and it can be very hard. When GPT-2 was published I was briefly intrigued with the possibility to train it on chess games, but when this idea was realized by at least two different people, the result was quite underwhelming. I don’t doubt that GPT-2 would in principle be able to infer the board state from the moves and learn to predict moves accurately, but this would take a lot of c...
Clarifying Factored Cognition
by Rafael Harth — 2020-12-13T20:02:38.100Z

This post is sort of an intermediate between parts 1 and 2 of the sequence. It makes three points that I think people tend to get wrong.

1. Factored Cognition is about reducing hard problems to human judgment to achieve outer alignment.

It's possible to lose sight of why Factored Cognition is employed in the first place. In particular, it's not as a way to boost capability: while the amplification step in IDA is implemented via Factored Cognition, the purpose of this is alignment: IDA would be more straight-forward (and more similar to AlphaZero) if each Ak were trained to approximate [Ak−1 thinking for longer] rather than [Haccess⟶Ak−1].

Recall from post #-2 that I've framed the entire problem of AI risk as 'once systems become too capable, it gets difficult for a human to provide a training signal or training data'. There are many approaches to get aligned systems anyway: there's ambitious value learning, there's trying to develop an new framework, there's impact measures, there's norm following, there's avoiding agents altogether, et cetera et cetera. But another option is to reduce the difficult of providing the training signal to human judgment (and do so for each instance separately), which is what IDA and Debate are trying to do. This is what Factored Cognition is used for. In a previous version of this post, I've dubbed the approach 'narrow alignment proving' since the system repeatedly 'proves' that it gives true answers. Case in point, the proper way to view Factored Cognition is as a tool to achieve outer alignment.

In stock IDA, this corresponds to the fact that we get outer alignment of each Ak by induction, precisely because each amplification step is implemented via Factored Cognition. In Debate, this corresponds to the debate game itself. Take out everything after the first statement by the first agent, and you get precisely the classical Oracle AI setup.

2. Debate ≈ IDA + decomposition Oracle.

My impression has been that some people view IDA and Debate as quite differently. They can get a handle on IDA, perhaps because of its similarity to existing Machine Learning techniques, but having two agents debate each other sounds exotic.

However, I think the proper way to view Debate, especially in the limit, is simply as stock IDA plus a decomposition Oracle. If you consider an HCH tree solving a problem, say deriving a math proof, you get a tra...
Writing tools for tabooing?
by abramdemski — 2020-12-13T19:50:37.301Z

I was recently reminded of E', that is, English without any forms of the verb "to be". Are there any tools for writing in E'?

More generally, it could be useful to have writing tools which help you taboo specific words, to try and write/think more clearly.

To be clear, I don't (currently) think there's a set of words which just should be tabood generally, including forms of "to be" -- but tabooing specific words at times can be very useful.

Another example is the idea (which is related to nonviolent communication) that we shouldn't use "should" and related words (such as "ought"). Trying to speak without these words for a time can help eliminate specific mistakes in thinking.

There's also Simple English, which is a restricted set of English words. This is kind of like tabooing almost everything. You can practice writing in Simple English using the XKCD Simple Writer.

Another tool for writing plainly is Hemingway Editor, which tells you when you use complex sentence structure, big words, extraneous words, or phrases with simpler alternatives. It also marks the reading grade level! Unfortunately, although it marks passive voice, it doesn't mark all occurrences of "to be", so it doesn't help practice E'.

The best thing (for me at least) would be a Chrome extension that makes it easy to taboo specific words whenever you want, anywhere you're writing on the internet.
Pseudorandomness contest, Round 1
by Eric Neyman — 2020-12-13T03:42:10.654Z
Link: https://ericneyman.wordpress.com/2020/12/13/pseudorandomness-contest-round-1/

I’ve decided to run a pseudorandomness contest — a reverse Turing test of sorts, if you will. Winners will get to send some of my money to a charity of their choice! Here’s how it’s going to work:

Round 1: Pseudorandom number generation

Should you choose to take part, you will have 10 minutes to come up with a sequence of 150 bits (i.e. zeros and ones). The deadline for doing so is Saturday, December 19th 11:59 ET. You will use the form linked at the bottom of this post for submission. (But read the rest of this post first.)

You may not use a computer or any other resources. For example, you may not use a watch to try to generate random numbers (though you can use one to keep track of the time remaining). No using books, or calculators, or deriving random bits from cracks in the ceiling, etc. The only exception is: you may use Notepad (or something similar) to write down your sequence of bits as you come up with them (but not for scratchwork — everything besides writing down your bits must be done in your head). I recommend this website so you can keep track of how many bits you have. (You may not use a counter that tells you how many of each bit you have.) You may use anything you currently have stored inside your head. You may not memorize anything in advance of participating, though you are allowed to think in advance about general strategy (without writing anything down). [Edit 1: Please do not look up strategies; any thinking you do in advance should be your own.] [Edit 2: Think about the rules this way -- ideally you'd be doing this in a uniformly blank room with a computer where the only things you can do are type 0, type 1, backspace, and go to some point in the string you have already typed, as well as see how much time you have left and how many bits you have already typed.]

Round 2: Distinguishing real and fake randomness

If there are n entries in Round 1, I will use a computer to come up with n sequences of 150 random bits. I will post all 2n sequences in a random order in a Google sheet. You will have as much time as you want (subject to a deadline — probably a week or so after I post the Google sheet) to try to figure out which sequences were generated by a human and which by a computer. Then, for every sequence you will submit a probability that that sequence was made by a computer (i.e. “truly” random).

You may use any resources you want to complet...
To listen well, get curious
by benkuhn — 2020-12-13T00:20:09.608Z

source

A common piece of interacting-with-people advice goes: “often when people complain, they don’t want help, they just want you to listen!”

For instance, Nonviolent Communication:✻✻
Nonviolent Communication, ch. 7.

It is often frustrating for someone needing empathy to have us assume that they want reassurance or “fix-it” advice.

Active Listening:††
Active Listening, p. 2

Similarly, advice and information are almost always seen as efforts to change a person and thus serve as barriers to his self-expression and the development of a creative relationship.

You can find similar advice in most books on relationships, people management, etc.

This always used to seem silly to me. If I complain at my partner and she “just listens,” I’ve accomplished nothing except maybe made her empathetically sad. When I complain at people, I want results, not to grouse into the void!‡‡
Empirically, I did notice that I usually got better results from listening than from giving advice. So I inferred that this advice was true for other people, but not me, because other people didn’t actually want to fix their problems.

Frequently the “just listen” advice comes with tactical tips, like “reflect what people said back to you to prove that you’re listening.” For instance, consider these example dialogues from Nonviolent Communication:§§
Nonviolent Communication, Chapter 7, Exercise 5.5, 5.6 and solutions.

Person A: How could you say a thing like that to me?

Person B: Are you feeling hurt because you would have liked me to agree to do what you requested?

Or:

Person A: I’m furious with my husband. He’s never around when I need him.

Person B: So you’re feeling furious because you would like him to be around more than he is?

I say this with great respect for Nonviolent Communication, but these sound like a 1970s-era chatbot. If I were Person A in either of these dialogues my next line would be “yes, you dingbat—can you turn the nonviolence down a couple notches?” I’d feel alienated knowing that someone is going through their NVC checklist on me.

Recently, I realized why people keep giving this weird-seeming advice. Good listeners do often reflect words back—but not because they read it in a book somewhere. Rather, it’s cargo cult advice: it teaches you to imitate the surface appearance of good listening, but misses what’s actually important, the thing that’s generating that surfac...
The Fermi Paradox has not been dissolved - James Fodor
by mako yass — 2020-12-12T23:18:32.081Z
Link: https://forum.effectivealtruism.org/posts/kvZshdx5FzTPjyhxG/the-fermi-paradox-has-not-been-dissolved

In this essay, I will argue that the analysis of Sandberg et al. is flawed in a number of key respects, and as a result the Fermi Paradox remains an open question. Here I briefly list the key problems with the Sandberg et al. paper, before proceeding to discuss each in more detail.

The method used of multiplying uncertainties of many small numbers, most of which have an upper bound of one, is biased towards yielding a result of a high probability of Earth being unique, while also leading to various dubious results.
The key result of the paper is driven largely by uncertainty in the parameter fl, which is modeled in an unusual way without clear justification.
Adoption of slightly different (and I believe more plausible) modelling choices and parameter values yields totally different results, which do not result in the Fermi paradox being dissolved. I illustrate this by re-estimating the Sandberg et al. models using different parameters and modelling assumptions.
Why quantitative methods are heartwarming
by KatjaGrace — 2020-12-12T21:50:08.259Z

From Twitter:

If you listened to my podcast w/Michael Sandel, you know we have very different views on whether markets are "degrading"
One thing I didn't mention to him: This bit in his book cracked me up -- because I remember my friends & I found this aspect of Moneyball SO HEARTWARMING <3 pic.twitter.com/9W6Op30vF8

— Julia Galef (@juliagalef) December 10, 2020

I haven’t actually seen Moneyball, but it does sound heartwarming, and I have had to hide my tears when someone described a payment app their company was working, so I’m probably in Julia’s category here.

If I didn’t feel this way though, reading this I might imagine it as some alien nerdly aberration, and not a way that I could feel from the inside, or that would seem the ‘right’ way to feel unless I became brain-damaged. Which I think is all wrong—such feelings seem to me to be a warm and human response to appreciating the situation in certain ways. So I want to try to describe what seems to be going on in my mind when my heart is warmed by quantitative methods and efficient algorithms.

When using good quantitative methods makes something better, it means that there wasn’t any concrete physical obstacle to it being better in the past. We were just making the wrong choices, because we didn’t know better. And often suffering small losses from it at a scale that is hard to imagine.

Suppose the pricing algorithm for ride sharing isn’t as good as it could be. Then day after day there will be people who decide to walk even though they are tired, people who wait somewhere they don’t feel safe for a bit longer, countless people who stand in their hallway a bit longer, people who save up their health problems a bit more before making the expensive trip to a doctor, people who decide to keep a convenient car and so have a little bit less money for everything else. All while someone who would happily to drive each of them at a price they would happily pay lives nearby, suffering for lack of valuable work.

I’m not too concerned if we make bad choices in baseball, but in lots of areas, I imagine that there are these slow-accreting tragedies, in thousands or millions or billions of small inconveniences and pains accruing each day across the country or the world. And where this is for lack of good algorithms, it feels like it is for absolutely nothing. Just unforced error.

Daily efforts and suffering for nothi...
An argument for personal identity transfer.
by Gadersd — 2020-12-12T21:03:13.552Z

I am very concerned with the general attitude towards cryonics and body preservation in general. People who reject these as worthwhile as far as I can tell fall into two primary camps: the probability of revival is too low to justify the monetary sacrifice or that personal identity is not transferred in the revival process. The first issue does not worry me much. Restoring brain function or some equivalent is an engineering problem, a practical problem. Monetary cost is an unfortunate problem, but it is also a practical problem. The other issue however is more of a philosophical one. Even if the technology to restore a preserved brain or upload it into a simulation becomes viable technologically and monetarily people may still reject it for philosophical reasons. Practical problems can be solved through sufficient research and design, but philosophical problems may never go away.

Regarding synthetic brains or brain simulations, I have heard time and time again people claiming that any brain created in such a way will not have the same identity as the original. If someone's brain is scanned while he or she is alive and a synthetic or simulated brain is created and run, then I agree that two separate identities will form. The problem, I think, is that people imagine this particular situation and generalize its conclusion to all possible scenarios regardless of context. Obviously if the scan is performed after the original brain ceases to function there will not be any parallel consciousnesses to diverge from each other.

Some people will then argue that a synthetic brain or simulation cannot even in principle carry over the original consciousness, that personal identity is not transferred. I will try to provide an informal sketch of a proof here of the contrary, that personal identity for all intents and purposes can be transferred over to a synthetic or simulated brain.

Assumptions:

#1 There is a brain device that manifests consciousness using neurons or some functional equivalent. It may be a natural biological, synthetic, simulated brain, or a mixture of these.

#2 There is a procedure that is to be performed on the brain device that will replace some neurons with functional equivalents such that neurons in the unaltered regions of the brain device will not behave any differently throughout time in the presence of the replaced neurons than they would if no neurons...