skysight-inc/HackerNews huggingface dataset:
RLHF entails f for ChatGPT because you cannot get ChatGPT without RLHF. This is before the feedback from users, and was part of the initial training process. Without RLFH, you only have GPT-3 which is very different from its successors that people are actually enthralled with and worried about.

RLHF is Reinforcement Learning from Human Feedback

Thus you cannot get ChatGPT without humans in the loop, making it quite sensitive and irreproducible.
"You forgot an infinite number of other answers!"

/whipped for each one
> Storing data in a database vs in files on disk is like application development 101

It's OK until you're dealing with, say, 130GiB of tensors, on what is effectively a binary blob that needs to be mostly in VRAM somehow.

I really don't want to read 130GiB of blobs from a database all the time.
You can tell because Goodwill is always filled with romance novels and cookbooks. They’re both very aspirational.
Wistron (Taiwan based) sold plants to Luxshare (PRC based) - so is Apple failing moving from China?
dont over follow recipes. Get the ingredients and the amounts if you care to and then use your judgment. I mean read it so you know what they do, but there are likely small details that are missing or that are included but not relevant. It also may just not be optimized for your preferences.
I’m using it to do the mundane tasks of unit testing and (some) documentation. I find that the code it spits out isn’t perfect but getting some boiler plate and fixing it up is pretty fast compared to writing from scratch.

I’ve used this enough that I wrapped some cli glue around it and wrote https://github.com/radoshi/llm-code

I’ve used this mostly to write Python and bash, with some Makefiles and Dockerfiles thrown in.

GPT-4 is better, albeit slower, than 3.5-turbo. HTH!
i get that for the first few seconds until my brain starts to filter the jitteriness

is there a way to apply a little stabilization to consecutive outputs, something like https://www.youtube.com/watch?v=X1WG5jc5SU0 ?
It really sends home the concept behind the phrase, “ain’t no war but class war.”

This “ruling” class just does the theater to keep us in the, sometimes actual, dark.
I also had some false starts with meditation and didn’t “get it” until I found good resources on mindfulness/Vipassana.

I used the “Waking Up” app (not free but has a 30 day thing that’ll get you through the introductory course), and it was as if the content was targeted at people who’ve tried and didn’t get it. Mantras are not the focus.

There are many types of meditation, but the kind getting all of the attention in studies (for good reason) is the mindfulness variety.

The reason Yoga suddenly clicked for me was that I brought the mindset from meditation into the Yoga practice. Exercise (cycling and long walks) is always beneficial, but my body awareness increased so drastically with Yoga that I feel like a different person.

That may have more to do with my history of dissociative tendencies, but I do think there’s a connection between a good meditation practice and Yoga becoming something more than just exercise. It becomes an entry to flow.
I've been admiring this trend since I started seeing these kinds of essays posted here, and felt motivated to start collecting them in a curated list[0]. Happy to accept new entries or volunteers to help maintain it!

[0] https://github.com/BHSPitMonkey/awesome-explanations
Thanks for the interesting discussion. Yes, the fog of time affects us all.

Funny you mention caption buttons (min/max/close/etc.) and menus and such. I meant to add a "Fun fact #2" in my first comment. So here we go...

In traditional Windows applications, none of those are child windows (or child controls, same thing). They are all part of the "non-client area". You may recall there being a whole series of WM_NC... messages like WM_NCMOUSEMOVE and WM_NCPAINT, with the same names as your usual messages except for the NC prefix. Your WndProc would receive all of these messages, but generally would just pass them along to DefWindowProc which handled this "non-client" activity.

Now here is the fun fact. OS/2 Presentation Manager took a different and cleaner approach. It removed the "non-client area" concept entirely, along with all those messages. Instead, all of those window doo-dads were child windows. The minimize button was a child window. Your "client area" was a child window. And so on. It was all child windows.

On this point:

> implementing behaviors that seem like their own child controls, but which are actually parts of the MDI child's own geometry, to reduce the number of event loops that need to be pumped.

To be clear, a typical application had only one event loop. It was your classic GetMessage/TranslateMessage/DispatchMessage loop. DispatchMessage would pass the message off to whatever WndProc it should be directed to.
Similarly: Why do drum teachers and instructional videos lie about how to hold drumsticks? I was told for years to hold them at the balance point. This is wrong! You generally hold them a few inches lower than the balance point which allows you bounce the stick loosely in your hand so that it rotates in the air around the balance point.
Those pesky people that don't want to create a culture where you are expected to die if you feel you are a burden you mean?
Forcing someone to provide things for you then?
> Unless they were absolutely terrible, professors received a standing ovation at the end of the quarter.

I guess I'll be the first one to point out that I never witnessed anything like that.
It might be true of the more stylistic chefs out there for sure -- but I'll pull an America's Test Kitchen recipe rather than a random blog that shows up in google.
> cards that were packaged with official Pi 400 kits were counterfeit

Why not? Company needed SD cards, they bought them and packaged with their kits. It could be from literally any source. Even with official SandIsk partnership.
Exactly! It's a fun natural development -those US breathless sales-based self-improvement books like 4-Hour Workweek, most books that Cal Newport publishes, are often blog-post level ideas inflated into a book for sales. It's only natural that people are finding ways to automatically trim the fat off to go back to the blog-level length.

"good" non-fiction like Thinking, Fast and Slow, The Selfish Gene, Structure of Scientific Revolutions etc. are books that are not easy, or even worthwhile to summarise that way. You'd lose too much information.
Chou Kung (3061bp)
You keep coming back to them? Really? Why? They're interesting articles but they just scratch the surface and I don't see how further readings would change that.
No, it doesn't make sense, people want freedom on their appliances as well (hint: printer catridges)
Over ten years! One cannot be a slob and get results you know. The difference between a normal diet and a super green is quite extraordinary.

The only superior diet in my long quest for excellence is a regular juice diet, which is either incredibly labor intensive or very expensive (much more than the pricey green powder.)

The (fresh) juice (greens and roots, less fruit) diet made me feel amazing yet impossible to keep on mass.
You're missing the point that, since a black hole is purely made of spacetime geometry, there is no "mechanistic explanation" of its spin in the sense you mean.
I suspect that many recipes benefit from being started a day or more early. I know bread does.
This was a response to a question asking why Russia has any influence over Ukraine's alliance choices. The US exerts similar influence over the foreign policy of its neighbors, as does China, which is right now aggressively pushing on Taiwan for this reason. It is the reality of larger powers interacting with smaller ones. No one said the current situation in Canada or Mexico is the exact same. Nor did I say any of this justifies Russia's choices. But IF, say Mexico, started to flirt with an oppositional alliance, this would be countered, with war as a last resort if the other strategies failed. This has been done several times before in US history (eg in Cuba, we backed a local revolt against Spain to take control of the island and other Spanish possessions. In the 1960s we again threatened nuclear war against the Soviets for expansion there and embargoed the country for half a century). And the fact you are so dismissive of US action in Mexico is a big blind spot. The cartels in Mexico control significant territory adjacent to the US. US federal agencies have been deeply intertwined with the Mexican government's efforts to curtail the cartels, but these have failed. Special forces already operate there. The likelihood of direct US action is not high right now but is certainly plausible within 20 years, especially if the cartels make further inroads within the offical government. US congressmen have just this year proposed legislation allowing "military capabilities" against the cartels. The world is different, but some things don't change, and more powerful countries dictating some of the policy of their less powerful neighbours is one of them.
In hot oil, small pieces of onion will turn into a crunch brown crisp in well under 5 minutes, and larger pieces can have the outside surfaces crisped.

I suspect the authors are not lying, but simply calling the above effects caramelization; not aware that what it means is for the onion to be evenly brown all the way through.
I think seeded randomness is wildly underused. It's much more often the case I was deterministic randomness rather than based on my system clock.

It's nice to see aimless.js support it.

Crazy to me that `Math.random` doesn't accept a seed as an optional param.
Maybe what we need is for a first world nation to finally do it, so the rest of the world can see what a catastrophe it turns out to be?
Evolution is working for it, on those smaller scales, though. Families will happily help each other, friends don’t need a detailed ledger to figure out who’s buying the next round. Most people have a good inherent sense of fairness I think.
You think it is good to "throw Taiwan under the bus" all for the sake of some organization being able to monitor a disease? Sad.

Following your logic, would it would be OK to "throw Canada under the bus" if it allowed some international organization to more easily monitor, um say their oil exports?

Taiwan did an excellent job dealing with COVID. China did a poor job. WHO should you listen too?
In my direct experience, with small rows and a carefully optimized workload and schema, 1 to 2 billion rows was totally fine in a single MySQL 5.1 table (InnoDB) on SSDs twelve years ago. Granted, that was on bare metal. But hardware has also improved a fair bit since then...
I don't think most physicists in this area think the Standard Model is "correct" in the sense of actually describing what makes the universe tick. They mostly recognize it as a set of curves that are extremely well fitted to experimental data, up to a point (at which point it fails completely and necessitates a switch to another model).
You run MongoDB as a backing store. Very web much scale.
Elsewhere she quotes herself instructing one of her students in one of her cooking classes: "Continue stirring, they won't burn!"
This was the case for a family member with undiagnosed apnea.

If you aren't getting enough oxygen at night, your body will want more sleep than necessary under normal oxygen levels.
Sifting flour.

Sautéing spices “to release the flavor” and then adding them to stew where they’ll simmer for an hour or more.

Pretending “restaurant quality” is anything besides twice as much butter and salt as you’d dare use at home.

“One tablespoon of olive oil” to brown absurd amounts of meat at high heat.

Once you start recognizing recipe BS, you see it everywhere.
The Pali Cannon, while orally transmitted by monks for 500 years before being written down, should not be assumed to be "the original, actual, oral teachings of the historical Buddha".

They are the closest thing we have to what the Buddha might have taught (assuming he actually existed, and like Jesus, there is no evidence outside of religious texts that Buddha did exist).
Certainly charging for upgrades with useful new and critical functionality is reasonable.

But reasonableness would also price even critical functionality relative to how hard it was to actually implement.

I.e. don't hold hostage a customers' investments in a software product. Don't force customers to pay a ransom to get a simple-to-implement feature.
Three. But many columns.
One thing I'm keen to understand is: how well does attention hold across huge context sizes, with respect to the usual transformer models, and also these proposed RNN models?

All these 2k/4k/8k context sizes that we've had recently are able to map pretty well to what a human could reasonably remember. What I mean is, you could ask a human to read some text with 8k tokens, and for the most part they could answer questions about the text coherently.

But what about 32k contexts, or beyond? At some point, as token size increases, the ability of a human to give a highly precise and detailed answer decreases. They must start to generalize. A human could not read Infinite Jest in one pass and then answer details about every single sentence. But could a transformer, or a RNN? As the context grows, is it harder to keep a high granularity of detail? Or am I wrong in trying to think of these models the way I think about the human mind, and they are actually able to handle this problem just fine?

I'm aware that we can cheat a bit, by adding a lookup step into an embedding database, to provide "infinite" context with "infinite" precision. But to me, that is analogous to a human looking up information in a library in order to answer a question. I'm interested in the inherent, emergent memory that these models have available to them in just one forward pass.
GPT-4 is clearly very impressive, they should show it off honestly and transparently. Instead OpenAI clearly treats these evaluations as a part of their sales and marketing, with inflated claims to match.
Literally pays for itself in scale
I find that hard to believe, tech workers just got their biggest pay bump in the last 2-3 years across all levels. If you can make the case for this bump being eliminated by inflation, maybe I'd believe you - you must show evidence of this. Otherwise, no, you're wrong. Definitively, you are absolutely wrong by the qualifier of experience based compensation.
"
Program criteria and eligibility

To qualify for our Open Source Sponsorship Program, projects must be open source and meet the following criteria:

Operate on a non-profit basis.
Include a link back to our home page.
"

So, funding in exchange for advertising? I accept that there's a cost for everything.
> if you worked diligently on the wrong thing, thats an even worse waste

So slow productivity in case the direction is wrong? Like he said, ideally, you package up the (correct) work task and hand it off to the worker.
They probably do the same thing, and don't notice that it doesn't coincide with the static balance point.
Trying to solve the three-body problem numerically? Haha you fools this was done years ago in FORTRAN
An indexed read should be no different than on a table with on the order of 100K rows. And that’s even with good ol’ spinning rust.

It’s all about logical I/O operations which, for an indexed read, would only be a handful regardless of size as it’s operating on a btree: https://en.m.wikipedia.org/wiki/B-tree

Now creating that index from scratch might take a while though…
Nvidia still doesn't sell their FE so much outside of US, so I don't think their intension is to selling more FEs.
It sounds like you agree that Walter's statement

> Now you have only $40 to spend on eggs, and (due to the Law of Supply & Demand) the price of eggs declines.

was utter bollox
U.S.-Made Technology Is Flowing to Russia
Dumb arxiv question (sorry); is it possible to see what journal a paper was actually submitted to, to help find the reviewed version when it comes out?
This model seem to be good when input context is large in comparison to OpenAI, would just wait till if someone productize it. Most likely it wine be just one paper but many new and old ideas combined
> This isn't me speculating wildly, everyone with even passing familiarity with the industry knows this.

I never questioned their market dominance, I questioned their overall company profitability. The cost of building a rocket can be easily fudged to create whatever financial picture you want. Not to mention Elon Musk is notorious for doing this. Unless you have access to their financial statements, all of this conjecture.

> The cost-per-KG is 10-20% of the competition.

Uhh might wanna check your source, from Wikipedia: "These varying cost and requirements makes market analysis imprecise."...
Building Chatbot with Laravel and OpenAI Embedding
Oh get over yourself. You’re little more than a username to me, I don’t give a shit about your kids, other than a general one about how all kids deserve care and respect from their parents. That’s your job, not mine.

My point is that ideology cuts both ways and it’s very easy to lose sight of the sharp edges in your own when you are endlessly focused on your enemy’s. I grew up deep in your ideology and have seen it play all the way out not only in my own life, but in my entire community and communities like mine (and yours). All I’m saying is that force-feeding your own ideas has a way of backfiring in the long run.

Amabo te… quit being a goofball and go have a conversation with your kids instead of threatening strangers on hacker news.
Honestly, this is the most secure and sufficiently functional password manager!
I stopped caring about industry trends and hype years ago. I was able to get away with it by coasting on what I did know for years, but otherwise I’ve been very narrow (not really true, but in the context of the job market it is). Funny enough this was somewhat on purpose as I was planning to leave anyway, but I thought I had a couple more years than I did.

On top of that, the jobs I still see seem want want increasingly obscure specializations I don’t have. I haven’t had a job for over half a year, and there doesn’t seem to be any sign of that changing. The recruiters dried up a while back and I’ve only had sparse interviews, which have been some of the most antagonistic I’ve ever had.

It’s not any particular “type” of company either. Both the tech profit center companies and the tech cost center companies don’t want me.
> We need to be mass producing missile frigates, oil-tanker-conversions as escort carriers, 1,000s of F35s, 1,000s of F22s, renovating every single plane in the Bone Yard that can be turned into a drone, and sending $1 Billion a month to SpaceX to get the StarShip operational for the High Frontier.

Why? This isn't all-out war nor do we need to militarize space.
When I want to read a non-fiction book but don't actually want to read it I go for the video lecture, if there is one, and for many there are.
Do you think twich should have moderation?
Foxconn is also a Taiwanese company (though its CEO is familiar with PRC)
Jesus H. Christ, first time I see a collaboration this big on a ML paper. How does a team like that even come together? This isn't the LHC.
You actually don't want too much oil when cooking steaks, otherwise they splatter everywhere. That's something I'm glad to learn.
When the butter solids brown (and blacken) that's burning the solids in butter.

Separating those from the clear golden liquid is how you make ghee.

Again, times given are stupid ridiculous.
When the Cymbals Come In
An open source CLI for Notion Tasks
Manage your ToDos from the command line, keep your team up to date with your Notion page.
Not sure if it answers your question but the paper notes something similar in its discussion of limitations:

> the linear attention of RWKV
leads to significant efficiency gains but still, it may
also limit the model’s performance on tasks that
require recalling minutiae information over very
long contexts. This is due to the funneling of information through a single vector representation
over many time steps, compared with the full information maintained by the quadratic attention of
standard Transformers. In other words, the model’s
recurrent architecture inherently limits its ability to
“look back” at previous tokens, as opposed to traditional self-attention mechanisms. While learned
time decay helps prevent the loss of information,
it is mechanistically limited compared to full self-
attention.

> Another limitation of this work is the increased
importance of prompt engineering in comparison to
standard Transformer models. The linear attention
mechanism used in RWKV limits the information
from the prompt that will be carried over to the
model’s continuation. As a result, carefully designed prompts may be even more crucial for the
model to perform well on tasks
We cannot, and should not come to terms with the prospect that the universe's complexities may be totally out of our reach. Best to keep trying in one way or another. Unfortunately, my comments are not constructive, I don't have a better avenue to suggest. All that I can tell is that phenomenon on human scale are easier to grasp. The bigger or smaller you get, the harder it is to observe.
The Nazi Connection to the John F. Kennedy Assassination
What would it mean though? If the information is embedded in the network, and we find some performance figure of merit, wouldn't it probably be about the same performance when normalized to power utilization?

Maybe it's about optimizing every clock cycle?
In this case there isn't a service provided.
This is why “warning. Game needs 64 mb and you have -2gb continue y/n?” Is better than just dying out.
A (16, 1) posit gives a maximum of 2 extra fractional bits of precision over float16, and (16, 2) gives you 1 extra bit. But also while the numerical distribution a posit assumes is slightly better but still not well geared to the distribution of values as seen in NNs in any case, it's not going to get you that much more (you hardly need anything above 10.0 because people like hyperparameters that look like 0.01 or whatever, even though with typical floating point you can multiply all the values by 10^-3 or 10^3 (or in float32, even 10^10 or 10^-10) and things would work just as well.

A posit is also more expensive in hardware, because the adders, multipliers and shifters are bigger (provisioned for the maximum fractional precision).

The trick is still in the scaling still. I've done full training in (16, 1) posit (like, everything in (16, 1), no float32 or shadow floating point values, like this paper) for some of these convnets. It doesn't work well out of the box without doing scaling tricks, and then it's ~the same as float16 with the tricks I find. It simply doesn't add that much more precision in such reduced space (that 1 or 2 extra bits at best).

What they benchmarked on here in this paper is ancient history too, not sure how these models are that relevant to modern practice these days.
uhhh it has 4 fewer arcologies than simcity 2000 so it is hard to evaluate the truth of this statement
Perhaps we need to make a special decrypt program, one that accepts any password and just produces some log files. They can't prove it is not the data (except by claiming they planted it)..
That was my first real smartphone.i loved it. Still miss it two. If you find the other 11 let's have a LAN party or sth.
How long does it take you to do a schema change?
Spices in a stew will only ever possibly reach 212. When blasted on dry heat they can get higher.

The tablespoon of olive oil is to kickstart the process, after that enough fat will render out of the meat to keep it going.
I often screw up roux, sometimes I get it and sometimes I don't.

For the times I don't, there's Nutribullet.
Are There Any Benefits to Tongue Scraping?
The ABC’s of business declined: arrogance, bureaucracy and complacency. It sounds like at least one of those were at play at RIM