skysight-inc/HackerNews huggingface dataset:
Cerebras and Opentensor are pleased to announce BTLM-3B-8K (Bittensor Language Model), a new state-of-the-art 3 billion parameter open-source language model that achieves breakthrough accuracy across a dozen AI benchmarks.

BTLM-3B-8K Highlights:
• 7B level model performance in a 3B model
• State-of-the-art 3B parameter model
• Optimized for long sequence length inference 8K or more
• First model trained on the SlimPajama, the largest fully deduplicated open dataset
• Runs on devices with as little as 3GB of memory when quantized to 4-bit
• Apache 2.0 license for commercial use.

BTLM fits on mobile and edge devices with as little as 3GB of memory, helping democratize AI access to billions of devices worldwide.

BTLM was commissioned by the Opentensor foundation for use on the Bittensor network. Bittensor is a blockchain based network that lets anyone contribute AI models for inference, providing a decentralized alternative to centralized model providers like OpenAI and Google. Bittensor serves over 4,000 AI models with over 10 trillion model parameters across the network.

BTLM was trained on the newly unveiled Condor Galaxy 1 (CG-1) supercomputer, the first public deliverable of the G42 Cerebras strategic partnership. Cerebras acknowledges the generous support of G42 Cloud and the Inception Institute of Artificial Intelligence. Cerebras also thanks Cirrascale, who first introduced OpenTensor to Cerebras and provided additional technical support. Finally, Cerebras thanks the Together AI team for the RedPajama dataset.

To learn more, check out the following:
- Blog: https://hubs.li/Q01YG_yr0
- Model on Hugging Face: https://hubs.li/Q01YG_HS0
Neat. I'm assuming these would be fairly limited to niche installations, since the cost would almost certainly not be competitive to current panels in an installation where you have the ability to just add more area (not to mention the difficulty of perovskites in general).

I'm trying to think of places these might be worth it -- I guess space constrained areas where ground albedo is high?
Because I happen to come from the gutter like most people. I was manipulated, bullied, I was a drug addict and I did a lot of other dumb shit. I am so extremely experienced in how gullible humans beings are it’s honestly sometimes hard to live in this world.

And if you’re wondering why it’s hard, it’s because those experiences define who you are. Do you think I chose them because I thought it would be fun? Nah, those are the things I had to experience and learn from.

Most people don’t want to learn from their mistakes because it means you have to give up a lot all at once. How are you going to be a decent human being if you can’t give up the old to let it the new? Your plan is to spend the rest of your life feeling sorry for yourself?

That is quite pathetic.
This looks awesome! It's exactly what I've been looking for; not a tool for investment geeks, but a practical, everyday, personal finance app. Just signed up for a trial!

Some feedback I have after I signed up and filled out my finances and financial plan:

- It would be nice to somehow get back to the original wizard that I went through when I signed up. Could swear there were some things such as "spend my remaining income" or whatever that I can't seem to find elsewhere in the app. (EDIT: I found this, it's at the bottom of the Plan page, and was a bit hard to find)

- I'd love to be able to export my data even though I agreed to cloud storage. Maybe that conflicts with something about the way that stuff is stored in the cloud, but just being able to download any of the data, even if it isn't all of it, would be nice. Maybe there is a way to do this, but I haven't figured it out.

- Thank you for providing dark mode!

- I don't know what "Starting Cost Basis" is when adding a Roth IRA, and it would be nice to just have a tooltip explaining it; I have a Roth IRA and kinda know what a cost basis is, but honestly have no clue what "starting cost basis" really means or whether it applies to me. Maybe I should know, but this would also be good learning opportunity for users.
>why did no one come up with this before

So it turns out someone did. Specifically google did. This exact same idea has been in flaxformers since at least November 2021.

https://github.com/google/flaxformer/blame/ee62754ebe5a5eeb1...

Specifically to save people a click it says:

> """Softmax function with an additional virtual logit equal to zero.

For compatibility with some previously trained models.

This is equivalent to adding one to the denominator.
In the context of attention, it allows you to attend to nothing.

And creates the exact same modified softmax as this essay. I suppose only time will tell why it was ignored publicly before, maybe it doesn't do much, maybe it just fell through the cracks, maybe google just didnt push it, who knows
In 2011 I moved to Fedora (v16 at the time) and have been using it since then. This year I stopped using GNOME and moved to KDE, seeing if I'll make the switch permanent.
Are young people "flocking" to astrology?

The article provides evidence that the market is growing but as far as specific support for "young people" I saw about two.
The first is the quote "The astrology field is booming — a trend that has been driven by younger generations, experts say, ..." and the second is from a YouGov poll.
In that poll 37% of 18-29 year olds believe, and 33% of 30-44 year olds believe, and then there is a large drop to 22% for 45-64 year olds and 16% for 65+ year olds [1] where 3472 people were surveyed [2].
The polling has a stated margin of error of 4% [3].

It seems more accurate to say the 18-44 year old range has a much higher likelihood of believing in astrology than those in an older age group.

Was there something I missed?

1: https://today.yougov.com/topics/entertainment/articles-repor...

2: https://today.yougov.com/topics/lifestyle/survey-results/dai...

3: https://today.yougov.com/about/panel-methodology/
I do like the idea that parts of a system can simply be turned off and when a number of faults have accumulated, a tech is assigned to deal with them all at once, amortizing the maintenance costs over more tasks.

Better overall to replace three drives at the same time, especially if the servers are a half hour away from you.
My buddy is an aircraft mechanic, and his stories could definitely make a person worry about flying!

Kind of like how talking to somebody who works in any restaurant kitchen will scare you from eating out anywhere.
Well duh, that's what the OG article is about, why the author of this one would need to repeat himself
Paypal, ebay, robinhood, vanguard, various crypto exchanges. I guess it's mostly financial stuff. But that's my most sensitive info.
What DOS and Win16 support? OpenSSL 3.x requires Windows 10 as far as I know. Even Windows 7 is not supported anymore.
German socialists also described Hitlers rise to power as a "coup" or implied that he "seized" the chancellorship unlawfully. It was too painful for them to admit that so many of their countrymen supported Fascism and that he won a democratic election. I'd imagine botencat is in a very similar situation.

Remember Trump's "Muslim ban"? The US courts declared the law unconstitutional and prevented its implementation. The judicial reform would make it de facto impossible for the Israeli supreme court to strike down whatever crazy stuff the right-wing extremist government comes up with. Death penalty for stone throwers? No problem.
It might be a better take that apple generally treats games on an equal level to every other category of software. They just don't get much special treatment, historically at least ... ios may have changed that drastically (apple arcade is certainly a step in a direction). Meanwhile MS (even pre xbox) carved out work for them with things such as directx directinput etc.
The family of sigmoid functions has nice gradient properties with theoretical backing. Good starting read: https://stats.stackexchange.com/questions/162988/why-sigmoid...
> generally want a permanent result

I've often thought Tibetian-style colored sand mandalas would be a fun way to use such devices.

Of course, you'd need a lego-sorter-style device to separate the sand into different colors afterward.
Plus vendor compilers often outperform GCC and LLVM… on Cray systems especially
I hope one day people are free from the grip of Veblen goods. Its probably impossible due to genetic causes, like a demand for status.

However we may be able to educate people that 'you cannot buy class'.

Its interesting how many lower income people have iphones, Jordans, Teslas/Mustangs/Corvettes but as soon as you hit the upper-middle class, these disappear in favor of the more practical/functional items of similar prices. Not to say that some in the upper-middle class don't waste money on a Ferrari/Lambo, but I imagine its the same thing, 0.1ers don't buy those products.
For those like myself who do better with visuals. Just under 2min direct explanation using a model plane

https://www.youtube.com/watch?v=BBqOyDviE-A
From z/OS to Oracle. Good Lord!
This is already a completely solved problem in most of the world. They simply cap the fees card companies can collect from merchants by law.
I honestly believe that Musk is doing this stuff himself.
https://flarum.org/ is really nice and modern. I donated to https://freeflarum.com/ and used my custom domain for their hosted free offering.
not a token, and not the transformers, but yes, commercial chat models are fine-tuned on text transcripts containing dialogues. (i believe llama-2 was as well)
Except this isn't about you. There are plenty of folks who can responsible take drugs, too, doesn't make trying to keep folks hooked any less destructive and predatory. All you're being is an outlier in the highly lucrative debt trap that credit cards offer the average credit card owner.

Congrats on not falling for it; for you, a thousand people who do.
My favorite thing, which I don't know if I ever saved in my favorites, was a thread on your favorite HN comments.

A user linked to a thread (which I don't have saved sadly) of a different user talking about how they ordered a random unknown sweetener from somewhere internationally.

The package came by months later, and when the user opened it - white powder coated their kitchen, and their tongue.

They apparently loved the taste of it so much they carried the powder in a shaker with them everywhere. I remember them saying something along the lines of: "This stuff is so good that I haven't bothered to look up any health effects. I don't want to know. I'm fine with dying early, as long as I can keep on using this stuff."

I really wish I could find this comment again.
Thank you for that catch, yes, idealistic monism. Oh, I should have added E for as I already mentioned monism. D & F are also interesting positions.

Neutral-monism sounds like an hegelian dialectic [the gap]:

Many neutral monists hold that the terms “physical” and “mental” apply only to groups of neutral entities, not individual ones, "The stuff of which the world of our experience is composed is, in my belief, neither mind nor matter, but something more primitive than either. Both mind and matter seem to be composite, and the stuff of which they are compounded lies in a sense between the two, in a sense above them both, like a common ancestor". (Russell 1921: 10–11)
I have 35% confidence in this
Browsers finally have somewhat consistency when it comes to SVG rendering. For a long time, it was almost impossible to get SVGs that had the exact same look and position (down to the px) in all browsers and platforms. Otherwise we would have been using SVGs for icons and logos for a long time already, it's superior in most cases when considering performance, resource usage, I/O, quality and more.
Is this a question?
I would agree the author was wrong, and I would hazard a guess that you're a lot more well read than most of us on this kinds of stuff, because this talk is widely seen as an insightful.

Is there a language that you feel solves this problem better than Spec could ever hope to?
> Also, the article says "generates VASTLY more electricity" (emphasis mine), but then in the body it says it's 20% more compared to single face panels?

A 20% increases in solar electricity production fits the bill for vastly IMO. If I was 20% taller I would be VASTLY taller.
> A decentralized platform is verifiable and less prone to being rigged. All that's missing is proof of personhood.

That seems to be an extraordinary claim, and I'm skeptical that 1) it's true, and 2) that's all that's missing.

In short, I've voted for many things in many contexts in the existing world. I don't think I've ever bumped into a problem with voting that any form of a decentralized platform would have improved.

> You are picking a really tiny edge case and saying the entire system can't work because of this.

It's just the first example I thought of. I'm sure that 8 billion people could crowdsource all source of other edge cases.

> You aren't considering the possibility that the edge case can be adequately resolved with enough thought and investment.

I wasn't the one making the proposal. The onus is on the proposer to explain how they're going to deal with the edge cases and convince the rest of us that this is a real, thought-through idea.
When they released an Apple TV with an app store, casual gaming seemed an obvious use.

But they came with a controller that seemed to be designed with the goal of ensuring that it was as useless as possible for gaming (also as useless as possible for entering passwords!)
Most jobs don't allow for deep focus or long-term thinking. How do we expect people to be good at it without practice while we encourage the opposite behavior.
"Don't forget that borosilicate glass also cracks with thermal shock".

Hardly. Borosilicate glass has a very low thermal expansion coefficient compared to soda lime glass. Which translates into much lower mechanical stress due to temperature gradients.

This is why laboratory glassware is made from borosilicate glass. Such glassware can safely be held in a flame (kitchen: stovetop use), which is not safe with soda lime glass (tempered or not).

Breakage of soda lime glass may happen due to this difference in material properties.

For example: an oven dish 'burns', what do people do: rush it out of the oven (no problem), place it on a towel (no problem), or dump it in the kitchen sink. In the latter case: oven-temperature glass on ambient temperature metal plate. Bonus points if kitchen sink is wet for improved contact.

This "water splashes in sink" (or on kitchen countertop) causes certain areas of the oven dish to go to near-ambient temperature, while rest of the dish remains at near-oven temperatures. This is what causes difference in thermal expansion -> mechanical stress -> breakage. Small puddles of water in sink or kitchen counter are easy to overlook when someone's in a hurry because smoke comes out the oven. :-)

Hence the advice to place oven dish on DRY surface that conducts heat poorly, like towel, wood, or silicone sheet.

Borosilicate glass is much more likely to survive this type of abuse.

Note this says nothing of mechanical abuse like chip damage / scratches, dropping on the floor, hitting other objects, or some combination thereof. Tempered soda lime glass may well have an advantage there (but any object & its material has its limits of course).
30% of Indian and Israeli VC backed startups set-up US HQ
I analyzed more than 38,000 companies that have raised more than $5M across the world over the last 2 decades. More than 30% of venture-backed startups operating out of Israel and India have set up their headquarters in the US.
> Fully dedicated cores aren't necessarily required because in the timesliced case the registers are unloaded and reloaded when different VMs are shuffled on and off the core. That said, they definitely prevent the cross-vm-data-leak case of this bug.

Registers are unloaded and reloaded when different processes / threads are scheduled within a running VM too. That should protect the register contents, but because of this issue, it doesn't, so I don't see why it would if it's a hypervisor switching VMs instead of an OS switching processes. If you're running a vulnerable processor on a vulnerable microcode, it seems like you can potentially read things put into the vulnerable registers by anything else running on the same physical core, regardless of context.
A Grandchild's Guide to Using Grandpa's Computer a.k.a. "If Dr. Zeuss were a Technical Writer" was written in 1994 and mentions microcode.

Microcode updates are always discussed when talking about microarchitectural security vulnerabilities (and other scary CPU errata like https://lkml.org/lkml/2023/3/8/976).

Microcode is always mentioned when discussing CPU design evolution.
> Every single one of my elderly relatives can use an iPhone after a quick intro

So, after an onboarding workflow, in other words. What's the difference between giving them a quick intro, and their being some form of onboarding built into the application? Consider people who don't have someone on hand to introduce them to the product: how should they gain that same level of competence?
No, “de” in French has the same meaning as “of” and sounds exactly as weird here. That’s clearly not where this comes from. Also the translation of string with this meaning is “chaîne de caractères”.
Nothing is free. It is estimated that state and local governments spend 9 percent of state and local direct general spending on higher education [1]. One could argue that the opportunity cost of higher education to taxpayers is 9% of their work life.

[1] https://www.urban.org/policy-centers/cross-center-initiative...
Frustrating that the study this article is reporting on (linked at the end) https://www.cell.com/joule/fulltext/S2542-4351(23)00224-6 is paywalled. To save others' time, the "download full issue" obtains a .zip of PDFs, but not including the relevant article, only other articles in the issue that presumably someone paid to make open access.
This was is the most obviously and definitely stupid, sure, but the other stuff was also very stupid.
> a good test would be to modify one of the smaller models (maybe nanogpt) and swap out all of the softmax calls for his quiet softmax.

You'd have to train the model with the quiet softmax before inferencing with it would work.
Inevitable. Humans have naturally adapted to becoming more comfortable with the screen than with actual humans.

If an AI girl is easier to talk to than a real girl, this was bound to happen.

And companies will ensure that AI girls flirt better than real girls with real debts, careers and other problems.
Yep. I worked at a software shop where they liked for developers to have roughly 6 projects simultaneous at any time. The constant context-switching drove me insane. By the I time I 'loaded into mental RAM' all the context for one project, it was time for a meeting about another project.

For an attention deficit programmer maybe this is a bonus. Not for me. I like to focus in on the task, take notes, figure it out, and get it out the door and off my plate. I don't like to nibble on code.
Hobbyists: "Wanna do IPv6? It's cool and we'll do our part for incentivizing people to also switch! It should only take a couple of Saturday afternoons." "Sure, sounds cool, let's do it!"

Meanwhile, at any average corporation: "Ok, so implementing IPv6 will cost us x$ and n person months, got it. And the upside is that we'd get a badge on some IPv6 readiness site, yes?

And what would be the consequence, for us, of not doing this? Interesting, absolutely nothing you say, at least until every single customer is at least on IPv6 dual stack, since we need to provide our existing IPv4 stack too? Cool, thank you for presenting, please don't come again!"

About the only large commercial entities that see any benefits of rolling out IPv6 are eyeball-heavy ISPs (since selling native IPv6 + CG-NAT for IPv4 is a slightly less horrible proposition than selling CG-NAT alone), and much less importantly any entity so large that CG-NAT capacity (memory and assigned IP/port tuples) is an issue for them.
But you are wasting some of the model's capacity to learn to ignore some of that information. I think it wouldn't hurt. However, if I followed the reasoning correctly, I think the biggest win is to reduce the range of the weights more than improving performance.

> This is what’s been happening in LLMs – for reasons that are only partially understood, Transformer models contain these outlier weights and are emitting Black Swan mega-activations that are much, much, much larger, like orders of magnitude larger, than their peers ...

meaning that once quantized you can either have a finer quantization since the range of possible values is smaller or you can pick a coarser strategy that saves bits for each weight.
Reading this I'm mostly thankful real brain power and the general smart programming community is seriously taking a close look at all these things. I barely feel the need to try to compete for insight gathering it feels very healtily analyzed from every perspective finally.
I think eventually we may see some tracked controllers, perhaps they shied away from them for launch because they would add to the already higher than expected price point and won't be used in most of the work-focused scenarios they seemed to present. Having to stop typing to pick up a controller to twiddle some 3d ui and then swap back to keyboard seems .. shitty.
> the problem of “systematic” (or “structural”) racism in STEM, which probably doesn’t exist.

The word "probably" is doing immense heavy lifting in that sentence.
I’ve always hated this style of argument in tech where someone says X can do what Y does as an argument against using Y (or for using X) because it always missed *why* people use Y in the first place.

See for example you can write safe code in C therefore rust is pointless (yes but rust makes it much easier to write safe code)

Or you can do machine learning in JavaScript so python is useless (you can but it’s a much smaller ecosystem)

Etc etc.

You can pick whatever tools you want for whatever reason but please give me a better reason than it *can* be done.
But who does that achieving of consciousness?
A random seed written on a piece of paper?
I'm a bit perplexed that you see trashing a man's character with certainty as right and pondering if his actions could reflect a struggle with mental illness as wrong.

I don't like Musk, nor do I trust the things he says, but I usually err on the side of granting people some humanity.
I think its hardware is what enables such reliable software.

Hot plug CPUs, memory modules, and DASDs. All survive the failure scenario gracefully.

Good luck trying to do that on a rack of x86 servers.

I think as in most engineering things. They do a different trade-off and cater to a different niche while x86 servers are the mass market consumables that most workloads -should- be using. The mainframe CPUs go for the widest IO capabilities, insane cache sizes and hardware offload because that’s what their target audience(banks, insurance companies, airline ticket systems) need.

Their parallel sysplex clustering solution is an engineering marvel but also tightly coupled to their hardware.

In some ways, the IBM mainframe is the Apple of niche critical enterprise computing. They are also the ancestor of cloud computing in paving the way for “pay per use” cost models and multi tenancy.
Oh, I had misunderstood something.
I'm not a big fan of Spotify but they have the largest collection of curated high-quality music others can't provide. Combined with the easy last.fm scrobbling, I've decided to keep it despite the increased price. Though their UX kinda sucks and lot of individual songs or even albums go randomly missing, which is annoying.

Youtube has practically everything but old uploads are sometimes so bad you can hear the compression and noise. And the rest of the streaming services don't have as much music, which means more downloading, tagging, organizing and storage needed for me.