skysight-inc/HackerNews huggingface dataset:
Ask HN: Has anyone considered using rich tokens for LLMs?
The tokens used by LLMs are the rough equivalent of the tokens produced during programming language parsing, but LLM tokens can only represent fixed character sequences. In programming language parsing, literals are parsed into tokens that have the literal value as extra baggage that they carry around.

Current LLMs map words to token IDs, such as "apple" -> '1', "the" -> '2', etc...

Why not have them map numeric literals to special token IDs that also has the numeric values associated with them?

E.g.: token IDs 0..1000 could be assigned to the first 1K distinct numeric literals being processed.

Subsets of the matrices at each layer could be special-purposed to do arithmetic on those tokens. E.g.:, if a "maths operator neuron" is activated with inputs that are IDs 533 and 712, then it'll "multiply" those two numeric values.

This could allow LLM-like systems to be built that can do arithmetic in the same way as a calculator instead of trying to do long-form multiplication like a human with pencil and paper.
I’m sure you were great but realize most of your student probably weren’t trying to major in physics? Maybe mostly engineering students and such who weren’t fond of the material or having to take it. I was experiencing this mostly in my upper division courses.
Where I live, the women's sports is on the AM radio when men's is on the FM band owned by the same station. There's also a BBC World Service mirror on AM. There's also a lot of non-English AM radio in the city I grew up in. I am under 30 and have used AM for these purposes quite often. Like a stone with lots of life underneath - just because something looks unused on the surface doesn't mean there isn't a bustling community.
Like many others, I've written my own correct-horse-battery-staple-based passphrase generator.

"gen-password" generates random passwords, allowing your to specify length and character set.

"gen-passphrase" generates correct-horse-battery-staple-style passphrases.

Both are written in Perl, using /dev/urandom by default for randomness.

https://github.com/Keith-S-Thompson/random-passwords
It's probably a random person. But, as long as it's not another story about the Surgeon General saying how social media is harmful for kids, hob-knobbing at the next COP conference next to oil companies, getting gigadollar fines related to GDPR? Am I missing anything? Have there been any actual good news lately?
Probably it's some risk analysis. Like they don't think the amount it bothers you is so bad compared to the chance a bad thing will happen multiplied by the potential damage of that bad thing. I don't know how you would argue against it.
>> the incentive structure around deriving your income from rents can systemically produce inhumane outcomes

It's also the only thing that produces new construction. Properly regulated and harnessed, the incentives are necessary to draw capital to building.
We might have different definitions my good sir. Granted I don't have a PhD in math and my math stopped at a masters. To me formalization is theorems and proofs, which are 100% comprehensible to humans. For reference: http://www.vdash.org/formal/#math

Everything in that link looks like programming, especially when you consider something like Haskell. Even formalism as a philosophy tries to add logic to natural language it self. So I am not sure where I am fetishizing what I saw at university. Care to explain without the snark?
Restaurants.

Usually just before opening or just before the dinner rush. You go over specials. assign sections, etc.

Construction.

Scrum is (allegedly) inspired by the morning briefing on a construction site where the electricians warn the plumbers they are going to turn on the power to floor X.

Military.

Before every mission and daily standing orders for officer of the day, officer of the watch etc.
No disagreement here. I think office workers should be productive too.
I think that is the preference/ideal scenario that Musk has in his head.

Take a morning standup with your coworkers about supply chain/scrum, head to the factory floor to show the technicians how to flash the latest firmware to the infotainment system, do some heads down work, go home, eat dinner and then take a phone conference late at night with your Tesla China/Berlin counterpart.

Probably impressions matter on the factory floor and when you scale up operations. You don't want a situation where the people on the factory floor just refer to the design team as "corporate" (i.e. "corporate" wants us to do this, guys we have to redo X over again on all 10,000 cars we produced today because "dude in corporate" thinks its not going to work otherwise).
> Information workers and factory workers are different.

Should they be, in terms of work-life balance?
The wavelength for 1700 kHz is 578 ft. Antenna would be half that, 289 ft.

AM stations usually have a tower that is the antenna.
My main argument against the AI doomsayers has so far been that the current scaling laws simply make runaway singularity style scenarios algorithmically impossible (if for each step of improvement you need 10x parameters and 100x training, you quickly run into a brick wall).

This is part of why I’m not worried about the current crop of generative AI. I am however both curious and concerned about what the tsunami of talent and $$$ chasing the current trend will achieve.

If this n^(4/3) alt transformer compute scaling is real (and there’s been many a pretender, so it’s too early to tell), then that could fundamentally change the overall AI scaling law, substantially lowering the brick wall.

And that could be a game changer.
No, JS and Canvas aren't the issue, there; the code you wrote will perform terribly in any language on any platform. You used a four-way recursive per-pixel fill algorithm. You should probably do a bit of reading on flood fill algorithms (maybe read the Wikipedia flood fill article).
When factory workers walk out on strike nothing in the factory gets made. It doesn’t seem insane to me that when engineers walk out the systems they maintain stop working.

It would obviously be career suicide to do something like turn off S3 for an hour, but it would definitely make a point.
> Those 30-page C++ compiler errors are definitely worse than whatever I had to do with (for example) Python.

That's definitely in the "programming language sucks" category. Good languages make it easy to debug problems with your types.
Everyone in an public or private position of influence over the economy should be barred from running more than a 7-11 after the covid fiasco.

Anyone claiming there isn’t any evidence the production of necessities is a zero sum game from the perspective of the average person is a deluded fool.

Inflation deflates the public to stabilize and grow buying power of imaginary asset owners. It’s a viral game of make up a reason why a minority are free of real work keeping them breathing.
All of what you say is true, for some landlords. Fortunately (around the world) its not true for many of them. That said the primary focus on money in the US makes the situation there much worse.

Western society also sees home-ownershop as a primary "good thing", and positions home ownership as a goal everyone should aspire to.

In truth, a large segment of the population is better served by renting. Those that move area frequently. Those who don't have sufficient capital or income to pay for such a large asset. Those who can't get a mortgage. Those who aren't ready to make a 30 year commitment to a location. Elder folk who don't want to die with a large asset, but would rather cash-out their house and actually spend that capital.

Since most Western governments do not provide rental housing (or are slowly reducing housing stocks) - it falls to the private sector (lumped together as landlords) to provide you need.

Incidentally shortages are not the fault of landlords, but of a population growing faster than housing supply. Supply is mostly constrained directly or indirectly, by existing home owners where the primary reason against any change us "preservation of house prices in our neighbourhood". This is driven by home-owners who have been persuaded that their house is an "investment".

So you may not like landlords, and like any profession there are plenty of bad examples to hate, but the removal of rent completely as a housing option would be disastrous to a huge segment of society.
>> attention should be directed at them to alleviate price pressures

Why is that the responsibility of small landlords who rent out a spare bedroom or second house? That's like saying it should be on independent gas stations to alleviate high oil prices.
Yes, my tenants certainly do. Thanks for the support.
Neither have I in NYC.
My neighbor at AWS did WfH pre-pandemic, went to "100% in-office" for about 3 weeks, and I think is at about 50/50% now. There's no real reason to go into the office without coworkers on the same team.

Offices = commute time and expense + noisy distractions - declining perks.

Butt-in-seat mentality (BISM) is antiquated.
We have 30 years fixed rates in Europe.

And most people here would consider taking out a variable rate loan to be a stupid move.

In fact, people in France are usually surprised and shocked whenever they learn of some random city nearby nearly going bankrupt because their local government thought themselves smart signing up for variable rate loans.
Huawei completed ERP switch in 88 overseas units in 15 hours
I’m actually surprised they have lasted so long...

Watching ARM's repeatedly eat unsophisticated investors, it makes a certain sense why.
> but anytime you buy stock, the goal is to benefit from the profits of the company without doing any work yourself.

Which is arguably also bad for the same reason. Not bad, morally, though, and that's kind of the key thing. When you live in a society where scarcity (either real, or cultivated) creates constant precarity for you and your family, doing what you can to exit that precarity is not a bad thing to do, even if the thing you end up doing ends up being something that leads to an overall trend of society into more misery for more people. After all, it's not like you individually choosing not to be a landlord means society would suddenly turn into one that hadn't created a social niche for landlords to occupy.
I'm just starting being a landlord and I am one bad tenant away from copying the inhumane practices associated with companies that deal with a plethora of bad tenants. I purchase my home last year and I am renting below cost.

If everyone behaves nicely, it works out well for everyone. But if a party doesn't follow the agreement, everyone is in for a world of hurt.
The main awesome thing about stuff like this is that user-extensions can (where applicable - I'm looking at you scumbag chrome android) modify the user experience at the user's pleasure. Send the toast messages to some other side channel, or just hide them altogether.

Strong personal bias but modals always felt like a huge sign of weakness, a narrowing of complex situations to make UI design easier/simpler, but not necessarily better.
A hell of a lot of Americans are descendents of literal frontiersmen and women who were subjected to cow thieves, roving gangs, robberies, Indian raids, revolutions, European proxy wars, dust bowls, depressions, and terrible diseases (with no cures in their day) hundreds to thousands of miles from the closest forts or cities.

Yeah no one in the country doesn't own a can opener AND a shotgun or rifle AND know how to can their own veggies. But if you don't have peace, you aren't going to be able to farm in peace.

Do you even have acreage to farm? Do you have tools and skills to farm? If not, how would you get access to it?
You mean, a morning review routine?

Yes, in all my pre-diploma and summer jobs (not computer related: administration, woodwork, music publishing office, energy engineering office) in the 90’s.

Only, never as dogmatic and unproductive as the ones I’ve seen as scrum/agile mandated stuff.

Just a natural human focused status checking.
Being a billionaire in America transforms the person in some kind of super hero. People are mentally incapable to disagree with them, even on basic topics that everyone knows are wrong. The media exploits this every day.
He's certainly not focused on minimizing carbon emissions - otherwise he wouldn't be making a luxury car company

He exploited tax incentives intended to reduce carbon emissions, sure, but that's not the same thing as focussing on reducing carbon emissions
I agree from a technological/logical standpoint but from a human/emotional standpoint I think there are issues. People love to complain/go to press about HR decisions. The press would love to portray Musk as hypocritical about treating production line workers as second class denizens. The headline writes itself.

I tthink the way you overcome it is by paying a premium to in office staff. In your scenario - onsite nurses likely get paid a very significant premium to the salary of a remote, phone nurse.

But in the "Elon" case its going to hard to show how your are paying a premium for onsite factory techs when the design engineers are making $$$ plus RSUs.
Having recently spent time watching many whales breaching well into your airspace, I wonder if you have any thoughts on avoiding right-of-way disputes with charismatic megafauna?
This is a rather weird point to make, considering the way Wikipedia has handled the collection and usage of funds. Investing in its technical infrastructure would be a less controversial usage of their money than what they are doing currently.
If Amazon could replace workers for 1/4 the price then why haven't they done it already?
Apparently, boiling one in clarified butter works OK:

https://www.youtube.com/watch?v=N1Biy5776ec
The amount of class warfare here is hilarious. Will you really parrot elite coined phrases like “laptop class?” What even is the laptop class? It’s a meaningless term created so it’s easier to dehumanize a group of people that have a massive salary range. There are WFH making $50k. They are not all vest and resters.

Do we really need to make the conversation be about the upper middle class being “lazy” and completely ignoring the fact that these RTO orders come from billionaires?
Yeah seriously! I was riding beside a river (for fun) listening to the interview and it came to that section, he said it and I laughed out loud... thought what a weird perspective. I'd never considered that before and there was a bench over looking the river a ways down so I pulled over, sat on the bench and debated it with myself for a while... let myself agree in one direction... see how I feel, let myself agree in the other direction, see how I feel. Just think it back and forth with myself for a while. As the other reply to this comment said, it's a great deal of fun to do this and I enjoy it a lot!
The same way that betting on a horse doesn't make it go faster, there is a one way relationship between companies and their stock
Young people all have 6 roommates with somebody turning the living room into an extra bedroom because rents are too high.

The isolation of being at home is countered by how many people are at home
Only a few people get it. You are in the vanguard, my
friend
CAFE was written in a way that protected UAW workers jobs at the expense of fuel economy and the ability to make and sell small cars. In the end, the only vehicles that were still profitable for the big three were vehicles riding on truck frames. Unintended consequences strike again.
does this mean we can get cheap containers?
Oops, only now do I realize that should have been “we never quark their wibbles” and “a client wants their wibbles unquarked.” (Hopefully doesn’t make a difference to comprehension since they’re nonsense words anyway, but there you go.)
The M112 18-valve V6 is not really considered the pinnacle of M-B gas engine tech; and the interior is certainly not what lots of people expected from a Benz, certainly if you had just stepped out of a '94 500E W124. The body when prepped right is comfortable and sturdy tho. I'm sure it was heavily used in much of the world as a taxi, like a Camry (except with an M-B turbodiesel).

A good endorsement of it is that the Chrysler LX cars, which shared more than a few W210/211 bits, have been on sale with only cosmetic updates since 2005...
That's not how most collective decision making goes though. You just make the rule and lets the costs be what they'll be. If you could omnipotently know the individual costs of laws people would oppose all of them -- real purchasing power is horrible right now, taking anyone's discretionary income for anything will be wildly unpopular.
Heh, I was just trying to picture Some Puerto Rican Guy on a bike, cycling into rich white neighbourhoods.

https://southpark.fandom.com/wiki/Some_Puerto_Rican_Guy

The only criminals on a bike that I'm aware of are bicycle thieves.
Oh, they are great products for consumers, but not for providers. Somehow they even out the rate for 30 years…but that doesn’t really work out so someone is caught holding the bag (owning loans whose interest rates are way too low vs current rates). Countries like Canada simply don’t have them (you can get one for 25 years, but at a rate that is pretty high compared to adjustables).
Show HN: I was being Drowsy when I drove my Car – Twilio and OpenCV
"40% of the on road accidents occur due to Sleep depravity"

Well, this is just crazy. So, I wrote a code helping drivers to not fall asleep as well as informing their 'emergency' contacts about how many times their loved person has fallen asleep or was about to fall asleep while driving.

The project has a room for lots of improvements. Your contributions are welcomed.
Outside of some mass extinction event where we'll probably be dead anyway, I never really understood what exactly the scenario this would be prepping for. Even if all of societies institutions collapsed overnight, the winners are going to be whichever group reorganizes itself quickest. Not to mention some group is likely to inherit a ton of heavy weapons that aren't going to just disappear from the planet overnight.
At any given time, yes. My thoughts above are somewhat rambling, but what I found most interesting was the adaptation over time. I'm sure that capacity for adaptation is variably effective amongst different people as well.
Being unable to repair & maintain cars, non delivery of long ballyhood features... There's a list of problems that show up as you move from a new hot maker to a been-around-for-some-time company.

But also yes, that guy too.
Chip war: Apple strikes major US-made semiconductor deal
Doesn’t housing have a better risk adjusted ( sortino) ratio than all other asset classes?
Makes sense given his position. He has a vested interest in his workers coming in.

He, unlike others, can afford to be extremely close to the office. There's all benefits and no detriments for his statements.
Nope. There too. It turns out that foot traffic and bike traffic do a lot more for small shops than cars do (because drivers go fast enough that they don't pay attention to 90% of the small shops).
The term that applies here is "petite bourgeoisie". They are nominally rentiers, but without the scale of economic control given to the actual bourgeoisie by their relationship to the means of production.
There's no actual difference in interteam coordination when all the teams you have to coordinate with are far away in timezones, or distance.

Calling a conference room from another conference room is strictly worse than a laptop to laptop call
No, it means you can ship cheap trash from point A to point B, but people on either end will be too poor to buy it.
me gusta neovim
a. Some guy got sick of his wife's lover.

b. Some warlord decided to turn his rival's bones into trinkets for British archeologists to find.
ye olde two hard problems in CS, naming and cache invalidation (and off by one)
The answer is to improve the lot of factory workers, not worsen that of the others.
Harmonic is the best HN client I ever used.
Because we can be sure that all the while -- companies are using their resources to engage in potentially anti-employee and employee-undermining activities. And in some cases not just potentially, but manifestly.

So it's to level the playing field, as least a tiny bit.
My favorite version of this observation is asking tenured professors how they picked up their sweet digs. Often the answer is "I knew I would be here indefinitely. I saved cash until the bottom fell out. I bought."
Perhaps of parallel interest I saw this GitHub repo which is a translator from CANbus to modbus to enable the use of used/salvaged Leaf EV batteries with stationary inverters from Fronius.

https://github.com/dalathegreat/BYD-Battery-Emulator-For-Gen...

The author has an introductory video about it.

https://www.youtube.com/watch?v=XHZWGLzT7gg
You are paying and also be part of the experiment, that is the best deal ever.

It replaced almost completely google (re)search engine for daily knowledge support.
I think the trend is away from semantic tokens, and away anything that is remotely like 'feature engineering'. Maybe even they will just use character level or byte level encoding in the future. They have used it before, and probably something like it will be used again if the Rich Sutton 'bitter lesson' continues to be true.
You can refuse to work for such an evil employer...
As others are saying, this isn't fingerprinting for parents, or for attending school events -- it's for applicants to volunteer at the school.
Potential problems:

1. It is extremely dangerous. Anything in the beam path is dead or seriously injured. So it’s only applicable in fringe applications, such as microwave communications links that are already in use.

2. The technology is a solution chasing a problem. Electric grids exist in the most densely populated areas. So it would be applicable in rural areas; but only where the cost and maintenance is acceptable.

3. Both microwave and lasers are affected (attenuated) by free space loss and additional loss due to atmospheric effects.

Potential opportunities:

1. Ideal for satellite or balloon power recharging.

2. Ideal for satellite and/or UAS disruption weapon.

3. Ideal for electric aircraft in-flight recharging.
Yes. The alternative is absurd. Should it be mandatory for everybody to work shifts just because some people have to work shifts?

I’ve never met a factory worker who resents the existence of cushy WFH jobs. On the contrary, many aspire to have such a job.
it isn't recursive in the sense of a function calling itself, just in the mathematical sense of using its own previous output as input

i agree that it's easy to better its performance substantially, but i wouldn't go so far as to say 'perform terribly', except in the sense that it's far from optimal. there are applications where it would be adequate
But they are used for commuting.
There's a Sopranos episode called “Remember When”, in which Tony gets mad at Paulie for living in the past: ‘Remember when is the lowest form of conversation’.

Later, he talks to someone else about it, who points out that Paulie has nothing in his life now, and that Tony should just accept it if he wants to dwell in times that were more meaningful for him.
> from a human/emotional standpoint I think there are issues

From an emotional standpoint, sure, but not from a moral standpoint, which is what your previous post was saying. There is no moral case for punishing an advantaged class if that punishment does nothing to help the other class.