skysight-inc/HackerNews huggingface dataset:
Some thoughts...
1. There is no tech stack in the world you can safely deploy and forget about for months or years. Security updates are needed, OS updates cause changes, browser technology changes, and your users change. Even setting aside security, let's say you build something that becomes popular, you've no IC lock-in, and you don't add features / adapt. Sooner or later someone is going to create a clone that does add features / adapt and you will start to lose business.

2. Adding new features is rarely 'easy', because you have to test it with the other features, do deployment, support older versions of the app (if possible), etc. Still, some stacks make it easier than others.

3. Simple deployment once you've configured your automation is an area where there has been lots of progress. You can look at Puppet, Chef, Docker containerization.

4. Dependencies change. Typically you should view changing dependencies like you would adding a new feature: have a justification for it, do full testing, and only then deploy.

5. Open Source is easy to find, and there are a lot of good quality candidates.

6. One commenter recommended to use what you know. Within reason, that's good advice. However, the old adage of 'when all you have is a hammer, everything looks like a nail' applies. Use the right tool for the job.

7. For the front-end, I recommend client side JavaScript, either Lit or React using Vite. Svelte shows promise, but is not fully mature enough yet to be acceptable when you're solo and using it in production.

8. For the back-end, I recommend Rust. It's reliable, it's performant, and there is a good ecosystem for it. If you go that route, I recommend Axum or Tide for your Web server, fronted by Nginx. You could also go NodeJS (or Deno) and Express or Next. A JS back-end though means you're at the mercy of the NPM ecosystem, which is rife with abandoned packages. Rust could get like that, but seems to be more well pruned. If you go with Rust and want easy new features, a REST based API using a plugin architecture can work well.

9. I recommend 3 repos: front-end, back-end, devops. Your deployment scripts and other things go in devops. Your front-end, if you follow 8, will be Javascript. Your back-end, if you follow 8, will be Rust.

Btw, Rust comes with some caveats. If you don't know it already it has a steep learning curve, however the benefits of learning it are high. You also can use an existing hardened version of Rust made by Ferrous Systems, called Ferrocene (I've no relation with them, other than maybe wanting to work there someday).

https://ferrous-systems.com/ferrocene/
> Parameter != token?

They are two different things. A parameter is not what programmers call "parameters/arguments", it's the total number of weights from all of the layers in the network.

A token is a number that represents some character(s), there is a map of them in the vocab list. Similar to ascii or UTF8, but a token can be either a single character or multiple. For example "I" might be a token and "ca" might be a token. Shorter words especially might be a single token, but most words are a combination of tokens.

> This sounds to me like it is choosing from 1 of 32k tokens when it is scoring/generating an answer. [...] I would have thought it is picking from 1 of 7B parameters?

The model doesn't do that itself. The last layer of the model maps it to a single list that is 32k long, and outputs that of tokens and each ones probability (the combination of a (token, probability) is called a logit). So the output is a list of logits.

The step after this is external to the model - sampling. You can choose the highest scoring token if you want (greedy sampling), but usually you want to make it a little random with either one of the top 20 tokens (top_k) or from the best 90th percentile tokens (top_p). There's other fun tricks like logit biasing.

Finally you map the tokens to their respective values from the vocab list in my previous comment.

> Where does the 7,000,000,000 parameters come from then?

Here's a walk through for how to count the parameters for Llama-13B. It's worth noting that most of these models are commonly rounded (that's why you might see llama-33B be called llama-30B).

https://medium.com/@saratbhargava/mastering-llama-math-part-...

https://github.com/saratbhargava/ai-blog-resources/blob/main...
> I am very worried about Evernote, and this is another clear sign that Evernote just doesn't "get" it.

My interpretation is that they believe the product can't compete for new customers. They're trying to suck as much money out of (a very large number of) paying legacy customers as they can before they shut the service down. They don't care about free users, since they don't expect new customers anyway. They implemented big price hikes a while ago.
Programmatic and multi-step use cases. If you need chain-of-thought or similar, tool use, etc. Generating data.

Most use cases outside of classic chat.

For example, I made an on-demand educational video project, and the slowest part was by far the content generation. RAG, TTS, Image generation, text rendering, and video processing were all a drop in the bucket, in comparison.

It would be an even wider gap now, and TTS is super-realtime, and image generation can be single step.
So what do you do with your combinatorics, math, and logic skills that you got from reading?
> A better eval is to use actual scientifically tested psychology test on text (the native and strongest domain for LLMs), for example the sort of scenarios used to gauge when children develop theory of mind (“Alice puts her keys on the table then leaves the room. Bob moves the keys to the drawer. Alice returns. Where does she think the keys are?”) which GPT-4 can handle easily; it is very clear from this that GPT has a theory of mind.

Aren't you confusing having a theory of mind with being able to output the right answer to a test? Isn't your proposed evaluation especially problematic because an "actual scientifically tested psychology test" is likely in the training data along with a lot of discussion and analysis of that test and the correct and incorrect answers that can be given?
Being a skeptic is not a problem in science until the hypothesis includes the end of the world. Nobody cares if someone questions fluid dynamics, airplanes still fly and boats still float, but climate change is not something that we want to test... and therefore the real problem here is being a skeptic on a hypothesis about the end of the world that will necessitate a specific direction for humanity.

I'm not judging either way. I'm making an observation on the meta-narrative. I think claiming there is no danger and using pseudo-science is just as bad as claiming that an existential risk is already settled science and therefore questioning it poses a threat to humanity.
Or there are enough of those examples in the training set that it can guess well. Not sure how such an example would prove anything when we know an LLM is just guessing the best words.

Nothing I’ve seen shows evidence of any sort of abstract concepts in there.
and that 30 minute offset makes quick mental timezone math doubly hard and error prone. What is the benefit of merging timezones?
those companies will happily outsource the development of their core competitive advantages to another company who does appreciate its value; they will then sell it to their competitors
Relevant john oliver link - https://www.youtube.com/watch?v=2wSarEVgjM0
Someone once said, there is accidental function, and deliberate function.

If your system doesn't work, and you just plonk around at values, until, very surprisingly, the system starts behaving well and you the call it working... well it might be working now. But it's just accidental correctness. As soon as something causes the system to bank left, something's gonna break and no one knows how to fix it - and you're back to square one.

On the other hand, as hard as it is, if you can clearly tell why your fix will restore function to the system without even applying it, you have deliberate correctness and function. If done right, it is very boring, because exactly and only the expected thing will happen. You should know about the unknowns and plan around those as well, so even if an unknown bites you, it's a known and handled unknown. This can be exhausting to make happen, because it is much harder, but those systems will just work.

But this is a fight I have with some development teams probably forever. "But we poked at the values, and that stopped the flames. It is fixed!" "but why?" "Dunny. But no fire anymore. All good." And then 2 weeks pass, and there is more fire and everyone is like "Oh but why would this happen? How should we have known for this to happen again"
Well one could always grab his "BBC BASIC (86) Plus for MS-DOS" from his site (https://www.bbcbasic.co.uk/bbcbasic/dosbasic.zip), and run it in some from of "DOS Box".

As his page about it states:

"BBC BASIC (86) Plus is an implementation of BBC BASIC for PC compatibles running MS-DOS™; it has been designed to be as compatible as possible with Version 4 of the 6502 BBC BASIC resident in the BBC Micro Master series."

(https://www.bbcbasic.co.uk/bbcbasic/bbcdos.html)

That should then give you something close to the form I used on my Model A back around '83.
:( I would've tried to determine the cause of the crash with that specific file in my off time, provided that I could isolate the code in the Grammarly extension in Chrome.

The main reason –other than curiousity– is to ensure that a future regression (in Chrome/Grammarly) wouldn't lead to it again.
The main thing is chat is just one application of LLMs. Other applications are much more latency sensitive. Imagine, for instance, an LLM-powered realtime grammar checker in an editor.
I mean, we have at times considered rats to be part of a literal plague, or in certain contexts where there are vast numbers of them, they've been called a plague metaphorically. But rats in general are not a plague in the context of the whole world, so for a number of reasons this doesn't follow.
Africa Does Not Exist
Well, they can certainly take our browser to another level: the ground level.
Maybe the name was odd `loader-spinner.gif`
Guide to fast websites with Next.js: Tips for maximizing server speeds and mini
If you can clear a medical debt with 0.7 cents on the dollar, how is this system nothing more than a big scam?
> Or is this some sort of recency bias at play?

Potentially it is recency bias. Like when you learn a new word, and suddenly see it everywhere. Also potentially it could be something more troubling, like being infected with malware. I don't know your setup or how vigilant you are, so it's impossible for me to just say you're infected or are being tracked.
It’s a bit more nuanced than that. Scalar code that does lots of branching and loads one value at a time (e.g. single character) often can’t hit memory at peak bandwidth rate. On the other hand well vectorized code that loads vectors of multiple values at a time can properly saturate the available hardware resources.
But Spain being on CET is absurd too. They basically run on GMT anyway, it just shows a later time on the clock (that's partly why Spanish people eat dinner really late).
> Amazon has chosen letting good people go

I work at Amazon and I can tell that it depends on the organization and politics.

Of course nobody is untouchable when it comes to the layoffs, but quite many people have made their spot pretty much resistant to firing. They are crucial for business to continue. (Again, maybe my organization)

Also, it is very common at Amazon; Nobody is doing the job they are hired for. As a developer, you are responsible for anything and everything. Management only deals with PIP and crunching out weekly/monthly/quarterly documents where their management doesn't give a damn about except the summary part.
Yeah, to be fair my tone didn't come across great there, I'm being slightly sarcastic :)

I've got two young kids (youngest 4 months) and I spend pretty much all my free time either playing with them, cooking, cleaning and life admin. I've only recently started carving out time again to do plural sight courses etc at home. And to do that I'm having to reduce time lifting weights in the garage (which was only about 1.5 hours a week). In fact I should be doing a plural site course right now!
I spent years of my life writing multiplayer games in Unity for the web. It's a terrible platform, and if you have any alternative at all, just don't do it.

One update broke the loading bars in Chrome (which will cause 80% of users to think the title is broken and leave), and I ended up having an argument on their forum with one of their developers about whether it should be considered a regression:
https://forum.unity.com/threads/changes-to-the-webgl-loader-...

The web platform is not a priority for Unity. They'll add big ticket features that make for blog posts, but it's full of bugs, builds slowly, requires a ton of tweaking and is all around just a bad time.

If I could do it again, I'd most definitely run with a js engine instead.
I hope the same is done for every US president who have all been complicit in an unethical military industrial complex at the expense of tens of thousands of lives and millions of people displaced from their homes
providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward.

This is why I have started using Fortran for writing AI model inference code. It natively handles array manipulation like python and has a `matmul` intrinsic and compiles into fast code. There are some rough edges, but it's great as a low level matrix programming language which was its original point.
Jay walking laws are slowly being walked back in some places.

The history of such laws are interesting, by the way.

That said, most of the time and in most places I've been, the de facto reality is that it's essentially lawless. Neither pedestrians nor drivers are being policed effectively and they all kind of do whatever.
"Docs in the military have a bonus structure based on their specialty area that's supposed to at least try to keep their pay from being ridiculously behind."

Tries is the key word there. Most of the time it's not even close unless you're a family practicioner or other lower paid specialty. On the other hand, there is a retirement program (although that's based on base pay and not as great for roles like this) and they don't need mal-practice. At least to me, it seems the biggest way they get docs is to pay for their med school if they commit to 8-12 years or whatever it is for them now.

"To bring an O-5 cyber person in, they would be expected to have enough experience as a cybersecurity professional to rate that rank."

I mean, that makes sense in a general sense, but doesn't really make sense the way it works now. There are plenty of people with experience that if they were external would come in at a higher rank than they are now. I'm not sure how you would design a good system for an industry where people jump companies every 2-3 years, are expecting high pay (at least O5 with 10 years), are used to rapid promotions, and are used to lots of perks. I mean, you could have someone with 5 years of internal experience with a great record be something like O3 and have someone external be called a senior analyst/dev or lead in 3-5 years and come in as an O5 with basically no difference in duties or performance.

Although I'm not sure how you would even measure performance in that role. The industry doesn't have clue how to measure it - just look at the article or all the stuff about interviews and leetcode. The military might set some more objective standards, but they're not going to remotely apply to external candidates.

I don't know, I guess in general I would imagine the best thing for stability, readiness, and even morale, would be to train and promote from within while focusing on retention. It sounds like they're already providing bonuses for all levels and saying they're focusing on retention, but clearly they have other issues. I doubt bringing people in at a higher rank will fix that.
Show HN: Collider – the platform for local LLM debug and inference at warp speed
ChatGPT turns one today :)

What a day to launch the project I'm tinkering with for more than half a year. Welcome new LLM platform suited both for individual research and scaling AI services in production.

GitHub: https://github.com/gotzmann/collider

Some superpowers:

- Built with performance and scaling in mind thanks Golang and C++

- No more problems with Python dependencies and broken compatibility

- Most of modern CPUs are supported: any Intel/AMD x64 platofrms, server and Mac ARM64

- GPUs supported as well: Nvidia CUDA, Apple Metal, OpenCL cards

- Split really big models between a number of GPU (warp LLaMA 70B with 2x RTX 3090)

- Not bad performance on shy CPU machines, fast as hell inference on monsters with beefy GPUs

- Both regular FP16/FP32 models and their quantised versions are supported - 4-bit really rocks!

- Popular LLM architectures already there: LLaMA, Starcoder, Baichuan, Mistral, etc...

- Special bonus: proprietary Janus Sampling for code generation and non English languages
FYI you can right-click and “copy as svg” for cross app pastes, or cmd+shift+c to copy as png.
I might finally switch back from Iceraven! Proper extension support was the only reason I switched away from the official Firefox for Android in the first place.
I work with https://nanos.org && https://ops.city - we can run thousands of these on commodity hardware.
If you want to be able to edit your add-ons/extensions you'll want the unbranded builds instead: https://wiki.mozilla.org/Add-ons/Extension_Signing#Unbranded...

Just watch out that Mozilla has disabled updates in these builds so you have to manually download for each new version/security fix.

The normal Firefox releases (except for the unstable alpha/aurora/developer version) don't allow you to edit your extensions without getting permission for each change from Mozilla's security theater automated signing portal.
Apple's smart ring technology may not be limited to fingers
What I don’t understand is why hospitals will sell debt for pennies on the dollar to debt collectors, but won’t make the same offer to patients. If they give a patient with $100k of medical debt an offer to pay it off for $1500, I bet they’d clear a lot more debt.
Recently there has been a trend in calling models with weights and code available "open source" even if the training data is not available. For safe deployment in health care and other safety critical fields, transparency on the training data and process are vital too, which means developing clear terminology for models full transparency! Even this article title suffers from this ambiguity.
Firefox's Android browser will have proper extension support again real soon
I moved to the US more than 12 years ago, and while I have become almost completely “Americanized”, “medical debt” is still something I simply cannot wrap my head around. It makes absolutely no sense whatsoever.
It's called marketing and learned helplessness. Some other keywords are doomscrolling and thirst traps.
One of the rare questions in a headline that you can answer with certainty with yes...
Let's imagine there was a 20 year old who dropped out of CS and spent the next 2 years reading Knuth and entered the work force at 22. Can you see any way that student would not be succesful?

I know someone who did something very similar (but in math) and jumped into a grad math program at age 17.
Summer time makes plenty of sense as-is, especially at fairly extreme latitudes like the Netherlands. Pithily stated, it is about keeping the wall-clock time (meaning the clock everyone looks at, the Schelling point in a society-wide coördination game) approximately aligned with sunrise/dawn. That way you always have sunlight in the morning, but the sun never rises too early making you lose sunlight in the evening.

> ...opposed to when they're TIRED (in the evening)

Morning darkness is way more dangerous. You don't want to have dark mornings when people are in a rush, commuting to work.
The US is notorious for special treatment by the politically connected. The lack of justice and accountability doesn’t make the war criminal allegation less accurate
Before anyone jumps to it, I want to state that what she did was very kind. Yes, it's not her money directly. Yes, the situation with medical care in the US sucks and this shouldn't need to happen. But don't let those take away from the fact that the action did more good than many people will put the effort into doing.

Anyone can point out that the system sucks. It's not clever nor is it hard.
If you found a bike tire this amazing, I can't wait for you to discover _anything even slightly more complicated than an inflated tube inside a rubber/cloth shell_.

This reads like someone just came back from outer space and was stunned by the complexity of all of the systems that got them there. I'm all for appreciating the little things, but we have to draw the line with overdramatization somewhere.
In Germany, it is literally the authorities and conscientious journalist, who live in fear of being sued by the criminals, rather than the criminals being in fear of being sued themselves.

It is agonizing to watch.
> It's not. They don't realize it, they're merely referring to stopping your internal monologue.

They certainly have realized that. It's one of the first things you notice doing awareness meditation; thoughts appear from nowhere even if you didn't try to think them.
If my signed integers overflow, that is wrong.
I want to be able to detect if it ever happens, not have that be defined as perfectly valid wrapping behavior.
If it's undefined, it's perfectly legal for the compiler to make overflow trap, so I can catch it, or confirm it never happens when running tests our debugging.
`-ftrapv` or ``-fsanitize=undefined` do this.
Bitcoin's thirst for water is just as troubling as its energy appetite
https://azsoaring.com

Maricopa had a bunkhouse, not sure about now.
Foxconn and other Taiwanese companies have always had issues with hiring workforces in countries without a significant Chinese speaking community.

Unlike Korean and Japanese companies, the Taiwanese companies employees would have a a weaker grasp on the local language or English and remain isolated and aloof. I remember visiting India as a kid fairly often, and Korean and Japanese kids would attend my cousin's normal English speaking Indian middle class elementary school, while the Taiwanese community had their own Chinese speaking school they attended. The Japanese and Korean companies like Nippon Sharyo, Samsung, and Hyundai would actually train locals from assembly to R&D to management but the Taiwanese companies couldn't care less.

It's the same in Vietnam as well - Koreans and Japanese will try to explore Saigon and mix with locals, but Taiwanese and Mainland Chinese will stick to District 5 (the Saigon Chinatown).

A good example is Foxconn in Mexico vs Vietnam. Foxconn Mexico has been facing similar problems as Foxconn India due to the lack of a local Chinese speaking community to hire from, but Foxconn Vietnam has been benefiting from hiring local management from the Hoa/ethnic Chinese community in VN or bringing in ethnic Chinese from MY or SG to manage operations. They even discriminate in China [1]

This is the same reason why Foxconn became a significant manufacturing player in China compared to similar Japanese and Korean companies - being able to source local leadership in Guangzhou is easy when most Taiwanese have familial and cultural relationships across the Strait.

Apple seems to be annointing Tata as their Indian Foxconn from the looks of it as well [0], and it makes sense. The same way Foxconn could open doors in China, Tata can in India.

[0] - https://www.reuters.com/technology/tata-make-iphones-india-a...

[1] - https://www.theinformation.com/articles/some-apple-suppliers...
Leadership Styles Every Manager Needs to Know