skysight-inc/HackerNews huggingface dataset:
I love Libby and the fact that I can get free ebooks through my local public library.

But the model is just kind of broken and never made any sense. Even the idea of "loaning" ebooks never really made any sense. The operation of "lending" an asset has two parts; once where it is given, and one where it is returned. Given a medium where perfect copies are essentially free, it is easy to "give" an ebook, but incredibly difficult to "take it back".

And the fact that it's associated with a local library is just kind of a silly fiction to keep friction artificially high.
Yes, we only use this encoding when it brings benifits. `wire format` is a good term to describe such cases, which the context are constrained to current message only. So the data for compression are small, and many statistical methods won't work since it's a `wire format`. The stats itself must be send to peers, which will have bigger space and cpu cost
They’ve all changed massively in other ways during the nineteenth and twentieth centuries too as linguistic and cultural diversity began to significantly dwindle under “modern nation-state” style cultural policies, try finding young Oc speakers in the southern half of France these days, followed by the collapse of multinational states (mainly the Habsburg state of course) followed by brutal ethnic cleaning and genocide that’s still going on in old pockets of that ex-empire today.

These places have experienced continuous and significantly rapid linguistic and cultural change for centuries now. If anything high levels of literacy and experience of integrating minorities by force over the last centuries might be making this latest set of changes fairly gentle and manageable…
How does this compare to the official Unicode compression schemes? [1] [2]

[1] https://en.wikipedia.org/wiki/Standard_Compression_Scheme_fo...

[2] https://en.wikipedia.org/wiki/Binary_Ordered_Compression_for...
Blog link (Feb 2024): https://research.google/blog/a-decoder-only-foundation-model...

Previous discussion: https://news.ycombinator.com/item?id=39235983
It would just stress me out. But I guess everbody is different.
The kinkshaming is not making your viewpoints sympathetic.
> This is an insane way to legislate, so at best, it makes me think of underlying issues like corruption and incompetency.

To me this feels more like saying "do not be hermetic and opaque when creating society-altering tools" which is a point worth considering.

A subtext to this discussion might be the slavering horde of managers who can't wait to jam AI into critical aspects of life.
Not sure why you're downvoted. Five minutes for a 500 Free is a pretty typical time for boys on highschool swim teams.
We will get better with understanding black boxes, if a model can be compressed into simple math formula then it's both easier to understand and to compute.
There needs to be some overhauls to fix the damage Disney et al. did to our copyright system. The Copyright Act of 1976 and later Copyright Term Extension Act did so much damage to the concept of public domain. Being able to own exclusive control over a work for up to 120 years is insane.
I actually just refer to myself as a Digital Janitor these days, even though I'm hired as SRE. Funny enough, it's met with much more understanding, and it is ironically kind of fitting as well.
Stretches of freeway in Bakersfield and Santa Clarita have been under construction for decades.
Technical positions is more than software engineers. 60,000 software engineers was the most I saw. And most other technical positions pay less.
Marketing vs. Reality :)
look for polars, https://pola.rs
Winning on press your luck is probably not a scam. Punching up his story to get on press your luck might be a scam. Selling shares in a fraudulent lottery was definitely a scam. Opening checking accounts with assumed names is definitely a scam.
I work with pyspark and parquet quite a lot. I never had to deal with parquet outside spark, but this is how I would do this:

- Write a pandas_udf function in pyspark.

- Parition your data into smaller bits so that the pandas_udf does not get too much data at the same time.

Something like:

```

from pyspark.sql import SparkSession

import pyspark.sql.functions as f

@f.pandas_udf(return_type=whatever)

def ingest(doc: pd.Series):
# doc is a pandas series now

# your processing goes here -> write to DB e.t.c

pd_series_literal = Create a pd.Series that just contains the integer 0 to make spark happy

return pd_series_literal

spark = SparkSession.builder.getOrCreate()

df = spark.read.parquet("s3 path")

df = df.repartition(1000). # bump up this number if you run
into memory issues

df = df.withColumn("foo", ingest(f.col("doc_column"))

```

Now the trick is, you can limit how much data is given to your pandas_udf by repartitioning your data. The more the partitions, the smaller the pd.Series that your pandas_udf gets. There's also the `spark.sql.execution.arrow.maxRecordsPerBatch` config that you can set in spark to limit memory consumption.

^ Probably overkill to bring spark into the equation, but this is one way to do it.

You can use a normal udf (i.e `f.udf()`) instead of a pandas_udf, but apparently that's slower due to java <-> python serialization
Brings a new meaning to "Charlie don't surf". At this point though, you have to pretty much imagine that the military has researched any and every mode/method/means of achieving the goals of a mission. I'm sure roller blades and skateboards have been considered at some point as well.
I was contemplating to upgrade to Windows 11 but I took the plunge and moved to Linux instead. No issues whatsoever - the games I play work with Steam compatability mode or just natively. The work I do is mostly in the browser.
It's incredible how much work went into porting the concept of scarcity over to the digital realm. Its pretty annoying if you want to read your book because you are offline, but can't open it because the app can't phone home.
He isn't stating a fact, from what I can tell. The actual quote in the article is "I don’t think it’s crazy to believe that half the white-collar staff at Google probably does no real work."

It's not controversial to think that there is likely waste at Google. This particular article, though, has no insight in it. It's an inflammatory statement with no conversation or intelligence about it. Maybe there's something more in the Substack they reference.

Then again, I don't think it's crazy to believe that 50% of Substack newsletter content is full of nonsense /joke
That's not his argument.

> "The company has spent billions and billions of dollars per year on projects that go nowhere for over a decade, and all that money could have been returned to shareholders who have retirement accounts."

He would rather that Google focuses on Ads and cuts the rest of the company.
Great UI/UX and very nice work!

I just tested it by typing "llama cpp gpu support" that's it.

Flawless instructions for Python, but when I followed up with

"in node"

It didn't know about node-llama-cpp. Is there a general knowledge cutoff, and/or is loading developer-specific stuff a manual process?
Hmm, in the earlier AlphaFold 2 paper they state:

> Input mmCIFs are restricted to have resolution less than 9 Å. This is not a very restrictive filter and only removes around 0.2% of structures

NMR structures are more than 0.2% so that doesn't fit to the assumption that they implicitly remove NMR structures here. But if I filter by resolution on the PDB homepage it does remove essentially all NMR structures. I'm really not sure what to think here, the description seems too soft to know what they did exactly.
Without doing the math, 80% staff reduction (~6500 employees) for 22% revenue decrease sounds pretty darn good to me.
I haven't deny your statement. Can you read my statement again? Where are you from?
You seem to be missing the concept of consent in your analysis. I’d say it’s pretty foundational.
Based on that tweet, surely you mean that it will be available six years ago?
Yeah, so the government can track you, and really anyone who gains access to the signature and trust chain can track you. The trust anchor also has to verify your identity to verify your age in order to issue the PEM file.

So to answer my original question - no, you can't anonymously verify age. Someone has to verify your identity (a central authority in my comment, which in your system is a trust anchor) and your signature can be tracked back to you (as a fearure).
So does that mean you feel programming language design peaked in 1972?
VCs have a lot of vested interest in getting more engineers out of companies like Google for working on their own startups. It might well be true the productivity rate is low for Google employees, but they also need to look at how much money the company is making per engineer and that number is very high - much higher than any pre/post IPO startup.
My argument is: weather.

I think it is fine & better for society to have applications and models for things we don't fully understand... We can model lots of small aspects of weather, and we have a lot of factors nailed down, but not necessarily all the interactions.. and not all of the factors. (Additional example for the same reason: Gravity)

Used responsibly. Of course. I wouldn't think an AI model designing an airplane that no engineers understand how it works is a good idea :-)

And presumably all of this is followed by people trying to understand the results (expanding potential research areas)
Yea good point, you can use (test@test.com and 111111) if you don’t want to sign up.
Adobe alum takes AI-enabled marketing beyond the prompt
You should reply to the person you are talking to for them to even have a chance of seeing what you said.
I guess this is a good thing?

If a lot of people in a company do no real work, that means there is potential to trim down the workforce, decrease costs and still get the same amount of stuff done?

On the other hand, people said the same thing about Twitter. Elon Musk bought it and fired most employees. Indeed, Twitter works kinda similar to before. But revenue is said to be down 50%.
Copilot does kind of suck, it’s incredibly slow. I switched to Supermaven and it’s significantly faster at suggesting and the suggestions are at least twice as good, at least on my project.
I'm a Tesla owner and want Elon fired because I want my car to be supported far into the future. I don't want him to tank the company like he did Twitter.
I heard or read a tasteless old joke (maybe on Gawker as I can't find it now) about a VC complaining you could lob a grenade into Twitter's SF offices after 5 pm on a weekday and no one would be hurt or even notice.
The thing is, most storytellers don't have nearly as compelling a value proposition. E.g. the best value a fiction writer can provide is a good story. Your value proposition is selling at least the hope of learning ways of bringing in money. It doesn't need to have a very high chance of success before it's perceived net present value is higher than the cost of your book, while fiction writers are competing with enormous amount of free or already paid for entertainment and fiction from a writer you don't know is often seen as having a sufficiently high risk of being a waste of time that the perceived risk adjusted value for a lot of people is negative.

Getting people to take unknown fiction even for free is an uphill battle.
> investors are supposed to know better.

Part of the reason that public companies are regulated far more harshly than private is that not all investors do; a lot of investors are basically just consumers.
I thought Shorten (.shn) was the lossless codec that came out of that scene?
They're not free but there's not a chance it is a dent in the assumed losses here in advertising alone.
> "It was fine when we did it, and cooperated with intel agencies to do it, but HE shouldn't be able to!!"

He has every right to do it. He promised not to, and you're still claiming he isn't. That hypocrisy is very apparent.
I'm curious why we seem convinced that this is a task that is possible or something worthy of investigation.

I've worked on language models since 2018, even then it was obvious why language was a useful and transferable task. I do not at all feel the same way about general univariate time series that could have any underlying process.
Not the point. Who's leading? That's the point. Not impressed with your example of a corporate behemoth using free labor. That's the wrong kind of Free.
I believe muskrats are declining outside of areas with nutria as well.
I adore technology, but the legal system has given enormous power for Technological Protection Measures to smash property rights & generally exact absurd levels of control. It's enormously saddening to see that which should be raising and liberating humankind, to see our tool-making, used to choke & restrain humankind; what an infernal curse!

Written up two decades ago by Mark Lemley, Terms of Use, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=917926
Bernie Sanders and Ro Khanna reveal bill aimed at eliminating medical debt
Interesting, will explore more.
Not sure about this. Sounds like it's somewhat arbitrarily reframing RPC over HTTP as OS system calls. But maybe I just don't get it.
Yep, going fast with flippers feels great.
It would be cool to see an airplane made using generative design.
"You're an idiot. Play a record."
The big secret I learned to late in life:

The higher up you go in the corporate ladder, the easier your job is. Less stress, more respect, less hours. Decisions you make are hard to tie back to incompetence

The same holds true for companies with too much cash on hand
H. L. Hunley (submarine)

https://en.wikipedia.org/wiki/Civil_War_submarine
Zip2 – Elon Musk's First Successful Startup
This reminds me of a funny interaction I had with one of my best friends and his father. My friend is an industrial engineer.

We were having a big dinner with a bunch of folks at a restaurant downtown after a wedding. My friend must have used the word "engineer" as least 5 times in 30 minutes while talking to some people that we had just met.

My friend's father interrupts him and says: "you keep calling yourself an engineer. You're an industrial engineer. That's like winning $10 after playing a rec league baseball game and then calling yourself a professional ball player."

Awkward moment at first but I think it was absolutely hilarious.
Principal Engineer if software related
Would definitely like if items weren't stuck in the form you got them.

A movie on VHS seems entirely reasonable to convert to DVD or w/e so long as you don't then sell the VHS while keeping the DVD.

Image not being able to frame a poster because it didn't come in a frame to begin with.
The YouTube channel Hyperspace Pirate[0] has been consistently uploading fantastic videos documenting their attempts to make their own homemade liquid nitrogen machine. They don't shy away from including all the technical details, and throw in plenty of hilarious deadpan humor too. Highly recommend them if you're interested in this topic.

[0]https://www.youtube.com/@HyperspacePirate
No AI yet - all algorithms are deterministic under the hood. Although we are considering tinkering with LLMs for query optimization, as part of our roadmap.
VCs who live in glass houses shouldn't throw stones.

Also, go read the letter to future shareholders published at the time of the IPO. They told you they were going to run the company this way.
In my case they do.

But I just use reader mode on basically all websites and that makes skipping the fluff trivial.

Goes for all other websites too - reader mode + ublock origin makes the modern internet bearable.
Show HN: I made DevOps as productized service
I'm normal contractor programmer, for the past few years doing mainly DevOps on AWS. I noticed lot of smaller companies don't have enough work for full-time devops person nor they have cloud/aws knowledge on the team. Lot of them has similar needs. So my thinking is I can help them for flat montly fee to do all the infrastructure on AWS for them. Obviously this is no way to get rich but my hope is to streamline my process and avoid chasing invoices etc. What do you think?
Am I wrong in saying if it was an generous estimate of 1 million per employee there is 6.5 billion, and that's insane generous. 20% being around 9billion... Back of napkin math would disagree..
A new-ish field of "mechanistic interpretability" is trying to poke at weights and activations and find human-interpretable ideas w/in them. Making lots of progress lately, and there are some folks trying to apply ideas from the field to Alphafold 2. There are hopes of learning the ideas about biology/molecular interactions that the model has "discovered".

Perhaps we're in an early stage of Ted Chiang's story "The Evolution of Human Science", where AIs have largely taken over scientific research and a field of "meta-science" developed where humans translate AI research into more human-interpretable artifacts.
Major breakthroughs happen when clear connections can be made and engineered between the many bits of solved but obscured solutions.
All excellent points. The best is when most workloads on the cluster use a temp table with an identical name. Then someone comes along and accidentally drops the TEMP from the their temp table ctas and it gets smashed into the public schema.

“Table already exists…”scheduled job run failure galore, fun times.
If you have a univariate series, just single values following each other -

[5, 3, 3, 2, 2, 2, 1, …]

What is the next number? Well let’s start with the search space - what is the possible range of the next number? Assuming unsigned 32bit integers (for explanation simplicity) it’s 0-(2^32-1)

So are all of those possible outputs equally likely? The next number could be 1, or it could be 345,654,543 … are those outputs equally likely?

Even though we know nothing about this sequence, most time series don’t make enormous random jumps, so no, they are not equally likely, 1 is the more likely of the two we discussed.

Ok, so some patterns are more likely than others, let’s analyse lots and lots of time series data and see if we can build a generalised model that can be fine tuned or used as a feature extractor.

Many time series datasets have repeating patterns, momentum, symmetries, all of these can be learned. Is it perfect? No, but what model is? And things don’t have to be perfect to be useful.

There you go - that’s a pre-trained time series model in a nutshell
>The alphafold work has been used across the industry (successfully, in the sense of blind prediction), and has been replicated independently.

This is clearly an overstatement, or at least very incomplete. See for instance https://www.nature.com/articles/s41592-023-02087-4:

"In many cases, AlphaFold predictions matched experimental maps remarkably closely. In other cases, even very high-confidence predictions differed from experimental maps on a global scale through distortion and domain orientation, and on a local scale in backbone and side-chain conformation. We suggest considering AlphaFold predictions as exceptionally useful hypotheses."
Poverty up from 60% to 70%.

> https://www.reuters.com/world/americas/argentina-inflation-c...

> We have some containers in the back where the garbage is disposed of and when you go with a box, you see 20 people coming up to you to see what they can take as a plate of food to their table

https://www.reuters.com/world/americas/argentinas-child-pove...

Another leader serving the rich and fucking the poor.
But what do VC's do?
In 2019:“ I think we will be feature-complete full self-driving this year, meaning the car will be able to find you in a parking lot, pick you up, take you all the way to your destination without an intervention — this year. I would say that I am certain of that. That is not a question mark." [1].

[1] https://www.businessinsider.com/elon-musk-doubles-down-on-cl...
How to think about the prospects of green growth Essays
It's clickbait journalism 101. Make incendiary or big claim in title to get clicks and then hedge and equivocate and walk-back claim in the article
I guess we can agree to disagree, but I find the 2020 rev Macbook pros have a good number of USB-C ports (2 on the left, 1 on the right -- all can do PD), a magsafe charger, headphone jack, HDMI port and SD card slot. How many USB-C ports do you need? Sometimes I wish there was ethernet but I get why it's not there.

I agree, the butterfly keyboard was shitty but I absolutely love the keyboard on the 2020 rev. It's still not as great as my mechanical desktop keyboard, but for a laptop keyboard it's serious chef's kiss. Also, I have yet to find a trackpad that is anywhere as good as the Macbook. Precision trackpads are still way way worse.

Finally, the thing that always beings me back to MBPs (vs Surfacebooks or Razers) is battery life. I typically get a good 10+ hours on my MBP. Battery life on my old Razer Blade and Surfacebooks were absolutely comically horrible.
There's a fun game idea in there! Imagine having to outmaneuver a constantly learning model. Not to mention the possibilities of using this in genres like bullet hell...
hjskvh
> What I don't get is, why are we still adding more undefined behaviour to the standard?

Why do you believe this is some kind of problem? Can you explain in concrete terms what issue you have with undefined behavior?

> However, in C++, std::unreachable just invokes undefined behaviour when called.

It does, by design and as a testament to great design choices.

> It's not usable as a guard from programming errors, it's just an optimization hint. You still have to write the guard yourself.

I don't understand what point you tried to make. Nothing in your comment has any relation with undefined behavior. Instead, you're complaining that in your personal opinion different languages have similar-sounding features that work differently.

Oddly, you should really read up on std::unreachable, because one of the reasons explicitly stated in it's references is to "trap them to prevent further execution".

To explain what this means in no ambiguous terms, the standard ensures that a) std::unreachable is valid C++ and made available to the world to have the semantics to specify that a code path is unreachable, and b) allow everyone to handle that as they see fit by flipping flags in the build system.

Think about it for a second. You want to call std::abort when std::unreachable is hit. Great, go with that. I don't, and instead I want the compiler to optimize away that code path and anything it would touch, but on debug builds I want it to output a warning and trigger a breakpoint. Have you noticed that all these cases introduce behavior that's entirely different, arbitrary, and implementation-defined? How do you, as a standards committee, get to cover all possible use cases?

By specifying it as undefined behavior.

https://en.cppreference.com/w/cpp/utility/unreachable
I didn't say that sub-par experience was a requirement, I said it was often a part of luxury products. Or, more precisely, I should have said that something being of excellent build quality and offering excellent, top of the line experience is neither sufficient nor necessary for being a luxury good.

It is true though that luxury goods are, often, top of the line as well. Cars and watches are often examples of this. Clothes are a much more mixed bag, with some luxury brands using excellent materials and craftsmanship, while others use flashy design and branding with mediocre materials and craftsmanship.

Exactly where Apple sits is very debatable in my experience. I would personally say that many of their products are far too affordable and simple to be considered luxury products - the iPhone in particular. The laptops I'm less sure about.