MurphyA/hacker-news huggingface dataset:
by wodenokoto | 2025-05-27 17:25:23+00:00
I’m building a poor man’s datalake at work, basically putting parquet files in blob storage using deltalake-rs’ python bindings and duck db for querying.

However, I constantly run in to problems with concurrent writes. I have a cloud function triggered ever x minutes to pull data from API and that’s fine.

But if I need to run a backfill I risk that that process will run at the same time as the timer triggered function. Especially if I load my backfill queue with hundreds of runs that needs to be pulled and they start saturating the workers in the cloud function.
by steeeeeve | 2025-05-27 17:25:24+00:00
If you have an algorithmic edge, eventually it will stop working. If it's a really good one, your trades become someone else's edge. The market is constantly changing and active traders are constantly looking at what's going on and trying to take advantage. The more I looked into trading, the more it was just a sea of "Use these charts", "don't really trust these charts", and "once you lose a ton of money, you'll get a feel for things"
by pavel_lishin | 2025-05-27 17:25:30+00:00
> You don’t really have to do any digging—most of the stuff I’ve gotten was sitting on top of discarded furniture. But you do have to rush. After I took the Lululemon haul upstairs, I returned to find city waste workers loading things into a garbage truck, off to a landfill.

I'm frankly surprised other dumpster divers didn't get there first. I used to cruise around my university's campus during the end of the semester - got quite a bit of stuff that way, although none of it quite as pricey as what Duke students are apparently tossing.
by tommihip | 2025-05-27 17:26:37+00:00
Thanks, really appreciate the feedback - we’re tackling this by moving away from the old “pipeline for every problem” mindset. Instead, we built a mesh that connects all your databases—regardless of tech or location. With our unified model, we can handle access, migrations, replications, and even obfuscation. The model understands schemas across different types of databases and provides version control.

On the sustainability side, we don’t sell services or ETL tooling. We license the core platform like you would a service mesh or gateway. It’s meant to be scalable, reduce tool sprawl, and cut down on all the manual stuff teams usually have to maintain. The goal is to make modern data infrastructure both smarter and lighter to run.
by Someone | 2025-05-27 17:27:09+00:00
Reminds me of https://en.wikipedia.org/wiki/Rhyming_slang: “The construction of rhyming slang involves replacing a common word with a phrase of two or more words, the last of which rhymes with the original word; then, in almost all cases, omitting, from the end of the phrase, the secondary rhyming word (which is thereafter implied)”
by kuratkull | 2025-05-27 17:27:11+00:00
Because to me this seems like a fantastic example of a highly possible mistake that a typechecker _should_ catch. Without defined types in this situation a couple of things could happen: 1) it gets printed or passed to some other Any method and the typechecker never yells at you and it crashes in production 2) the typechecker catches the error somewhere long down the line and you have to backtrack to find where you might be appending a str to a list[int].

Instead it could mark it as an error (as all the other checkers do), and if that's what the user really intended they can declare the type as list[str | int] and everything down the line is checked correctly.

So in short, this seems like a great place to start pushing the user towards actually (gradually) typing their code, not just pushing likely bugs under the rug.
by gchallen | 2025-05-27 17:27:37+00:00
We've been doing this at Illinois for 10 years now. Here's the website with a description of the facility: https://cbtf.illinois.edu/. My colleagues have also published multiple papers on the testing center—operations, policies, results, and so on.

It's a complete game changer for assessment—anything, really, but basic programming skills in particular. At this point I wouldn't teach without it.
by throwanem | 2025-05-27 17:27:54+00:00
The link I posted leads to a paper you cited. You've attributed a causal claim to the paper which it not only does not make, but even in its abstract very carefully avoids. If that isn't intentional falsity, then it is certainly a remarkable demonstration of intellectual negligence. In any case "desperate" is not how I would describe the simple fact that I did a better job checking your sources than you have, which by the look of the thing is to say that of the two of us I'm the only one who bothered actually investigating your argument at all.

You could not by now have done more to prove my point that you aren't bothering to actually know anything about what you present yourself able knowledgeably to discuss. Thanks for that. Feel free to embarrass yourself with further flagrant scientism if you like. Enjoy your day.
by ETH_start | 2025-05-27 17:28:02+00:00
This is nothing more than a trope to absolve government of its responsibility for worsening people's lives. The government imposes massive fines on huge banks, when they are found to have inadvertently banked criminals. And by criminals I don't mean people who were convicted of a crime. Just people a regulatory agency alleged — at some point in time after the bank serviced them — are criminals.

So what banks do is become increasingly paranoid about who they will they bank, meaning anyone who falls into the "high risk" category, is extrajudicially punished through systemically high risks of being debanked. In this way, the state can punish certain groups without due process.
by dcreager | 2025-05-27 17:28:13+00:00
[ty developer here]

We are happy with the attention that ty is starting to receive, but it's important to call out that both ty and pyrefly are still incomplete! (OP mentions this, but it's worth emphasizing again here.)

There are definitely examples cropping up that hit features that are not yet implemented. So when you encounter something where you think what we're doing is daft, please recognize that we might have just not gotten around to that yet. Python is a big language!
by muglug | 2025-05-27 17:28:40+00:00
I’ve built a few typecheckers (in different languages) that hew closer to Pyrefly’s behaviour than Ty’s behaviour.

If you have a large codebase that you want to be typesafe, Pyrefly’s approach means writing far fewer type annotations overall, even if the initial lift is much steeper.

Ty effectively has noImplicitAny set to false.
by silveira | 2025-05-27 17:28:48+00:00
The shapes are called tetrominos, four squares connected orthogonally. Tetris the game, was created with pentominoes (5 squares) and scaled down to tetrominos. The game was inspired by physical puzzles where fill a rectangle with the shapes. So these shapes existed before Tetris.

Would they go after the shapes? Possibly. See "Tetris Holding, LLC v. Xio Interactive, Inc.".
by ta1243 | 2025-05-27 17:29:42+00:00
Many years ago UK ISPs particpated in an MBone with BBC, ITV etc providing live broadcast

https://www.bbc.co.uk/multicast/tv/channels.shtml

Brandon Butterworths note about "why"

https://support.bbc.co.uk/multicast/why.html

Shows the growth of the backbone and CDNs:

> The Olympic audience is expected to be around 50K streams, delivering 10Gbit+ is on the limit of sensible unicast delivery.

In 2020 the BBC's internal CDN was delivering 100 times that [0] for 250k users, and 5 years later I suspect it's another order of magnitude given that iplayer does 5 million concurrent live views quite frequently [1,2]

[0] https://medium.com/bbc-product-technology/bbc-online-2020-in...

[1] https://www.bbc.co.uk/mediacentre/2024/audiences-flock-to-bb...

[2] https://www.bbc.co.uk/mediacentre/2022/england-v-iran-bbc-li...

By 2035 and TV turnoff there's no reason to believe that the infrastructure won't have been able to scale another 100 fold and handle 500 million concurrent live streams. Makes no sense to multicast out 30 different formats, to people on phones and tablets and TVs hanging off wifi. It's a very different consumer experience than a PC wired into an ISP like it was in 2007.
by nancyminusone | 2025-05-27 17:29:54+00:00
>it also feels mildly wrong to take it

I never feel any guilt about taking things from the trash. About trespassing to somewhere you're not supposed to be? Sure. But if it's something that's 100% clear is in the trash and will be going to the landfill and not going to some charity reuse place or something, and I want it, I'm taking it.
by AStonesThrow | 2025-05-27 17:30:10+00:00
I think it's also got some connotations of "Build a Better Mousetrap"

https://en.wikipedia.org/wiki/Build_a_better_mousetrap,_and_...

And boomers know that the grand-master of designing better mousetraps was Rube Goldberg:

https://en.wikipedia.org/wiki/Rube_Goldberg_machine
by kikimora | 2025-05-27 17:30:18+00:00
>On a serious note, the systems being built now will eventually be "correct enough most of the time"

I don’t believe this would work. File a “good enough” tax return one year and enjoy hefty fine 5 years later. Or constantly deal with customers not understanding why one amount is in the dashboard and another is in their warehouse.

Probability of error increase rapidly when you start layer one probabilistic component onto another. Four 99% reliable components sequenced one after another have error rate of 4%.
Show HN: Idiot-proof restaurant deals on demand
by Superjock | 2025-05-27 17:30:34+00:00
The patented business model of this app is looking for or a small team (tech + business) to re-fresh existing MVP. MUST have fundraising capability (angel/pre-seed level). The easiest way to create a restaurant deal, is let the guest say 'how much' they'll spend and 'when' (today). That's actually an RFB, isn't it? This IP moves electrons, not soggy fries. I'm the 71-year old inventor and patent holder. I'm open to JV, acquisition, or exclusive licensing. Jeff@LessEat.com
by ecb_penguin | 2025-05-27 17:31:24+00:00
> what do you have in mind that has a structure similar to an exam?

All of life! An exam is a time boxed challenge. Sometimes it's open notes, sometimes it's not. I've had exams where I have to write an essay, and I've had exams where I've had to solve math problems. All things I've had to do in high pressure situations in my job.

Solving problems with no help and a clock ticking happens a million times per day.

We even assign grades in life, like "meets expectations" and "does not meet expectations".

Even still, you missed the point of my comment. You keep focusing on how interviews should be done, not how they're conducted in reality.
by MeetingsBrowser | 2025-05-27 17:31:31+00:00
> teams can always add runtime checks like pydantic to ensure your types match reality.

That's the problem with bugs though, there's always something that could have been done to avoid it =)

Pydantic works great in specific places, like validating user supplied data, but runtime checks as a replacement for static type checkers are not really feasible.

Every caller would need to check that the function is being called correctly (number and position of args, kwarg names, etc) and every callee would need to manually validate that each arg passed matches some expected type.
by zamal | 2025-05-27 17:33:01+00:00
DeepGit is a project I created to solve a frustration I've always had: finding truly relevant, high-quality GitHub repositories beyond just keyword searches. Most existing tools rely on simple keyword matching, which often misses out on context and deeper insights.

DeepGit tackles this by combining advanced techniques:

Dense Retrieval (Semantic Search): It understands your queries contextually, not just literally, to find repositories that genuinely match your intent.

Cross-Encoder Re-ranking: Results aren't just relevant—they're also ranked using neural models that deeply understand the content and relevance.

Detailed Analysis: DeepGit reads README files, entire documentation, explores repository structure, checks hardware compatibility, and even evaluates repository engagement and activity.

It's designed to surface hidden gems, useful projects you might not discover otherwise, and it provides meaningful justifications for why each repository fits your search.

DeepGit is fully open-source, built with LangChain and ColBERT embeddings, making it efficient and powerful. It's gotten great early traction and community feedback, reaching over 500 stars within days of making open-source. You can check out the code, use it yourself, or contribute here: https://github.com/zamalali/DeepGit

There's also a live lite demo on Hugging Face to see DeepGit in action immediately:I ran out of the GPU limit so you can duplicate the space and run it yourself: huggingface.co/spaces/zamal/DeepGit-lite

I'm always looking to make it better and go to place for all the devs to find the best repos. Feedback and contributions are very welcome.
by kuratkull | 2025-05-27 17:33:01+00:00
I hope I didn't come off as angry or anything, I was just very surprised by the behaviour :)

I am talking from some experience as I had to convert circa 40k lines of untyped code (dicts passed around etc) to fully typed. IIRC this behaviour would have masked a lot of bugs in my situation. (I relied on mypy at first, but migrated to pyright about 1/4 in).

But otherwise it's good to hear that this is still in progress and I wish the project the best of luck.
by ecb_penguin | 2025-05-27 17:33:04+00:00
> a schema update is accompanied by application updates

1. DB -> App = Two updates, the schema and the app

2. DB -> Service -> App = Three updates, the schema, the service, and the app

In both cases, obviously the DB changes.

In both cases, obviously the App changes.

In the second case, you also get to update your services.
by duxup | 2025-05-27 17:33:38+00:00
>But the illegal wiretap claim and Caldwell’s denials fueled a breakdown in trust between the Pentagon and the White House, where the Trump advisers tracking the investigation have privately suggested they no longer have any idea about who or what to believe.

If your leader is happy to lie about just about anything as long as it benefits them, and you bring like minded people onboard, isn't this just a natural consequence?
by bradly | 2025-05-27 17:33:58+00:00
Great job. I think this is a great area to focus on.

I am a solo developer who after trying to run local llm models for code and not being satisfied with the results is back to copy/pasting from browser tabs. I use vim so getting llm/lsp integration working reliably has felt questionable and not something I enjoying tinkering with. I tried aider with Google's Geminis models, but I never got the IAM accounts, billing quotas, and acls properly configured to get things to just work. I thought it would be fairly straight forward to build a local model based on my Gemfile, codebase, whatever else and have a local llm be both a better and cheaper experience than claude code which I blew threw $5 results that weren't usable or didn't save time after.

The sign up experience was really smooth. Like anything it else, is so easy to over complicate or be too clever, so I commend you for having the discipline to get it straight forward and to the point.

After account verification I didn't feel I understood what to do when landing on the Add Code Playground experience. It took me a while to grok what the three editors were doing and why there was JavaScript on the left and python on the right, but with an option for JavaScript. I found https://docs.relace.ai/docs/instant-apply/quickstart in the docs and at myself would be a better place to land after signup. I'd even recommend having the tabs on those snippets to be able to just grab a curl command and tip my toe in.

I think my biggest miss was my own assumption that a custom model was going to be a local model. Not that it was represented that way, but my brain was lumping those things together prematurely.
by Night_Thastus | 2025-05-27 17:34:22+00:00
Infrastructure can't be scaled down easily once it is built. You can't just shrink roads, water lines, electricity lines and otherwise once a population has shrunk. Nor can buildings like hospitals and fire stations. So no, expenses are not proportional.

Those costs continue, and now you have fewer young people to pay for it - and more elderly that will need assistance. It is vicious.

Some countries like the US have skirted past this problem due to immigration. Others like Japan have not have much immigration and are beginning to suffer the effects.

But every country will face it in the long run.

The cold hard fact is that when people are educated and have access to healthcare, they simply stop having enough kids to meet replacement rate - even when wealthy and when having great support systems.

Humans do not act like most species. We do not form neat equilibrium with our environment based on available resources. That kind of logic didn't apply the moment agriculture was developed.
by oldpersonintx2 | 2025-05-27 17:34:45+00:00
what does "resistance" look like?

RTO mandates have been just that, mandates

RTO or you will be terminated with cause

pretty simple really, not sure what there is to "resist"

most CEOs talking through this have said quite clearly that they don't care about case studies (Jaime Dimon was quoted as saying "I don't give a fuck what your studies say"), evidence-based arguments etc...so not sure why people are studying WFH vs RTO

you have to decide if you hate RTO enough to try and find a new job
by dcreager | 2025-05-27 17:35:06+00:00
> Gradual typing means that an implicit "any" (unknown type) anywhere in your code base is not an error or even a warning. Even in critical code you thought was fully typed. Where you mistakenly introduce a type bug and due to some syntax or inference limits the type checker unexpectedly loses the plot and tells you confidently "no problems in this file!"

This is a good point, and one that we are taking into account when developing ty.

The benefit of the gradual guarantee is that it makes the onboarding process less fraught when you want to start (gradually) adding types to an untyped codebase. No one wants a wall of false positive errors when you first start invoking your type checker.

The downside is exactly what you point out. For this, we want to leverage that ty is part of a suite of tools that we're developing. One goal in developing ty is to create the infrastructure that would let ruff support multi-file and type-aware linter rules. That's a bit hand-wavy atm, since we're still working out the details of how the two tools would work together.

So we do want to provide more opinionated feedback about your code — for instance, highlighting when implicit `Any`s show up in an otherwise fully type-annotated function. But we view that as being a linter rule, which will likely be handled by ruff.
by gchallen | 2025-05-27 17:35:14+00:00
Immediate feedback from a good autograder provides a much more interactive learning experience for students. They are able to face and correct their mistakes in real time until they arrive at a correct solution. That's a real learning opportunity.

The value of educational feedback drops rapidly as time passes. If a student receives immediate feedback and the opportunity to try again, they are much more likely to continue attempting to solve the problem. Autograders can support both; humans, neither. It typically takes hours or days to manually grade code just once. By that point students are unlikely to pay much attention to the feedback, and the considerable expense of human grading makes it unlikely that they are able to try again. That's just evaluation.

And the idea that instructors of computer science courses are in a position to provide "expert feedback" is very questionable. Most CS faculty don't create or maintain software. Grading is usually done by either research-focused Ph.D. students or undergraduates with barely more experience than the students they are evaluating.
by mulmen | 2025-05-27 17:35:36+00:00
Honesty doesn’t mean much when leadership doesn’t understand or respect the business.

I can say “I don’t know how to do brain surgery and furthermore I have no interest to learn.” You may correctly determine I am being honest but this in no way qualifies me to do brain surgery.

Knowledge and experience matters. Welchian leadership aggressively and deliberately dismisses all knowledge as a cost to be minimized. The idea is you can make up for any organizational shortcoming if you just abuse people enough. It’s a very leaded pipe era way of thinking.

The disastrous results of this philosophy speak for themselves.
by fernly | 2025-05-27 17:35:48+00:00
The address given for the Byte Shop, "1063 El Camino Real in Mountain View", is ambiguous. It needs to specify either 1063 EAST El Camino Real or 1063 WEST El Camino Real, two quite different locations.

Neither of those matches the store that I remember patronizing circa 1978 or so, to buy a California Computer Systems S-100 box. That would have been on El Camino just north of Grant Road, circa 80 W El Camino.
by icehawk | 2025-05-27 17:36:02+00:00
As far as I'm aware "a session is configured as IBGP on one end and EBGP on the other end" isn't possible.

You can't configure it like that, most of the BGP implementations I'm familiar with automatically treat the a same-AS neighbor as iBGP and a different-AS neighbor as eBGP.

Juniper explicitly has 'internal' and 'external' neighbors, but you can't configure a different peer AS than your own on an internal neighbor or the same peer AS on an external neighbor.

BGP sessions also have the AS of the neighbor specified in the local config, and will not bring up the session if it's not what's configured.
by cshimmin | 2025-05-27 17:36:52+00:00
There's a board-game version of Tetris that I've been playing with my toddler, and is pretty much the same low-stress approach. It's also multiplayer (the next piece everyone must use is decided by drawing cards from a deck). The only shortcoming is that you can't do the maneuver where you slide a piece sideways just as it hits the bottom to slot it into place under another piece. Highly recommend!
by bee_rider | 2025-05-27 17:36:59+00:00
Something I didn’t love about mandatory homework was that it provided an implied “you are done” point, when really it is the bare minimum (or maybe less than the bare minimum—there’s a pretty strong downward pressure if the instructor actually wants to provide thoughtful feedback).

Before college, when I was a kid, I just had the textbooks, so I read the chapters and did the assignments… it was much better than sitting and listening in lectures, then doing some small assigned subset of the problems…
by xyzzy_plugh | 2025-05-27 17:37:13+00:00
> That's what AGPL says.

I mean no disrespect, but it doesn't say that. This is your interpretation. There is insufficient legal precedent to validate this interpretation, and thus there is inherent risk.

The risk specifically is that if I use PgDog as a component of a larger application (e.g. a SaaS app) then the entire application may be in scope.

You can fill this webpage with text stating otherwise but it doesn't matter in the slightest. If your intent is that only users connecting directly to PgDog are entitled to receive the source, then I encourage you to develop a license saying as such. But that's certainly not the AGPL, unfortunately.

(Caveats being I'm not a lawyer, and this is not legal advice. I certainly don't enjoy imagining how one would even define direct connections in a court of law.)

FWIW your heart is in the right place. I wish the world worked the way you describe.