skysight-inc/HackerNews huggingface dataset:
Show HN: Build a log explorer in minutes and scale it to billions (for real)
Built a log explorer because it's a common, hard and painful problem for devs, but wanted to showcase the process and some tips for scaling that can be applied to similar use cases.

Hope you can learn something new

· How it started => https://www.tinybird.co/blog-posts/build-a-datadog-alternati...
· How I scaled it => https://www.tinybird.co/blog-posts/build-a-datadog-alternati...
· My Cursor + sonnet process => https://x.com/alrocar/status/1892653985330258283
· GitHub repo => https://github.com/tinybirdco/logs-explorer-template

For the demo app I loaded 1.5b rows, there's a mock data generator in the repo.

[we open sourced it and are featuring it in product hunt today in case you want to take a look]
Nice, I like the perk system. Played a few rounds. Quick thoughts:

At first I didn't know that the coins were coins, it just looked like brick particles for visual effect. Maybe having a current coin counter or visual difference would've helped.

Speaking of coins, there's a couple times when the ball and coins were too similar in color, making it hard to find the ball amid a mass of falling coins. Maybe this is by design, but some contrast between coin color and the ball would help.
The Hero Trap
Notable approaches to compatibility with C might be: 1) LLVM, like Rust and Zig did (Zig stopped using it in 2023), since LLVM IR is good for being compatible and optimizing. 2) Other backends like libgccjit, I mentioned this because rustc has a libgccjit backend and its author who is a libgccjit maintainer loves the simplicity of it, one could also think of it as a programmable GCC. 3) Code generation directly to C, that's what Koka does.

So I was talking about the direct C codegen approach, but there's still much of some mess like one needs to choose a C standard and knows how to verify the generated code.
Gross, unhelpful comment, and essentially just whataboutism. Plenty of American cities are also unsafe and OP didn't say otherwise.

That isn't 'white panic' lol. Are you perhaps insecure about your own race?
Or do we only think wolf puppies are cute because we co-evolved with dogs?
> WASM was birthed out of trying to make JavaScript even faster.

WASM was birthed as a Flash replacement, but ubiquitous to programming languages. It, according to its maintainers and originators, has nothing to do with JavaScript.
And collect a full salary? That sounds rather like admitting there's an SWE bubble, and I can't imagine indicates good things for the future job market...
> The number one reason we see customers opt to not use cellular when
> another connectivity option is on the table is due to concerns around
> cost and reliability.

For "reliability" read "availability". Cellular networks are profitable because carriers serve customers that are easy to serve and leave the rest alone. Vehicle 2 Vehicle communications mediated by cellular is a non-starter because people drive a lot in places where cell phones don't work. Wireless internet has been a false hope that's spoiled realistic plans for universal service because it's only affordable in markets that are already served. Large-scale government and corporate IoT programs find that 20% or more of stations are in places without service, etc.
>Is it not equally plausible that the data included social media content or other user input that interchanged trump and racism?

That's equally bad and paints Apple's AI development efforts poorly if they let sloppy training data like that get into production. Garbage in garbage out. They should have done better QA, they're only the richest company in the world by market cap, they can certainly find some change between the couch cushions for that.

>When did it become hip to cheer for people being fired?

When they inject their own biases I never asked for into products I spend good money for. I don't want my phone made in China by factories with suicide nets by a company embroiled in scandals on patent trolling, tax evasion and monopolistic practices[1], lecturing me on ethics, virtues and politics. This is who you choose to be your beacon of morality?

I hope firings continue until the message sinks in that the product is there to serve the customer regardless of their political orientation, and not the other way around. I won't put up with entification and receiving lecturing on my own dime.

[1] https://en.wikipedia.org/wiki/Criticism_of_Apple_Inc.
They're making a rational judgement based on factual information about current events.

The authoritarian coup currently happening in the US is real. Do not let your conditioning cause you to view it a conspiracy theory.
TypeScript types can run DOOM [video]
"quite a while ago - 2020"

I think in the context of this article, 2020 is very recent. But otherwise, yes I agree.

Incidentally, my own site started out in 1999 as a personal site, became a poster for my writing, then back to a personal site and is now a poster for my tutoring. All the old content is still there, just with a different emphasis https://richardtaylor.co.uk/
So redoing the calculations, a dozen eggs were a quarter day of laborer wages, and more than a full day of wages for a maidservant or swineherd. Still considerably more expensive than the price of eggs today.
My last job was SAFe. When I started I was given Staff level title and had dreams of maybe moving into lead or management at some level. Once I saw the process I became completely unmotivated to go in that direction.

For them I understood some of the motivation. Hardware & equipment manufacturer, which involves scheduling complicated industrial processes for months/years out. So you need some semi-coherent vision of where things will be, so having a multiquarter waterfall-esque plan was going to be needed.

Not that any of that actually seemed to work.
What's the finite group in this case?
Microsoft Windows NT OS/2 Design Workbook
Using Generative AI to Build Generative AI – O'Reilly
Yeah sounds like bartering exists only in the world of thought experiments. Gift economies were the norm
The whole proposal can be distilled to Connections => Mind. We've been struggling with the causal effect, even the definition, of what the `=>` means in humans. Seems a long shot that the study of, and breathy coverage, of plant signals will illuminate much. Maybe it will, but I'm not holding my breath.
Awesome stuff! Just published something similar today

Just curious, what is the most challenging thing in your opinion when building such log viewer?
From my experience (having hit the Deep Research quota), I wouldn’t use it to build data tables like the article did, but for qualitative or text-based research, it’s incredibly useful. Useful enough that I’d justify multiple accounts just to increase quota. People keep saying hallucinations but in the reports I have built I have not noticed it to be a problem, again I am not doing quantitative analysis with it.
> In other words, work in the domain that is natural to your data.

Why would multiplication be more "natural" to a domain than convolution, as opposed to just simpler to calculate?
It was Apache2, see this (preserved) fork.

https://github.com/Dramaga11/vsc-material-theme/blob/main/LI...
Such is not life friend. Society is whatever we want to be. The rules are made up and we, the people, can decide what society should value.

It doesn't have to be this way, it can change. Maybe not in our lifetimes but we plant the seeds so future generations can live better lives.
Cell-Phone Technology on the Moon Gets a Test Run
Thanks!

If anybody has any suggestion in preparing for a PhD in a major university, I'd appreciate it: I expect these four years to be fun but also very challenging.
I was expecting a link to the PDF...
Am I correct in reading that the RAG pipeline runs in realtime in response to a user query?

If so, then I would suggest that you run it ahead of time and generate possible questions from the LLM based on the context of the current semantically split chunk.

That way you only need to compare the embeddings at query time and it will already be pre-sorted and ranked.

The trick, of course, is chunking it correctly and generating the right questions. But in both cases I would look to the LLM to do that.

Happy to recommend some tips on semantically splitting documents using the LLM with really low token usage if you're interested.
Didn’t SQLite developer(s) famously receive a flood of phone calls because McAfee antivirus used it in a way that was visible (and “suspicious”) to its users?

One does not simply “not accept bug reports”.

https://github.com/sqlite/sqlite/blob/e8346d0a889c89ec8a78e6...
This is so completely reasonable, and yet so unexpected that the "please leave now" actually reads like a joke punchline. In the last few years ads have become remarkably dystopian and unapologetic about it.. I mean it was always "No one will love you if you don't buy our thing!" but they didn't quite come out and say it before. The "less likes, more love" campaign comes to mind as ironically appropriate.

I'm not in a position where I deal with actual human salespeople that often, but we're all products of the culture we're embedded in. So I assume that's gotten more masks-off crazy and feral too.. just very openly trying to find your anxieties and squeeze.
The word used in the OP was “like” not “love”
I found the artifact itself interesting. I believe these are the contents:

https://computernewb.com/~lily/files/Documents/NTDesignWorkb...
I think the root issue is that the word “love” in English has far too many meanings.

When you “love” a company, it’s not the same as “loving” your wife.

And “loving” your wife is different, still, than “loving” your brother or best friend.
Injections don't always need ''. The statements

1=1

and

1=0

if injected into a query will give different answers if SQLI exists.

There are MANY other tricks that don't involve ''.

Besides, consider the number of valid queries done by the application that involve '*'. You are not going to turn that off.
Bezos says WP opinion page will focus on 'personal liberties' and 'free markets'
That's not the difference. That's aligned with my point.

If you don't have an iPhone or Meta account or NFL pass (or ...) then you can complain without being a hypocrite. That's great.

Or you can say those things aren't as significant to you right now, so they don't compare. In which case, you'll surely realize that others may come to similar conclusions about Tesla. Say, valuing the lives of third world workers more than the political state of the US
This isn’t fascism. It’s the President using his bully pulpit. Fascism would be Apple suffering extrajudicial consequences.
You do like money. Try not having any.

https://upgrader.gapminder.org/q/32/
Ercot is on a hot streak during a cold winter
Scanning a QR code is far faster than typing a URL, and you need some sort of a computer to access the URL anyway, so providing a human-readable URL doesn't achieve anything.
no was just sharing what i found online. do not use it, too expensive.
Given the fact that every human being has his personal point of view of the word, life and everything, and it's fine, imho, the problems arise when one has 'too much' power (money? influence? visibility? say it.), in a complex word this lead to -big- problems without proper balances.
Or in others words, when ideas -too big- try to fir in minds -too small-, and success and whealt has nothing to do with true real intelligence.
Or in others (prosaic, i know, sorry) words 'A Good Man Always Has to Know His Limitations(H.Challagan)'
That is true, but you could always move to another region and there would still be some level of fish & game.

a) the ability to have grain stores, etc thru the winter; and b) the long food cycle of plant in spring, harvest in fall; really put everything on steroids in terms of what "territory" was important (the grain stores, the planted land) and what territory was less important

You can see this in the development of forts, castles, etc later on where the "keep" was the most protected thing, and held the grain store for an entire village
anybody knows how to remove that pop up?
> we should assume it will hurt us and purposefully retard the advancement of technology?

This is the position of Geoffrey Hinton, and other AI doomers. Their POV does not appear to being taken seriously right now, at least we're not seeing much evidence of international collaboration on AI safety protocols.
> Money is selfish.

I used to believe so too but after reading this I'm no longer so sure: https://meltingasphalt.com/wealth-the-toxic-byproduct/
Yeah, Fanni is still used in Hungary.
how to remove that pop up which keeps coming?
Even if someone does like money, the line of argument Ruben examines is trying to convince you that the ONLY thing you should like is money. One can 'like' money, and still recognize that other things are more important than money.

I like cake, but I don't want limitless cake to the exclusion of every other concern.

.

You know who else liked money? King Midas.
mRNA Vaccine Shows Sustained Immune Response in Small Pancreatic Cancer Trial
Is "fanny" an insult at all?
The daemon wasn't truly evil. Being in the bath house that the witch was running to extract money from the spirits is what brought out that side.

The aunt from Grave of the Fireflies who convinces the kid to sell his parents' stuff for food money and then complains he did nothing to deserve the food she made was a piece of work though.
With Youtube at least, you can buy Youtube premium, so you don't have to sit through Youtube ads without needing an ad blocker (though you'll still have to sit through any ads the Youtuber directly adds into the video itself).

Disclosure: I work at Google.
Taiwan is not some small random island in the Pacific. It is a country of 23 million people, one of the best-functioning democracies in the world, and the only Mandarin-speaking democracy. It would be a tragic loss for the world if it were swallowed by the PRC.

Is it important enough to start WWIII. That’s another question.
> Productivity increases -> wages go up -> "30% of salary on rent" goes up -> rent goes up -> cost of everything goes up

Yes wages might go up (someone who can use Sharepoint[0] is probably more efficient than someone who can use a physical filing system) but you need far fewer people to do it. So costs can still go down.

> We could arrest pretty much all cost of living growth if all renters (both businesses and individuals) just decided not to continuously agree to pay higher prices for the same land/apartments/buildings.

I agree that most of cost of living is housing-related, but rent price increases are a function of population increasing faster than housing. That's the thing to address.

[0] or something good
How do the underpants fit into this?
Consider this your reminder to swing by the home improvement store and buy that backup generator you've been thinking about getting for a while.

Gonna be a lot of discarded food in Chile over the next few days...
Literally, the second comment in:

> Welp, the kernel is going to shit over an untested meme language.
The litmus test for this is whether they are willing to fund sweatshop-automator-pro-max.
I fundamentally disagree with making money from ads. I have no problem giving money to people who make things.
Money is more often than not an impediment to doing or achieving things (especially these days when the ultra rich are trying to hoover it all up themselves to enable power).

Imagine if money did not exist, and things were decided on actual merit? "Poor" people with good ideas might - dare I say it - do things that improved the world for everyone! ;)

There'd be many ways of achieving this, so I'll not bait people with C or S words.
Org mode is great for plain text info storage. Just difficult to use if you havent joined the church of emacs
I don’t understand this. Last time I bought insurance it was easy, just go online, pick one of the options and fill out a form. What does this have to do with “bowing” to anyone? It’s no different than many other transactions.

Personally, I like that I have options about what insurance to buy.
before copilot the first item in their release notes was always accessibility, which I though was a very nice touch. Now Copilot took the prime spot
What I find remarkable when I go to NYC for work is that I see many black people in the subway or at street level but when I go past the guard at the desk and go up the elevator black people become scarce.

When I go to NYC I find numerous opportunities for networking that I read about online or that I find out about from my friends and co-workers. For instance an organization that accredits accountants puts on great free conferences, one of which brought together established finance organizations such as the DTCC with blockchain startups.

If I lived in the city I know I could go to free or inexpensive events, chat people up, and find opportunities. Somehow that doesn't work for black people or I'd see more of them working in finance and other industries concentrated there.
[dupe] https://news.ycombinator.com/item?id=43151580
https://archive.is/lzXRz
Overall, i like this guys papers but they strike me as someone who is very smart but hasnt looked through the literature carefully. Many of the techniques he is proposing were already done about 5-6 years ago. However, I imagine that because the field is flooded with new humans, they are not aware of this research or think it will lead to a fruitful end (which many other researchers have already thought of this and it didn't lead anywhere hence why it was abandoned). Overall, it seems we are starting to recycle ideas because there isnt enough lit review and or mentoring from senior deep learning / ML folks who can quickly look at a paper and tell the author where the work has been already investigated.
I have received class action settlement payments from Verizon, Apple, and others for things I hardly noticed at the time. So maybe your idea of what precedent considers “damages” here is incomplete.
Show HN: Emdash – Slack/Zoom alternative for distributed team collaboration
Hi HN, I’m Phil, one of the co-founders building emdash. Previously, I was an early engineer at Facebook and led Customer Products at Square.

We’ve focused on making chat and video work together so distributed teams can stay aligned without drowning in information. You can try it here: https://emdash.io.

It frustrated us how easily important conversations would happen and then disappear. Slack never quite matched how we worked. Channels were too coarse which led to noisy notifications and broken search. Zoom meetings weren’t much better–unless someone took perfect notes (which rarely happened), video calls became black holes of lost knowledge.

We spent too much time trying to find the information we needed to do our jobs.

To address this, we’re testing a few concepts and would appreciate your feedback on the value of the following:

(1) Automatically record, summarize, and transcribe your team’s video chats. We store meeting content directly inside discussions to facilitate search and discovery.

(2) Make it easy to manage & organize conversations of varying scope. A chat between team members can be forked into a dedicated Discussion with its own audience permissions and subscription. Individual messages or entire Discussions can be moved after the fact. Conversations can evolve unpredictably, so having the right tools to keep them organized post-hoc was important to us.

(3) Improve search with AI and hierarchical information retrieval. We use LLMs to uncover insights, summarize content, and connect the dots across related discussions, meetings, and documents. You can ask questions like “What are the team’s priorities this week?” or “What did we decide to do with feature X?” and get back a generative response AND deep links into the original chats and meetings.

Try it out: https://emdash.io and tell us what you think!
Depending where you look your numbers, Chile's crime and homicide rate is very similar to the United States.
>I hate that I'm forced to devote so much of my life and mind to it.

Your life would be better if you devoted so much time to food, clothing, and shelter? That's what money is. If you dislike money, a mere change in attitude such that you believe you're pursuing the essentials should suffice.

>I hate that its used as a weapon by those

This seems doubtful. Those in warzones have my sympathy, but I think that it's pretty likely that you live in a place where it's not used as a weapon at all. Your family remains unstarved, no one's burning your crops or confiscating famine relief off the UN trucks.

>But I yearn for a better way.

You live in an era of material abundance beyond the wildest imaginations of your ancestors. Even poor people are fat. Clothing is so cheap we throw it away if it gets a cheeseburger stain... your grandma didn't slave away for years to gift you that roughspun, sackcloth shirt. Etc etc.
Attitudes toward AI: Through the librarian's lens
Big picture, you're absolutely right and I do what I can to plant those seeds. However, while that could help future generations and so is worth doing, it doesn't actually help me in the here and now. There's nothing I can do that will positively change things on this count within my lifetime.
this is not about who they vote for, it's the system that is neoliberal in that allows and incentivizes only maximum profit and puts very little barriers
As an example, if you're trying to "smooth out" some signal, what you're really trying to do is remove the high frequency components. So using a Fourier transform to convert it to the frequency domain lets you directly work with the frequency data, rather than indirectly in the time/space/whatever domain. The fact that the operation is simpler in the frequency domain is a good hint that you've picked the "natural" space in which to look at the problem. Of course, there's no formal definition of "naturalness," and at the end of the day, you get the same result either way.
MySQL hasn’t had nearly as many innovations. That said, for a long time, it was often faster for simpler queries, and especially range queries, IFF your schema was designed to exploit the fact that it’s a clustering index, so rows are physically located next to the PK. If you’re using UUIDv4 PKs, you’re throwing away its entire advantage, and Postgres will almost certainly be faster.

One notable exception for MySQL is INSTANT DDL in 8.x, thanks to contributions from Tencent, which is an extremely nice QoL upgrade that Postgres doesn’t have. The other is logical replication, something Postgres does support now (and has for several years), but didn’t for a long time.
I always likened those rebuttals to the “it’s just how we’ve always done things” refrain, spouted by people who believe they have more to lose by changing the status quo than to gain, but refuse to investigate why and adapt accordingly if true.

Because you’re right, fandom toward products and enterprise is objectively bad. It’s taking the same hooks actively exploited by religion and cults, but now towards iPhones and B-Corps. They are being exploited, and choose not to recognize it because their identity is tied to their fandom.

Yeah, life is a lot harder being the wet blanket in the conference room, asking pointed and specific questions about why we do things this way, could we do them better, etc, but it’s also liberating to see just how much of the world is blind momentum built on fear of literally any other alternative.
Java runtime without miltithreading? No thanks.