skysight-inc/HackerNews huggingface dataset:
I think you’re on the right track that trees are good at feature engineering. But the key problem is that DL researchers are horrible at feature engineering, because they have never had to do it. These folks included.

The feature engineering they do here is absolutely horrible! They use a QuantileTransform and that’s it. They don’t even tune the critical hyper parameter of the number of quantiles. Do they always use the scikitlearn default of 1,000 quantiles? No wonder uninformative features are hurting- they are getting expanded into 1000 even more uninformative features! Also with a single quantile transform like that, the relative values of the quantiles are completely lost! If the values 86 and 87 fall into different bins, the model has literally no information that the two bins are similar to each other, or even that they come from the same raw input.

For a very large dataset a NN would learn its way around this kind of bone headed mistake. But for this size dataset, these researchers have absolutely crippled the nets with this thoughtless approach to feature engineering.

There is plenty more to criticize about their experiments, but it’s probably less important. E.g. Their HP ranges are too small to allow for the kind of nets that are known to work best in the modern era (after Double Descent theory has been worked out) - large heavily regularized nets. They don’t let the nets get very big and they don’t let the regularization get nearly big enough.

So. Bad comparison. But it’s also very true that XGB “just works” most of the time. NN’s are finicky and complicated and very few people really understand them well enough to apply them to novel situations. Those who do are working on fancy AI problems, not writing poor comparison papers like this one.
Fine, not the point, they did it - implies the invitees weren't trusted 'they won't/doesn't matter if they do pwn each other' customers.
The general workflow I'm finding useful is to generate a ton of simple logs, ingest all those logs into a SQLite database (that acts like a "data lake"), and then use the customized REPL to generate some charts (typically .png files), CSV, or ascii reports.

I could see using a REPL to generate a TUI, e.g., "run a query and generate your own `top`". This seems like it could be a lot easier than trying to generate a PDF, interactive Excel, or complex HTML report (which usually means a bunch of javaScript).

There's something about text interfaces that just fundamentally "flows" from most programming environments.
According to [0], it's Australian voice actor Lee Perry.

[0] https://en.wikipedia.org/wiki/Wear_Sunscreen#Baz_Luhrmann_ve...
Yeah the bloomberg terminal is still good value if you're going to be using that data, and a relatively good price. Good point about the exchanges
I don't know what's so demonstrably false about it; there is credible evidence (cited in another comment) suggesting mineral/electrolyte loss from drinking demineralized water and GGP admits to purposefully consuming nutrients from other sources to compensate. That only helps to prove my point?

If you have to compensate from other sources than I don't understand how something is necessarily harmless, without the qualifier that it should only be used in the context of a proper diet (which many/most don't have, particularly the types that might be susceptible to quack wellness claims)

Besides, one anecdote does not a disproven theory make
Yeah Teams is probably one of if not the worst pieces of software I've had to use in the last 5~ years.
I think this is not about Teams. Probably more about security critical backend code.
Are US threat assessments outpacing military realities?
Because they will use ML on the blockchain to prove single source origin of the cloud. And shareholders will eat it up.

Less facetiously, however, shoe culture is a thing with a huge resale market. Clothing and shoe piracy is a thing. I could see them using this as a pitch to overcome some of those issues to try and establish a clothing culture to rival shoe culture.
How to think like a computer – short videos for teaching kids (2016)
I think you can, it has a pretty powerful configuration editor. I wasn't specifically looking in TC-like config, because Far hotkeys are hard-wired in my brain and I felt that they are mostly the same in Marta.
If the boss is putting you in a form of solitary confinement like the Japanese bosses do, then yea that would suck, but I would be surprised that an American boss would do that without HR getting involved as singling out an employee for extra special rules without a pip is an easy court case
Ask HN: Who Has or Can make my dream iOS Task app?
All I want to do is:

1 - click the app
2 - dictate a task/note
3 - have it show up in a simple to do list.
*bonus points for integration with GCal Tasks and Trello!
True! I guess there’s only one way to find out.
Makes sense. We can also learn from this!
Issues:

Discovery - Other than spots where part of a lava tube's roof has already collapsed, how quick & cheap is it to discover and map the lava tubes under some non-trivial part of the lunar surface?

Scale - Let's say there's a lovely lava tube, X meters in diameter and Y meters below the surface, in a location where we'd like to build a moonbase. For a give size of moonbase, there will be fairly narrow ranges of X and Y values where the lava tube is actually suitable for housing the moonbase.

Cut & Fill Competition - Lava tubes aren't magical. If an area on the moon has deep, stable "soil" (vs. near-surface bedrock) containing ~few larger pieces of rock, then "dig a trench, install tubular habitat, fill trench back in" may give a far lower cost and/or higher predictability than "starting looking for the perfect lava tube...".

Stability - If you're not familiar with the things that often go wrong when humans start digging and building structures, Practical Engineering ( https://www.youtube.com/channel/UCMOqf8ab-42UUQIdVoKwjlQ ) is a pretty good intro. Note that we have a lot of experience with and data on how to do such things on the Earth. On the moon...not so much. Some very painful and expensive lessons seem inevitable.
Apologies, I didn't realize you'd left to work at Shopify.
You can shave some slacking off time by skipping lunch and dinner and instead chugging some Silicon Valley energy drink mix, marketed with a fancy name.

Wasting time with meals is for unsuccessful losers!
This article isn't about the app called Microsoft Teams. It's about a lowercase-T team of people.
I started with Gosmacs back in 1984 at CMU. Time does fly.
FWIW, about 12.8% says https://ourworldindata.org/energy/country/iceland#how-much-o... . (Wikipedia says 15% at https://en.wikipedia.org/wiki/Energy_in_Iceland but that uses a 10 year old source.)
I think part of the problem is that the upper bound on neural nets, as far as we can tell, might very well be general intelligence, and things like self-driving cars, and other nearly magical use-cases that seem within reach. Whereas tree based models, for a series of reasons, many related to scaling, don't offer that feeling of limitless potential.
I guess it's because it's a serendipitous discovery by roboticists rather than an experiment explicitly set up to discover "alternative physics," but wouldn't it be much more efficient to feed the NN actual physical measurements rather than a video if discovering physical models was the goal?

It's amazing that the AI found two variables that corresponded to actual physical phenomena just from video footage but it seems highly unlikely that it found an alternative description of physics given how far off the answer was. The NN could be modeling any kind of weird "fantasy" for how the visual signal it was being fed could be produced that might not even correspond to 3D space in a meaningful way.
So my version of that story is less analog and arguably less academically honest. I had a somewhat challenging mid-level math class in college where after each homework assignment and test the professor would give us a URL to a PDF of his scanned handwritten completed version of the work. The URL path was something like /math-321/2019/fall/test-1.pdf, and the professor diligently made sure that each file wasn't available until after each test was graded. Unfortunately, the professor was not sufficiently diligent to remove URLs using the exact same pattern for previous years and semesters of the same course. I discovered throughout the course that there had been some trivial changes to the assignments and tests (moving questions around, slightly changing constants, etc.) and only a few non-trivial changes.
Once I stayed in an Airbnb owned by Karl Friederich Gauss' distant relatives in Brazil.

It was a very cozy cabin in the mountains around Rio and I was celebrating a two-year anniversary with my girlfriend. There were a few books arranged in a short rack, mostly teen stuff, but one aged book stood out. It was an English version of Gauss' Theory of the Motion of the Heavenly Bodies, apparently borrowed from an university library in the 1970s but never returned. Inside, I found two documents from 1969, a voter registration and an exam card. They belonged to a woman with a Brazilian first name and Gauss' surname. Later, I had to transfer money to the Airbnb host, and she also had Gauss as a surname.

I was pretty thrilled with the whole thing. My girlfriend was more entertained by the cabin's cat.
Thats very baaaaaaaahd.
Best software engineer career paths
I enjoy buying used books partly for the random items and marks. It's surprising how many old books include scribbled notes or underlined/highlighted sections. Sometimes the notations make sense, but often times I have no idea why a certain part was highlighted and have fun trying to figure out the connection between various marked passages.

The most common item I've found is a receipt. Sometimes it will be for the book itself; often it will be totally unrelated, sometimes quite old.

There is something enjoyable about having these little connections to a distant stranger as part of an otherwise solitary activity.
Well, this is Google so they probably A/B tested a bunch of random things and this one resulted in 0.0002% better monetisation or whatever...

At this point, Google's problem is not "lazy workers", is lack of vision and direction.
Meh, my dad ran a small window cleaning business in a small midwestern state. He clearly saw a difference when his existence was known on the internet versus with paper fliers only. Advertising is letting people know you exist.
Oh man, you have got to visit Virginia City. Yes, it's really sitting on top of mines. And you can do some mine tours. It's a bit on the touristy side, but I have a picture of me and my family visiting there in the early 80's, and then another pic from almost the same spot (quite coincidentally!) just a couple of months ago, and not a lot has changed.

There are also excellent museums in Carson City. And, you can take the train from VC down to CC! Something I have yet to do, but am going to do soon.

If you need a local contact, hit me up. username at gmail.
I’m not sure where you got the idea that I harbor some absolutist opposition to regulation in any industry, or even in the social media industry for that matter. My comment listed specific examples of social media regulation that I would support.

My goal was simply to highlight the naïveté of thinking regulations alone will solve the problem, unless the USPS is going to send a weekly pill bottle of amphetamines to every citizen who is vulnerable to the dangers of social media addiction.
Summary: Five short fun videos that kids (and the grown-ups) might enjoy. Created by 'BBC Teach'.

- What is decomposition?

- Recognising patterns

- What is abstraction?

- What is an algorithm?

- Checking your algorithm
I would love to learn more from a neutral source.
One benefit is that you can operate “blindly”, without needing the hand-eye-coordination feedback loop. This tends to make frequent operations more automatic and decrease cognitive overhead.

Another benefit is that keyboard input is higher-bandwith than mouse operations. Imagine a mouse-only interface for everything you can do on the Linux command line, or all the operations you can do in Vim or Emacs (beyond actual text input).

It’s more powerful in the sense of “power user”.
If you peruse instagram accounts that post tutorials of how to do things (cooking, art, engineering, etc.) you will quickly realize that there are thousands of positive comments and likes on instructional videos that teach things completely wrong. Anyone attempting to follow the video would be baffled. This article definitely comes up with a good term for the phenomenon behind the success of those videos.
Yes thanks, I meant SFTP :).

But my point was mainly that in Dolphin I can open a terminal panel in the directory I'm in, and that this works even for mounted remote directories. Which is frankly quite awesome :).
Same! I hate the fact companies can’t understand they fill a specific niche.
Categorical variables in the datasets of large tech companies can take a lot of different values.
It saddens me too, but it still seems necessary. In my (quite long and varied) experience, most developers do not appreciate the value of regression tests. Therefore, reminders and positive reinforcement are still beneficial.
Don’t be too impressed. Every company is interested in “opening things up” when it’s to their competitive advantage.

You notice that “open” Android is increasingly dependent on closed source “Google Play Services” and Google uses the closed source parts to beat OEMs into submission?
Another one that is elementary and high school oriented, and CC licenced, is the Utah's Open Textbook Projects, https://www.uen.org/oer/.
Some book publishers also release new versions that change nothing other than make slight changes and reorderings of the exercises so that you can't easily use an old version for your homework assignments.
The kinds of trees that come out of these algorithms are so huge they really aren’t any more interpretable than a NN.
So now email is unsafe for MFA/password reset messages too? (actually I've long argued it's not really any safer than SMS, at least in countries where there are decent regulations around porting numbers).
Run FreeBSD 13.1 for ARM64 in QEMU on Apple Silicon Mac with HVF Acceleration
The following text seems to be mostly talking about things that have nothing to do with square pixels, it just sounds like that's what was how the dude programmed his generating algorithm. To the best of my reading, the goals he achieved with the patterns don't interact with the squareness.
If I got it right, if adjustments are needed, it is to keep being synchronized with the tilt of earth rotational axis... namely with the seasons.
Three built-in “multiple file management” systems in notepad++
one question is is possible in any kind of way that that passwd or any other binary that stays that you dont need has a security vulnarability that could if someone got into the container in one way or another(most likely your app) cause trouble on the host.

hint yes it is and that could be a problem a giuant one
Next up: mandatory updates.
As a mid-level engineer, lately I’ve come to think that improving my writing (or more broadly, communication) skills is the best way to grow my career.

All of those great ideas you have aren’t worth anything if you can’t communicate them. We try to make technical decisions as objectively as possible, but often some amount of persuasion is required to align everyone on your preferred outcome.
It opens a separate xterm window on debian. Does it have an integrated terminal on windows?
From day one Google’s business plan was ads - except for the brief time where they were selling “Google Search Appliances”.

Every company whose business model isn’t “I give them money and they give me stuff” is shady.
47? This is my year. And in a recession. Best time to start, right?
I found $20 on my
block this morning in New Orleans. It belonged to my neighbor across the street fixing his car. He offered me a beer at 945 in the morning. I guess this city is a high trust environment :D
Also Kevin Woodcock in Private Eye in the 90s had a cartoon working on this concept.

https://www.itsnicethat.com/articles/private-eye-a-cartoon-h...

https://media.itsnicethat.com/images/527cddc45c3e3c348f000ea...
Do you think there is any room for induced demand of timber? Like in 40 years we cut down a bunch of trees to sequester the carbon and people find new innovative things to do with the lumber, not just in construction but also in material science, packaging, etc.

I’m thinking about salt as an example. It is a by-product of many industries and cost have gone down a lot since the industrial revolution. I’m imagining that today’s salt demand is heftily induced as a result. I mean, it is cheap enough to spray on highways to melt the ice.
I admire your work. Can I ask how you deal with upkeep? I can't possibly keep updating software to deal with new JavaScript versions, and fielding reports like, "It fails on xx browser running on yy platform" are a real impediment to progress.
EDIT: Missing the terminating backslash, which surprisingly, is required.

The correct link (to Promit's comment) is: https://www.gamedev.net/forums/topic/666419-what-are-your-op...
https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...
I remember first hearing it in college, a few years after it came out. It was around the time I'd first heard of Baz Luhrmann for other reasons (Strictly Ballroom, Romeo + Juliet, and Moulin Rouge, all of which I loved at the time), so I feel like he was in the collective consciousness of my social circles, and that increased its popularity. I remember not finding out the true origins of the prose until quite a few years later.

I hadn't known about the sampling from "Everybody's Free" from R+J! Always something new to learn.

Listening to it again today... the parts about getting to know your parents before they're gone for good (mine have both passed since I first heard the song), and being kind to your knees (I'm still recovering from a knee injury from back in January), really resonated with me. One major thing I am pleased about is that I have managed to hold on to those "precious few" close friends over the years; my life would be very different without them.
Ask HN: Projections as an Alternative to HTTP APIs
I work at a large FinTech in a team responsible for a federated GraphQL service and, recently, related datasets in Redshift. We've come to see our datasets as another kind of API representing the same underlying data from our GraphQL API.

Our service is event sourced and follows CQRS and so our Redshift dataset is just another projection. This got me thinking, can other services simply use this projection directly instead of a GraphQL or REST API?

In general, services could provide an API for mutations that updates a projection which other services can directly query using SQL. Although services are accessing a database directly, it's not breaking encapsulation because the data is a projection intended for use by other services like an API.

There's a few benefits I see here. Firstly, if all services were modelled like this, querying multiple "services" could be handled with simple joins. Latency is reduced as data doesn't pass through an intermediate service. Scaling reads is straightforward. The same "API" works for both transactional and analytical use-cases. And finally, low-write services could be scaled down and no-write services could just be data pipelines.

So HN what do you think about a database as a service API?
I've done some more research.. looks like Prometheus + Loki could be the options of choice for me.. still not finished researching
They don't even contain a kernel - Docker containers use the host kernel. Container runtimes based on VMs like Firecracker's firecracker-containerd typically supply the kernel themselves.
> ext chat is meant to be async.

Not necessarily.

An SMS, sure. Because that's more of a message than a chat.

IRC, that's strictly chat and that doesn't have to persist. I can duck into and out of it as I wish.
> If the scientists' gravity analyses are correct, the lava tube near Marius Hills could easily house a large U.S. city such as Philadelphia

https://www.nbcnews.com/mach/science/gigantic-lava-tube-coul...
Hire a test engineer to manage all of that - it’s a full time job but an important one!
Nice use of complected. Rich, is that you?
I once found, in a second hand birding book (Collins Guide / Fågelguiden), a "congratulations to your degree"-card with an embossed crown on it, signed by Frida + Ruzzo. Some additional details lead me to the conclusion it was written by Anni-Frid Lyngstad of ABBA. (Ruzzo was Frida's late husband, who was prince of House of Reuss)
Toward customizable timber, grown in a lab
Ooof, yeah. I injured one of my knees back in January, and at 41... well, it's still recovering, and the physical therapy is slow. I'm otherwise in good health, but it's just frustrating to see how we don't heal as well as we get older.
People in my family have sacrificed meaningfully for these values as well, but I think it’s important to evaluate objectively whether those sacrifices are honored by the people who wield power or taken for granted.

https://www.cambridge.org/core/journals/perspectives-on-poli...:

> But the picture changes markedly when all three independent variables are included in the multivariate Model 4 and are tested against each other. The estimated impact of average citizens’ preferences drops precipitously, to a non-significant, near-zero level. Clearly the median citizen or “median voter” at the heart of theories of Majoritarian Electoral Democracy does not do well when put up against economic elites and organized interest groups. The chief predictions of pure theories of Majoritarian Electoral Democracy can be decisively rejected. Not only do ordinary citizens not have uniquely substantial power over policy decisions; they have little or no independent influence on policy at all.

> […]

> When both interest groups and affluent Americans oppose a policy it has an even lower likelihood of being adopted (these proposed policies consist primarily of tax increases). At the other extreme, high levels of support among both interest groups and affluent Americans increases the probability of adopting a policy change, but a strong status quo bias remains evident.
Stack – Seamless data version control for ML teams
Hi HN!

We are the co-founders of Stack, a tool for seamless ML data version-control and sharing inside organizations. We are developing this tool to help ML teams stop wasting time looking over old backups or old snapshots to do root cause analysis or retrain models.

We just opened a waitlist to access our beta, which we aim to release in September. You can sign up at: https://www.getstack.ai/

We are actively looking for feedback. Feel free to share any comments or thoughts!
Bought a book at a lovely 2nd-hand bookstore in Utrecht, Netherlands - reading it weeks later I noticed there was a 1.5 inch wooden-wedge jammed tight into the gutter. It's still there.

That's one serious bookmark!
It's probably also worth mentioning https://hefferon.net/linearalgebra/, https://hefferon.net/computation/index.html, and https://hefferon.net/proofs/index.html, all by one Jim Hefferon.
Advice about what? Not snarky, genuinely interested. I released v 1.0 of an open undergrad text just yesterday so I have some ideas anyway.
From Ariadne's update:

> The TSC [..] has concluded that there is a general need to begin decoupling hardcoded preferences for BusyBox from the distribution.

That's a bit stronger than just "we want to reorganize our script packaging". It still isn't explicitly "reducing dependencies on Busybox", but removing hardcoded dependencies is a prerequiste for the former.