skysight-inc/HackerNews huggingface dataset:
Building the Ultimate x86 and Arm Cluster-in-a-Box
Months ago I posted a link to Arc (https://news.ycombinator.com/item?id=26573930) a declarative method for defining repeatable data pipelines which execute against Apache Spark (https://spark.apache.org/).

Today I would like to present a proof-of-concept implementation of the Arc declarative ETL framework (https://arc.tripl.ai) against Apache Datafusion (https://arrow.apache.org/datafusion/) which is an Ansi SQL (Postgres) execution engine based upon Apache Arrow and built with Rust.

The idea of providing a declarative 'configuration' language for defining data pipelines was planned from the beginning of the Arc project to allow changing execution engines without having to rewrite the base business logic (the part that is valuable to your business). Instead, by defining an abstraction layer, we can change the execution engine and run the same logic with different execution characteristics.

The benefit of DataFusion over Apache Spark is a significant increase in speed and reduction in execution resource requirements. Even through a Docker-for-Mac inefficiency layer the same job completes in ~4 seconds with DataFusion vs ~24 seconds with Apache Spark (including JVM startup time). Without Docker-for-Mac layer end-to-end execution times of 0.5 second for the same example job (TPC-H) is possible. * the aim is not to start a benchmarking flamewar but to provide some indicative data *.

The purpose of this post is to gather feedback from the community whether you would use a tool like this, what features would be required for you to use it (MVP) or whether you would be interested in contributing to the project. I would also like to highlight the excellent work being done by the DataFusion/Arrow (and Apache) community for providing such amazing tools to us all as open source projects.

Edit: format
Microsoft Research - Sr Research Software Engineer - Redmond, WA or remote

Microsoft Research is building new tools to automate the search for unknown malicious software. We are the growing, evolving team behind Microsoft’s AVML, OneFuzz, Project Freta, and other new security tooling initiatives. The goal of our group is to make a difference for Microsoft’s customers by building high-trust discovery mechanisms into Azure, the world’s computer. Our group builds and sometimes Open Sources memory-safe security tooling with the Rust language. Our distributed team includes remote work opportunities and regularly interfaces with engineering teams across the globe. We invite you to consider joining our team as we learn together.

In this role, you will be part of a small team environment involved in rapid prototyping, compatibility engineering, and the invention of new capabilities in the security space. If you’re excited about joining the security field, growing an existing security career, learning or continuing with Rust, or building a more trustworthy cloud- please seek out this role.

* Data security research
* Rust

https://careers.microsoft.com/us/en/job/1214584/Sr-Research-...
Researchers Have a Method to Spot Reddit’s State-Backed Trolls
At a high level there are pros/cons to JS-based frameworks and "compiles to native" frameworks. Performance is one part of that, but also ability to bind to existing native code also, among other things.
I totally understand. This is a fun way to do it.

A cheaper option would also be to run a lot of virtual machines on a single device and only assign them limited CPU and RAM budgets.

You can actually do this all within a single Raspberry Pi 4 8GB with ESXi: https://www.servethehome.com/getting-started-with-vmware-esx...

Not going to lie: I'd still enjoy having one of these Quad-Pi boards to play with, though.
Considering the timezone, her getting up at 4AM GMT+8 is ahead of the world, but still 3 hours before Australia.

Australia gets up before everyone, they should be dominating.
This has entered programming language design. It's not longer "educate the programmer so he doesn't make stupid mistakes", but "design the language so that stupid mistakes are detected by the compiler". Mechanical verification is far more reliable than hoping people don't make mistakes.
> If no jobs exist at all those voters will bring in someone who will tax companies like yours and spread the wealth.

And those companies will calculate the ROI for moving into a lower tax burdensome area. After the company leaves the people wont have a job or the tax revenue.
How do they draw all the figures in the routines? How long would it take a graphic designer to do one? Is there an automate way? I'm just wondering because the figures look so consistent.
Ask HN: Lack of useful music sharing in a social media context?
Now that Spotify's 2021 Wrapped was just released, it got me to thinking that there doesn't seem to be a platform that encourages sharing music recommendations, albums, playlists, etc. between your friends. It seems on Spotify the only way to "find friends" is to connect the app through Facebook which is not ideal. Do people just not care about this feature? Are people generally reserved about their music tastes and prefer to keep things private?

I remember back in the day there was Last.fm and the concept of scrobbling as an interesting way of seeing what your friends are listening to. Playing your song on Myspace was also a great way to show your friends what music you're interested in and finding new bands. Why is this so difficult in 2021?
Yes but Dart although an OK language is clearly not up to C# brilliance.

A lot of libraries don't exist in Dart or are platform specific (for example you can do SFTP only on Android and iOS, but not on desktop).

On the other hand, Flutter as a UI framework is much better than anything Microsoft makes (and they make a lot!). It's way more easy to build something beautiful with flutter.
Study suggests Sun is likely an unaccounted source of the Earth's water
Bug report: If you navigate to the start page and click the Get Started button, the browser’s Back button stops working (you can’t return to the start page by going back).
YouTube – Dislike Count Gone
I know it has been 2 weeks. I was hoping for it to be reverted the next day. But here we are.

How the heck am I supposed to figure out credibility of the information in a video. Especially the educational ones.

Hopefully, it's not to stay.
React: Class Components vs. Function Components
Xamarin (soon to be MAUI) is the best option in my opinion, especially if you enjoy C#.
Capuchin Monkeys – Keeping Them as Pets
Here is another one restored to a better shape [1], close-up [2]. I must admit it doesn't look as perfect as replicas (e.g. in movies), obviously we have much better and precise technology/craftsmanship these days. Probably "as new" may not differ that much from what we restore to.

1. https://www.smithsonianmag.com/smart-news/archaeology-intern...

2. https://www.archaeology.org/news/8493-200302-germany-roman-d...
Cray Customer Service – Memories (1979 to 2014) by Charles Clark
I don’t know: we don’t do the background check until the employee has said they want the job (by signing a conditional offer) precisely to avoid doing unnecessary background checks. It’s not a cost issue, but a moral one (fairness). When they sign the offer they explicitly agree to a background check.

Obviously they’d be foolish to give notice until the check comes back clean but it’s typically a same day thing.

Never had a problem with a background check that didn’t come up in a reference check anyway. Personally I don’t even care about them except for people who handle money, and would rather only do them for those jobs.
What does the full version have, compared to this web version?
SEEKING WORK | Remote only, based in Canada/USA
Rid Industries is a small development firm comprised of (5) moonlighting engineers. We're looking for part time development work (15-20 hours per engineer per week) in the web application or mobile space. We have experience building out large scale service applications as well as smaller mobile/web applications.

We'd be happy to give you a tour of our portfolio!

Some technologies we work with include

• Python / Django

• Java

• Go

• Ruby / Rails

• JavaScript / Typescript / React

• iOS / Android

• Various database technologies (relational/nosql)

You can learn more about us at https://www.rid.industries or shoot us an email at team (at) rid (dot) industries
See my comments in this thread above:

https://news.ycombinator.com/item?id=29404255

A low level sandboxed file system api (which is 100% coming) will let developers do exactly what we want in incredible future proof way and not be tied to out of date and none extendable apis.

I want sql in the browser, in my opinion websql was the wrong way to to it, Wasm is.
It's open source and we're looking for contributors. Let us know if you'd like to help out.
From the article: Nearly 91 percent of the U.S. population has had at least one dose of the two-dose measles, mumps and rubella, or MMR, vaccine...

The article says 94% of current mumps cases are amongst the vaccinated, but the number of cases is quite small: they give 231 in 2003, 6584 in 2006.

Isn't this a case of random fluctuations in small numbers of infections, combined with a high vaccination rate? We know that vaccination isn't 100% effective in preventing disease, so even with a 94% partial(!) vaccination rate, we'd expect some mumps.

Posting this link at all strikes me as helping to normalize anti-vax propaganda. It's not a good article, it's in the "objectivity" vein, where it leaves out a lot of nuance to "objectively" present a viewpoint.
How about entering in a URL from an offline mention, whether that's a print ad, being mentioned in a podcast or shouted from the stage of an indie band playing in a bar? When I started a band in the 90s, there was a great deal of consideration made in choosing the name for there being an available domain name (at the time there were about two dozen English words which had not been registered as domain names, of which exactly one was a potential band name (osteoclasts) but we decided against that—we ended up with the suboptimal dream-in-color.net since the name was unavailable with neither dashless nor .com/.org options available). One important thing was that the domain name would be something that a semi-drunk person might remember later.

Non-technical people will likely not recognize four three-digit numbers separated by periods as being an address for a website.

And phone numbers have been largely supplanted by contact lists on phones (I've occasionally forgotten my own number). Looking over my call history, out of the last fifty outgoing calls, six are not in my phone book and all but one of these were dialed directly from search results on the phone.
Fair point. But again, that would be less of an issue if there were like 3 or 4 big search engines with different algorithms, all changing and adapting on their own schedules. Would be harder to game the system then.
Please people, step out of your USA bubble.
I (Dutch guy) haven't had a more than 4 day workweek in the last 20 years.

It is _your_ choice to "work" 5 days.

The 40 hour workweek wasn't created for the worker's benefit, it was created because the owner of the factory saw that that was the most efficient.
Paying people to work longer didn't give him benefit.
One of the solutions I saw today was in Intcode (the language from 2019), it was pretty incredible.
What's the difference between this and Blazor?
You're so right. It's sad that even if you remove 99% of something, to the outside it looks like you are removing 0% because you have no reference.
Any chance you could support .pm and .it domains? Would love to move over my remaining domains to you
A Large Numbers Hypothesis Returns
CIV
80gb of ram is now mid range???
Probably because its origins are as a UWP-first framework which runs natively on platforms that support it. Skia for Linux was likely added on later to allow the original app to run on other platforms.
A trip to Iceland would cost 5x-10x times more though (not that I am saying it is not worth 10x+).
> Can an employee opt-out?

I am curious to know this for a different reason: if opt-out is possible, it will become a standard term in all employment contracts from now on, so this law might as well not exist. We know all too well from the cookie law that a law with opt-out turns completely useless before the ink is even dry.
Smoking a pack a day for 30 years is not really the same thing as making a momentary error in judgement.

Yes, non-smokers can get lung cancer, too. But at a far lower frequency.
Amazon Privacy greenfield project seeks REMOTE developers

(Also maybe mention me since I want credit)

SDE I: https://lnkd.in/d4mh3nMN

SDE II: https://lnkd.in/d2czJAS5

SDE III: https://lnkd.in/daCzGjbp

TPM: https://lnkd.in/dgmJSQUF
Sounds alright that people complain about not knowing about external storage. They will be educated at that point.

This sounds like something that can only be mandated. It doesn't make much from a business point of view.
It most certainly is easier to deal with the in-process version. I'm specifically referring to database transactions, which are much harder to have available in a distributed context. That failed network call complicates your life so much.
Thanks for sharing!

I'm a solo founder. Still looking for my better half/halves while building the startup. And because of this, I spend a decent amount of time reaching out to other founders for advice & mentorship.

Getting a coach is on my to-do list once we have enough traction/funding.
It’s definitely not just a US thing. In NZ the vast majority of people work 5 days. I suspect probably the same in AU and UK.
If the job involves handling money (CFO’s office, especially A/P and A/R). Security guard and such.

Otherwise I think that they are an unwarranted intrusion, but our HR and Legal say otherwise.
Location: Paris, France

Remote: No

Willing to relocate: No

Technologies: Electronics design (schematic design, PCB layout, testing), embedded systems firmware (bare-metal, RTOS or Linux kernel), FPGA

Résumé/CV: https://www.linkedin.com/in/matthias-rabault, full CV including details about current position only available upon request

Email: matthias d o t rabault at gmail.com

I'm looking for a position that is focused on electronics design, but I'd also be happy with a hybrid electronics/firmware or electronics/FPGA job as long as the electronics part is prominent.
Moth Minds: Fund individuals doing work you believe in
You are building your skills while you are working. If you can find time outside of work, you are just accelerating things.
First Resonance | Los Angeles | Onsite | Full Time | https://www.firstresonance.io/careers
First Resonance was founded by engineers who worked at companies like SpaceX, NASA, Zoox, Toyota, and more. We saw a shift to a faster, more dynamic world and found existing manufacturing systems insufficient in handling the rate of innovation needed to deliver world-changing products.

Our team creates and operates the software that advances factory operations of companies building electric airplanes, 3D-printed rockets, autonomous vehicles, and more. Today, our customers benefit from a far better UX from engineering to the factory floor, fast and democratic access to data and powerful APIs to automate hardware processes.

Our tech stack includes Python, Postgres, SQLAlchemy, GraphQL, Typescript and React

We're hiring for:

* Backend Software Engineering: https://boards.greenhouse.io/firstresonance/jobs/4299082003

* Senior Software Engineering: https://boards.greenhouse.io/firstresonance/jobs/4299069003

* Account Executive: https://boards.greenhouse.io/firstresonance/jobs/4572159003

* Many more! https://www.firstresonance.io/careers

I'm happy to set up 30 minutes to chat, ping me at firstname @ firstresonance.io

Skyler
Blazor is a way to run native code in your browser. Uno is a way to run UWP code as a cross-platform app.
S&P Global - ClariFI team | Remote (US) or Office (Boston, NYC, Denver) | Full Time

ClariFI is a platform for constructing, testing, and deploying quantitative equity strategies, used by Investment Management professionals across the globe. As part of the team, you’ll be working with a variety of languages and platforms, but primarily focusing on Java and Scala. We incorporate Functional Programming paradigms into several layers of our technology stack, so this is an excellent opportunity to learn or grow these skills.

https://spgi.wd5.myworkdayjobs.com/SPGI_Careers/job/Virtual-...
https://spgi.wd5.myworkdayjobs.com/SPGI_Careers/job/Virtual-...

We are also looking for database developers to write and optimize SQL queries and evaluate data quality for use in quantitative strategies. Postgres experience is preferred, but MS SQL Server is also OK.

https://spgi.wd5.myworkdayjobs.com/SPGI_Careers/job/Boston-M...
Now Zink (GL over Vulkan) runs faster than OpenGL itself on supported platforms.
Controlling a website with electronic drums? Sure, just try WEBMIDI.js v3
Between this and their employee-hostile changes to equity vesting, Stripe went from the company I was most interested in and one that I wouldn’t bother interviewing with.
Well, that's a blast from the past. I knew a bunch of the folks involved from undergrad and did some mostly informal beta testing of Steve's games from Infocom and Legend. As I recall, I also provided some long-ago tech advice for Boffo in exchange for some pizza.

My favorite game of Steve's was actually one of Infocom's only modestly successful titles: A Mind Forever Voyaging. It was more truly interactive fiction than a puzzle game which probably explains both why I really liked it and why it was less popular. Planetfall was another Infocom favorite. (There's a very inside joke about me in it.)

(Steve also wrote Hitchhiker's Guide to the Galaxy with Douglas Adams.)
Please define baity. The title on the page isn't deceptive or vague.
COMPASS Pathways | New York, London, San Francisco or Remote (UK/US) | Remote OK | Visa sponsorship available | https://compasspathways.com/ We are a mental health care company. Our first programme is researching how psilocybin therapy could help people with treatment-resistant depression. A great opportunity to make an impact from day zero and promote best practices. We are a public company.
Also see HN discussions about us: https://news.ycombinator.com/item?id=29169160

Tech Stack: Next.js/Blitz/Chakra UI/Python/AWS/Terraform/Github

* Senior Full Stack Engineer: https://compasspathways.bamboohr.com/jobs/view.php?id=30

* Senior Devops Engineer (AWS, Terraform): email me

* IT support engineer: https://compasspathways.bamboohr.com/jobs/view.php?id=51

* iOS engineer: https://compasspathways.bamboohr.com/jobs/view.php?id=26

If you have any questions, email me at demeter.sztanko@compasspathways.com
Which is what the OP is suggesting Google should have done in this case
Many of the ski resort companies do offer housing, which is great. There are two primary issues with that at the moment: not enough housing and, the bigger one, when your housing is directly tied to your job what happens if you are fired? Colorado, for example, is an "at will" state--that means someone can be let go for any or no reason with no notice. If the company is providing housing, then the person is forced to up and move...but where do they move to?
Assuming it's a built-in iOS/Android feature, hopefully app review would catch apps trying to circumvent the fake permissions, and the risk of blocking real users would probably discourage them (I almost never use SMS unless it's for a verification code, and I have no reason to keep those messages).
Indeed. I did simple calculation for myself. We kind of split a service in 5 microservices. Now if I look at e.g 5% integration related issues between 2 services, which would obviously not exist in monolith, we are actually looking > 20% chances of error in an end-to-end transaction.

To me this is pretty high chance of error on day to day basis. And in our case some errors are convoluted, so possibly getting missed and causing slow data corruption.
He was literally talking about how it was his choice to stay, and you're calling him a slave. Over only getting half a day off on Christmas Eve, which is not even a holiday. I honestly cannot tell what is sarcasm anymore.
It is using a WebRTC Data Channel. You can send binary/text data over the same connection that is receiving the media.

The other exciting thing about Data Channels is that you can send lossy/unordered messages and get flow control. You can know how much data you can send and how fast. You can build faster and more responsive data APIs that way!
Definitely also still the case in Denmark, which is a stone's throw away from the Dutch. Don't think this is in any way illustrative of a US-bubble.
There's always some cool stuff done with Raku, I'm giving it a chance this year as my "second language", we'll see how it goes.
Hell I’m in the USA and I work a four day work week (four six hour days!!). Last time I was job hunting I simply told every prospective employer my desired schedule and I found one small organization that was fine with that. It’s life changing!
A related problem is, how do you make sure the domain goes to whoever values it most, as opposed to whoever thought of the name first? In the .com space a lot of good domain names are taken up by people who are not putting it to good use. I am not talking about parked domains, but people who upload an "under construction" gif and then don't update the page for the next 5 years. Maybe domain names should cost more.
The Ghost of Itanium – and HPC – Give HPE Long Sought Profits
What's the best way to learn "PROPER" Python? I am doing exercises a lot but not learning anything new, because my current knowledge mostly solves problems. Any tips / recommendations?
Well said, cheers to a free world :)
Themis | Senior Rails or React Engineer | NYC (HQ) and/or Remote (Anywhere between Texas <=> Ukraine) | Full-time

We are github for governance.

Governance of fintech doesn't have to be an obstacle. New finance companies need tools which simplify governance on a daily basis to help them focus on their platform's core value.

Themis streamlines sharing the information necessary to demonstrate good governance. We're currently working with new disruptive companies in defi, crypto, and neobanks and established community/larger banks. Essentially, Themis is putting governance on rails for companies who previously had to maintain large teams of employees for governance.

We have seed funding and are in pilot testing with our first wave of customers. Read about us at https://www.askthemis.com/

We're seeking a senior engineer to join the founding team to lead the engineering effort, provide guidance on architectural choices and deliver new functionality.

We're also seeking junior engineers who are looking for rapid growth. We strongly believe in mentoring and helping team members grow to their career goals.

Senior Rails or React Engineer https://jobs.wrkhq.com/themis/20275

Rails Engineer https://jobs.wrkhq.com/themis/22265

Tech: React/Ruby on Rails/AWS
I was surprised the other day that my photos weren't being stored locally on my iphone, but in the cloud. I finally found a setting that turned that "feature" off. Obviously, it had defaulted to "on".
Busybox: A small package with a large code base
What other sites do you recommend with less math-focused problems?
Please do not post nationalistic flamewar comments to HN. It's one of the things we're trying to avoid here.

https://news.ycombinator.com/newsguidelines.html
People are incredibly quick at both skimming/parsing the problem, coupled with a language like python where you can generally express stuff incredibly concisely and not have to worry too much about types, it makes it very quick.