skysight-inc/HackerNews huggingface dataset:
They might not be doing anything wrong, per se. But if your expectations are that 500/s is alot (or even 4000/s for log ingesting), then your architecture will reflect that.

Here's what they're doing:

> Now, when the Bearer Agent in a user's application sends log data to Bearer, it goes into the Envoy proxy. Envoy looks at the request and communicates with Redis to check things like rate limits, authorization details, and usage quotas. Next, the Rust application running alongside Envoy prepares the log data and passes it through Kinesis into an s3 bucket for storage. S3 then triggers our worker to fetch and process the data so Elastic Search can index it. At this point, our users can access the data in our dashboard.

Given their goal and their problems with GC I can tell you right off the bat probably what's the problem with their various architectures from day 1--too much simplistic string munging. If your idea of log ingestion is using in-language regex constructs to chop up strings into pieces, possibly wrapping them in abstract objects, then its predictable you're going to have GC issues, and memory bandwidth issues in general, and poor cache locality in data and code. But 99% of the time this is how people approach the issue.

What a problem like this cries out for is a streaming DFA architecture, using something like Ragel so you can operate on streams and output flat data structures. You could probably implement most of the application logic and I/O in your scripting language of choice, unoptimized GC and all, so long as you're not chopping up a gazillion log lines into a gazillion^2 strings. The latter approach will cause you grief in any language, whether it's JavaScript, Java, Go, Rust or C. The number of objects per connection should be and can be a small N. For example, at 10 distinct objects (incoming connection object, log line, data structure with decomposed metadata, output connection object, etc) per connection times 500 connections, that's 5000 objects per second. Even Python's and Ruby's GC wouldn't break a sweat handling that, even though internally it'd be closer to 10 * (2 or 3) objects.

Here's a big problem today: nobody writes their own HTTP library or JSON library; everybody uses the most popular ones. So right off the bat every ingestion call is going to generate hundreds or thousands of objects because popular third-party libraries generally suck in each request and explode it into huge, deeply nested data structures. Even in Rust. You can't optimize that inefficiency away. No amount of fearless concurrency, transactional memory, fastest-in-the-world hashing library, or coolest regular expression engine can even begin to compensate. You have to avoid it from day 1. But if your expectations about what's possible are wrong (including how tractable it is with some experience), it won't even occur to you that you can do better. Instead, you'll just recapitulate the same architectural sins in the next fastest language.
I mean, they often have guns. So they could do something if really necessary.
https://cryptopals.com/, by Matasano/NCC Group (originally @tptacek iirc)
I wish FFmpeg still had NDI support

NDI is super useful for low-latency local network streaming
Erm, no.

The first woman to get a PhD in computer science was a Wren, not a nun: Beatrice Worsley. https://en.wikipedia.org/wiki/Beatrice_Worsley of note

> When World War II ended, Worsley was the only Wren at the NRE to choose to remain in service.

She is tragically forgotten despite she wrote the first program to run on a Von Neumann architecture computer (that being the EDSAC) which you could simplify to say she wrote the first computer program as we today understand such.

Also, she got the first PhD in CS when CS wasn't even a thing yet.
Yeah, that makes sense - for a simple scheme like this I'm sure there's some expectation you can't just go around abusing bankruptcy like this.
Makes sense to me. Good luck!

> Ironically, this sort of cross functional work is highly valued, but if you're drawn to this path, you'll find yourself doing much more cutting through red tape than real work.

I find myself in this position. I've been with my company since it was tiny (10 employees total) to now (~150). I have functioned as a generalist for most of that time and I'm somewhat divided between architecture and coordination efforts versus actual coding. The original core dev team has mainly become the backend team and we have a newer frontend team. I recently shifted from the backend team to frontend, and now I'm able to help bridge between them so I'm hoping to help avoid some of those siloing pitfalls.
BSD isn’t like Linux where it’s the roughly the same kernel and user land but a different distribution of packages. OpenBSD and FreeBSD are entirely different operating systems albeit with a shared lineage.
"Daddy, where did Spot go?"
"He's living on a farm now little girl."
Generally speaking, the problem sets might seem harder but the average score is also lower, so it's a wash.
Do you live in the US? The country with the largest military in the world where we go around spreading destruction and chaos to “bring Democracy” to other countries?
This is a great explanation thank you! Makes much more sense now.

Will indeed read into the all-intra format.
Definitely not nothing. My point is that this treatment, however effective, isn't effective early enough in the disease's progression such that it allows us to reduce public health response measures by any significant amount, whereas a treatment that was very effective at an earlier stage of the disease would.
Thanks for the reply - that's great. I'm on mobile and looked for 'search', but now I see how to do it.
pay 500$ more and have it hand painted by some starving artists in NYC
The amount of time I spent on correcting wrong chapter marks (like the first four books of the Wheel of Time) is masochistic. But I am absolutely happy that I have the possibility of doing so after the conversion :)
I consider clap is a different type of dependency from lalrpop and regex. Clap is part of the application interface and most likely does not change that often. Lalrpop and regex are part of the domain logic and probably change fairly often.

Today, we can separate a Rust application into "app" and "lib". Most of the change would happen inside lib and app would only need to be compiled for infrequent changes and releases. This allows one to use all the fancy features of clap and mostly ignore the compile time cost.

It is annoying to separate a Rust application into "app" and "lib" though, so most people do not do it. They instead opt to replace clap with gumdrop. I think both approaches are valid.
German University of Prague.

If you can read German or know how to use Google translate
https://de.m.wikipedia.org/wiki/Deutsche_Universit%C3%A4t_Pr...

Insigniestreit....
Czechoslovakia, "victim" of ww2...
Not unless they buy uBlock first, which is what they did with Pocket. And unlike Pocket, which was already a company looking to get bought, uBlock is very much not that.
The founders of NLP used to joke that there are many paths to mental health, but none so lucrative as Freudian psychotherapy.

I tend to try to seek out CBT people, but I found that a lot of them started out as classicists themselves, and then we find our way back into being "non-interventionists".

My mental model now is that the cycle of life goes something like this: Born -> WTF is Going on Here? -> Death

Maybe that's cynical, but it helps me believe that I'm not the only one who is fucked up between the ears .
Those are gonna be some expensive coats.
New Review Apps: Expanded Options for Greater Control, Automation, Easier Access
> the Wikipedia article explains the publication history of the idea and has references to the relevant papers

The idea referred to in that article and its references is the speculative hypothesis I mentioned earlier about the total energy of the universe. None of those references do a calculation such as you described, showing that "disassembly" of a gravitating body like a star ends up with zero energy left over.
> I'm not sure what you mean, it's not like procrastinating is a mental disorder or permanent disability like being blind

It can be symptom of a mental disorder when it impacts normal functioning.
I bet his slaves would beg to differ....
Agreed. I have no problem with companies engaging their community this way, but the right thing to do is to openly share what is contributed.
https://en.wikipedia.org/wiki/Superconducting_Super_Collider

The US has one too.
I think that the restriction to 32-bit apps is based on the fact, that only the patents covering the early iterations of x86 ISA have expired. Microsoft wouldn't want to be sued by Intel for infringing on later Tech (like x86-64), so emulation is still restricted to this subset.
Useful Git commands for common Git tasks
Giant worry stone [but the other things are cooler, IMO]
With so much video processing research using neural networks now I hope ffmpeg gets better support for it. They have some filters that use them, like sr[0] for super resolution and dnn_processing for general processing, but the user experience isn't great. They need a model file that's not included, and you need to train one yourself since there doesn't seem to be any included. Hopefully they add better support in the future, together with more dnn filters.

[0] https://ffmpeg.org/ffmpeg-filters.html#sr
Was that the one where the captain let’s out a big sigh, pushed a button, and the ship takes off into space?
No it absolutely isn't. I worked at Dropbox for years, there are new codebases being written in Rust all the time, including in critical code paths.
I can't wait for the James Webb Space Telescope to launch -- really excited to see what we've been blind to so far. Sadly, it's been delayed again due to COVID-19 (it was most recently scheduled for March of next year), but _whenever_ it launches it's sure to bring us eye-popping new images of space.
If you are the representative of country, I suppose so.
I worked there, it's totally false.
We've been hearing FreeBSD is dying since before it's been the year of linux on the desktop.

Is FreeBSD ever going to be hip and trendy? Probably not, it's not 1994 anymore. But, I don't need a hip amd trendy OS, I need an OS that provides a stable base for me to work upon and supports enough hardware that I can chose decent parts. As a bonus, I've found it easy enough to muck with the internals when it has benefited me or my employer.
You mean the same constitution that allowed slavery and counted slaves as 3/5 of a person?
Copy-paste a CodePen contact form and turn it into reactive UI under 9 mins
Would you have rejected a Harvard admit ? Are you just salty you didn't get in ?
I thought about calling it "search" but went for "add" since that is technically more accurate with the URL inputs--and it's already a fairly common occurrence for people to enter search terms in those instead of URLs. Though your comment makes me thing that maybe I should just call it "search"...
The original ARM team was given a power budget of 1 watt, but when the first silicon was made they found they had accidentally made it so efficient that it ran on a tenth of a watt.
There is a nice little first-hand history of the early days of ARM here: https://www.theregister.com/Print/2012/05/03/unsung_heroes_o...
The Ten Commandments of Computer Ethics
Springer Nature Transformative Open Access Agreement
Ultimately this gives the admissions committee a lot less transparency in terms of whom they choose to admit.

In light of recent scandals, I am not sure that is a good thing. It may be well-intentioned today, but will ultimately be to the benefit of legacy and donor applicants, and to the detriment of others.

Perhaps Harvard with its endowment doesn't need the money and won't give in to the temptation, but this also sets the standard for other universities - and many of them could use the money.
The Canadian admission system and the American system is very different to start with.

In Waterloo Engineering (and presumably other schools as well), they take your high school marks, add some adjustment factor for your extracurriculars and essays, and then pick the top N students.

In top US schools, they practice holistic admissions, which means they pick N students from the applicants, without specifying an explicit ranking system.

In the first case, an applicant can be relatively aware of where they stand based on the marks they apply with, and they know that the process used to select will be mostly transparent.

In the second case, from the applicant's perspective, it's pretty much a black box as to what the outcomes will be. When one of only 2 predictable parts (SAT scores and grades) are removed, I think it's understandable that there's some concern.
There are not many tasks you can complete during such a short break, especially if you don't know if the break will last 20 seconds or 2 minutes.
Picking MI to stand for the midwest, the median wage for a Software Developer, Applications there is $89,460 vs a US median of $107,510 and CA median of $130,440. https://www.onetonline.org/link/localwages/15-1132.00?st=MI&...

There's a much greater spread of wages in the US than in other comparable countries, so it's attractive for those of us towards the higher end of the scale. Coming from the UK though I do miss the feeling of freedom from being able to take a long break between jobs and not having to worry about healthcare.
The United States was not the only country to ever employ chattel slavery - it's not unique in that regard. It is unique in that it's the only country to end slavery due to civil war - a fact I think is very important in this discussion. There are to this day a number of "rebels" as they call themselves who believe themselves to be intellectually and morally superior to black people, they want to keep black and whites separated, and they want to remind blacks of their inferiority by prominently displaying statues of Confederate Generals in the public square and promulgating the use of terms reminding blacks of their inferior social standing. Terms such as blackmail, blacklist, and master/slave. There's way more behind this than just slavery and snowflakes getting their feelings hurt.
I'll just leave this here: https://youtu.be/y3RIHnK0_NE

google + boston dynamics = skynet :p
I love FreeBSD. Been using it in a server context for going on 20 years now.

Trying to use it on a desktop takes me back to, well, trying to use Linux on the desktop 20 years ago.

Recently started shopping different distros because I’m not comfortable with the direction Ubuntu is going and rather than upgrade my 18.04 when it’s EOL’d, I’m looking at just hopping to something else for my work laptop.

Took hours to get X working with my laptop’s graphics card (though most resources I found online said it wouldn’t work at all, so that was a nice bonus). Not “get accelerated graphics working” just “get X to start”.

Spent another couple hours screwing around trying to get the trackpad working. And not just “get it working to my liking” but get it working at all. Turns out it’s entirely not supported as a lot of modern laptops uses a I2C connection rather than USB/PS2/etc, and there’s just no driver support for that. Never did get that resolved.

A lot of desktop apps that I use are either unavailable or require a lot of dicking around to use at all. There is no Slack desktop client. You cannot run Docker natively, and instead I ended up setting up a Debian guest OS under bhyve, which harkens back to how Docker is run on Windows/OSX and all the caveats that entails.

You’ll pry my FreeBSD servers from my cold dead hands, but it’s certainly not my go-to for a desktop OS.
They already have a good laptop ARM chip....
Careful now. Everybody knew the Jews were persecuted, had their rights and belongings taken away and were rounded up in ghettos and camps, from where they eventually were "evacuated to the East". What civilians could justifiably claim is they didn't know where or how the killing happened. Information about this was kept vague. While the concentration camps where public knowledge, the extermination camps were kept secret. So German civilians could say they didn't know about Babi Yar or Treblinka and the gas chambers - but that is a far cry from saying they didn't know about the persecution and genocide.

It should also be noted that the T4 euthanasia program was scaled back due to public protests. This shows not only that people actually knew it happened (even if it was kind-of secret) but also that protest could actually work. The Nazi party actually cared a great deal about public opinion which is why propaganda was so important to them. One of the reasons the extermination camps were kept secret is they feared protests.

There were also the Rosenstrasse protests, where (non-Jewish) wives of Jewish men protested against deportation of their men. This actually caused the men to be released.

So "we didn't know and couldn't have done anything anyway" is ahistorical. I can understand why people cling to it, but it is a dangerous myth.
For systemic inflammation problems (including uveitis, joint pain, and intermittent brain fog), have you experimented with probiotics, especially acidophilus (including L-92), as well as cutting gluten and other potential allergens from your diet?
Sloppy compared to what? What would you point to as better? Where would you put the urethra, for example?

Many of your examples are contrived. Sure they have some failure modes, but they also are succeeding trillions of times a day.

I’d bet a lot of money a “human designed human” would not perform better, except for taking ideas from other humans, and only when done in very limited cases.
I love ffmpeg but those release notes suck.

If all you can say is how much time there's been, why isn't this just 4.2.4?

I suspect there are major features and improvements buried in the 30 pages of commits.

https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/refs/heads/rel...
The fact that sexual orientation has anything to do with a person's own sex is crazy. We need to normalize better terminology that just says who you're attracted to.

We only really need it to determine whether we're compatible with other people anyways: I like A + you are A + you like B + I am B = compatible. (Where A and B might be different or equal)

Ideally we'd live in a world where even that doesn't matter and everyone just chooses partners based on whether they enjoy that person's company etc. but I feel like that world is much further into the future :)
Wait, so an m4b stores playback position in the file itself? As in, the checksum will change and file syncers like Syncthing will re-upload every time I hit pause?
There are many potential answers to the Fermi Paradox.

https://en.wikipedia.org/wiki/Fermi_paradox
Open source is great for tech-savvy professionals. Pick what you like, change it if you want, and it will never be taken away from you.

Open source has completely and consistently failed in the consumer market, and its adoption in the enterprise market has been mixed. In those markets users are either not knowledgeable enough or too busy to do the things open source asks of them.

Syncthing is cherry picked as an example in the original article because it's very atypical for open source. For open source it has a superb degree of polish and is quite easy to use.

Still I wonder how it would fare in a professional environment. How do I invite someone to syncthing the way I can with dropbox? How do I integrate it with my corporate management or IAM solution? How do I...? I assume there would be a long tail of weaknesses.

I also wonder what happens if syncthing gets abandoned or needs to undergo some major redesign to cope with some significant change to UI technology, OSes, or platforms. Who will do that?

Don't get me wrong. I like that syncthing exists and even that it lacks those features, but I also realize that it does not address the whole market and probably never will.

Open source as it stands has some fairly significant limits.

BTW the pathology described in this article is also seen in open source projects vying for mindshare, adoption, contributors, sponsorship, etc. It's not specific to commercial software. I do agree that it's more prevalent in commercial software because companies need to find a steady source of revenue, but it's gotten worse in recent years because nobody pays for the most basic core features of software that they want anymore. People only pay for fluff, or pay if they're wrangled into a closed ecosystem and then brought to the point of payment via some kind of dark pattern. You get what you pay for, and people no longer pay for just good software.
FFmpeg is hands down one of the most powerful and feature-packed tools I've used out there.[0] The associated complexity is also daunting, but thankfully there's a lot of documentation out there and it reflects the low level nuances of audiovisual formats.

I highly recommend anyone struggling to utilize it to write wrapper scripts around it so you only need to figure out things once. Here are some things I've done with it by that approach:

* Extracting any embedded subtitle files from MKVs. Nice if I want to search them or make changes.

* Back when GIFs were more popular, I converted any that were over 3MB to video to save space. If the output wasn't small enough, it would do a second pass with different settings to get it more compact. Not needed that much these days.

* "Barcodes" for videos, that is, it takes every second converted to a vertical sliver and combined you get an overview of how the average color of the film changes through its duration.

* A tool for creating video excerpts that lets me specify a start time and end time in more flexible timestamp formatting, and other things like a simple parameter for the output width.[1] It also allowed specifying a target filesize and did the math so the right bitrate would be chosen. I even include metadata so I know which original file it was made from and the parameters specified.

* Thumbnail previews. A lot of file sharing sites will include a file that includes some timestamped screenshots in a grid with encoding information at the top. This is good for movies so you see a high-level overview. The best part about doing this myself is that I could make it highly configurable, like choosing exactly how many images I want, the interval, whether I want timestamps, etc.

Note, for some of these, I also needed ImageMagick.

Also, when compiled with the right flags and libraries, FFmpeg has some really neat features: things like embedding subtitles, stabilizing video, hiding logos, etc. I recommend looking into the filters.

Thank you for all the manpower that goes into the project!

[0]: Two other ones that are also powerful are ImageMagick and Pandoc.

[1]: I initially wrote this in Bash, but later converted the code to Python to better handle command line arguments and allow things like using config files.
CSS Modules didn't become the default way to go until sometime in 2016. Before that point people were still stumbling over how to manage their CSS in a component-oriented fashion, and it was only once that happened did React "win".
>complicated codebases) can also change in ways you don't like. You can further mitigate this risk by using simple programs that have reached a "finished" state.

That's exactly the kind of thing I'm working on, to be able to do simple 2D GUIs without having to directly depend on the complex depths and uncertain future of any specific GUI library: https://github.com/jeffhain/jolikit/blob/master/README-BWD.m...

I often had personal GUIs idea but never implemented them due to not wanting to depend on a particular library. Now that I built this abstraction layer, I'm starting to create GUIs.

I feel like building some software for you that depends on proprietary code, or on code too vast and deep to be easily "kept afloat" on the platforms of the day, is basically enslaving yourself to others, or building your house inside of a prison or on the Titanic.