skysight-inc/HackerNews huggingface dataset:
Ask HN: Which tech to use for real time data processing? Kafka or Kinesis?
Hi hackers,

We have a requirement in our organisation where a continuous stream of balance info events for users are being generated(Essentially an event has information of user's current balance after this event, balance type - debit/credit/coupon etc).

Our goal is to process this continuous stream of data and get below information out of it:
1. balance snapshot at the end of each day for each user (starting and ending balance for each user).
2. invoice for each user (starting balance, ending balance, coupons applied etc) at the end of the month.

I'm confused as to which data stream processing tool would fit our use case better. Kafka or Amazon Kinesis.

Since I'm new to designing systems, any thoughts on which tool to use and how to achieve it (high level design, database design) would be really helpful.

Thanks.
Indeed it's economically worthy to have a safe market rather than a rushed one that will backfire.
Note that they had issues just pulling a simple communications cable, I can't imagine that a 2ft diameter rubber tube is going to be easier.

However, attempts to install the cables have been unsuccessful so far, Maj. Gen. Bancha Duriyaphan said. One cable suffered water damage as divers transported it "around small passages." Teams are attempting to take in a new one.

https://www.cnn.com/2018/07/04/asia/thai-cave-rescue-intl/in...
What animation tools they used?

Could be anything, but since there is a lot of 3D my bet would be on Cinema 4D with some cel shaders.

See this: http://www.eyedesyn.com/tutorials/using-cel-shader-to-cast-s...
Google Play Music has podcasts, that might be the closest thing.
Isn't it ethically preferrable for Waymo to push their own, better, safer system, as far as they can, rather than to improve competitor systems in a lesser way by sharing their secret sauce that competitors can only partially implement?
I think you are on to something. There are powers with an interest in keeping the status quo....but I think it is more often than not on an issue by issue basis and less often about overall social control.

Fact is some folks straight up vote against their best interests and others don't vote, ultimately just bad choices are a part of democracy.....and that plays a big part.

Also unions in the us really do suck sometimes. My wife is a part of the teachers union and it is a mess.
Glad to hear it, but eternal vigilance etc. Keep an eye on them.
> Theoretically I could watch amazon prime by plugging something that doesn't spy into an HDMI, and the TV can report very little spy data back.

I’d be interested in people’s recommendations for this. Can you use a raspberry pi, for instance, and get good performance, and access to online services like Netflix? Do most people use some open source set top box software, or just raspbian and a browser?
I don’t have numbers but I’d bet 0. I think it’s more a process issue though, and there might be opportunities.

My understanding is that big companies spend a ton of money in events at top schools, where they collect a pool of candidates for internship. This is already a big work and fills the pipeline with enough candidates, so they don’t have to search further.

In principle, this doesn’t prevent a student in a community college from applying, but getting into the process at the right time might be cumbersome. In addition, the probability of success is low, just looking at the number of applications vs positions.

I think there might be a big opportunity, however, in aggregating pool of candidates, and for example get paid from the companies to acccess these resumes. A sort of niche linkedin for internship.

Does anyone have numbers on how much a big tech company spend to attend to a school for recruiting interns?
Well, we often differentiate between "security by obscurity" and "obfuscation" since former became a common insult/no-no. The former is when enemies not knowing how something works is the security strategy. Lots of those systems fall easily. The latter is when it's an extra defense on top of otherwise good, security practices or at least a baseline not much different than what's common.

I've found obfuscation an essential tool to delaying or defeating high-strength attackers. I tried to apply it to every level of the computing stack in my research. Here's a write-up with examples you could use:

https://news.ycombinator.com/item?id=11856351
So you're saying many libertarians got caught up in the Neo-Con fervor of the early 2000s.

I don't think anyone will disagree. Trends are trends. Those who booed Ron Paul in 2008 started agreeing that Iraq was a mistake in 2016. https://www.youtube.com/watch?v=7v8qtZ3I5AM
I don't disagree that some may be over-estimating how much cost can pragmatically be cut, but I also think you may be glossing over some of the costs and inefficiencies that come with operating a large heterogenous operation comprised of individuals and personal or rented vehicles.

A company deploying large mostly-homogenous autonomous fleets may be able to benefit from:

- we probably don't need to tip AVs
- economies of scale on obtaining vehicles, fuel, maintenance, and cleaning
- tuning maintenance/cleaning schedules across a fleet towards keeping vehicles in service for longer
- may be able to perform cleaning/maintenance during off-hours, vs owners who'll often have to trade
working hours to do these tasks
- autonomous systems may drive (and be tuned towards) in ways that also help preserve long-term vehicle
value and minimize costs
- lower insurance, legal, and PR costs if they outperform human drivers, don't molest/murder
passengers, etc.
- minimizing costs around acquiring and managing a human workforce
- there are probably many small ways to optimize the positioning and functioning of an AV fleet that
just won't work with a large contractor fleet

That said, the potential for a lot of these savings depends on current prices actually reflecting these components. It may very well be more expensive to perform some of these activities, no matter how efficiently, than to exploitively externalize their costs on drivers and riders.
Have you considered using 1 script? For example, you could have a script called foo with multiple subcommands that do various jobs, then the process of discovery is as simple as running foo help occasionally. Coupled with some good docs, the occasional tech talk and sharing command examples on IM will soon make it part of your culture.
Yeah traffic can still be bad in the UK, but ghetto wise the US have it worse because of their gun laws (and seen from the murder rate). Also student loans are paid for by the UK government, and you only have to pay it off after graduating, and even then it's basically a small percentage of your earnings. Also after 25 years they write off the debt anyway.
We give our staff anything between 25 days to 30 days paid holiday. In addition they receive 8 paid bank (aka public) holidays a year. So that's a total of between 33 and 38 paid days off.
That is a train or a tram, not a car.
Singapore Regulator says Grab and Uber merger harms competition – proposes fines
You can probably do this with simple tools. It's all about the layers; If you observe closely, you can see all the objects in the frame have very simple movements.

The 3D effect is just an optical illusion coming from all these simple movements in 2D space.

Some creativity goes a long way!
Was on there from 2009 ~ 2016. Being queer & trans, it was the first online social platform where I could be exposed to new narratives and interact with others.

Previously it had been forums and YouTube, but neither felt as personal. I ended up meeting a former partner and current one on Tumblr.

Tumblr however was hemorrhaging money. The way they commercialized it following Yahoo's purchase discouraged trans authors: content was marked as NSFW no matter how innocuous and no longer readily accessible to unregistered members, tags that were used to find trans content became inundated with prn spammers.

Nothing has really filled the gap left by Tumblr.
Because:

A, constituents are not interested.

B, the politicians that OK it would be long out of office before the results are in.

What you get when you base your politics on marketing and homo economicus...
Where does it say "strip"?

>-Support HDR & HDCP 2.2 Pass-through

Which is not stripping away anything.
Can a building host a party so toxic that the building itself is socially irreparable?

We have a different obligation to software than we do to physical things; digital hangouts are after all just a computer program. What makes a thing popular seems to be a product of the number of people present crossed by the social weight of the people present. We might say that a party becomes uncool once a certain class of people has joined, or is uncool once a certain class of people have left.

The article says that the people on Tumblr are often horrible to each other and that it ruins Tumblr; we can burn that digital building down but it will just be rebuilt, with the same swelling of cool people coming and going, and undesirables lingering, in a cycle that cannot be broken.

I think that social media must necessarily follow the forest fire model, with lots of expected, regular burnings-down to allow for regrowth.
That's one of the reasons wealth inequality is problematic.
Type safety, constexpr, template metaprogramming: I don't know why anybody does anything in C in embedded systems when C++11 is right there. Compile time programming is what it's all about.

Support has been slow, but even the non-GCC embedded toolchains have been improving substantially over the last few years.
Affective death spirals:

https://wiki.lesswrong.com/wiki/Affective_death_spiral

Evaporative cooling of group beliefs: (Where heat == intelligence)

https://www.lesswrong.com/posts/ZQG9cwKbct2LtmL3p/evaporativ...
They change the interiors of the car to accommodate. There is a reason the public transportation doesn't feel like my Ford Fusion. It is made for lots of different butts to be in and if a mess is caused they come in and spray it out. A bus for example can be cleaned with a pressure washer (sometimes inside and out.) Seats, headboards, etc. etc. can all be put on by snaps or simple screws. Anything stolen can easily be monitored and the last occupant can be charged for it. There are a lot of ways to get this done and make it extremely cheap and durable.
Bought a Vizio in 2017 (because 5 HDMI ports, 4K HDR, dolby vision), and I just couldn't find one non-smart TV that was 4K HDR.

So I resorted to use it with Apple TV only (its great that Apple TV also has Dolby Vision support, so DV content looks stunning on this display).

I thought I was smart in not connecting the TV to the internets. Once I did connect it for a few days, later at some point Vizio forced me to connect it because, they changed the physical remote, sent it to me for free, and then the UI was blocked until I setup the new remote, which needed internet. For a moment, I thought of returning the TV to Costco saying "it is not working". But I didn't have the patience to do that :-/

So technically, Vizio or any smart TV can prevent you from using the TV after you start using it, if you refuse to update the firmware, remote etc.

I'm willing to pay 2x the price I paid for Vizio, if Apple comes out with a display, because thats the only brand I trust for privacy. But only problem in that case, will be HDMI ports. I doubt an Apple Display would have more than 1 or 2 HDMI ports (looking at how they are port unfriendly on Macbooks :)
Using Rundeck is always an option: https://www.rundeck.com/
> This is wrong.

Async "callbacks" in Javascript cannot have semantically meaningful return values, because a callback context cannot read the values. EVERY lambda has a return value in Javascript, but this is more an implementation detail of dynamic typing than a statement of intent. They don't have callers that accept values of any type, so they're effectively a function of (Context -> IO ()). They're executed strictly for side effects.

If you have a meaningful return value, it's not properly called a callback. You can google about this because obviously the "spec" is "how it is used in literature and the common vernacular." As an example, the first SO hit on Google I say has top rated answer saying " An async callback function can return a value, in other words, but the code that calls the function won't pay attention to the return value." https://stackoverflow.com/questions/6847697/how-to-return-va...). You can surely find more, this is what distinguishes constructions like "callbacks" from "hooks" (which often return booleans, but may modify internal state passed to them or execute side effects as a callback) and further from "Promises" (which compute a value once and allow useful synchronization on the completion of that computation).

> Either that, or async/await is more powerful than callbacks, because most of the `async` functions I've seen have a return value (coroutines can and do yield results, and async/await is a way of implementing coroutines).

Well of course they are. I don't think I said they were equal in every respect, did I? I was a bit overly specific in talking about how Promises are often just used as callbacks, so sorry if I was unclear.

Async-await is a continuation monad. That's strictly a more powerful construction than a side-effect-only callback.

> I also don't really think that this solves the same issue as multiline lambdas in all cases.

I agree. It solves one very small slice: not having multi-line lambdas for filters and maps, nor a compiler that can fuse them.

> But multiline lambdas don't particularly clean up `if (x := re.match(...)): process(x)`. Which I'd argue is the more motivating example.

This PEP is even more exquisitely awful if that is truly the dominate use case. "We saved a line for the sake of saving a line" is a brutally frivolous reason to introduce that much new machinery.
Every Organization needs a meat-grinder- a mechanism that swallows contributing individuals after a certain time and removes them from the organization.

In a democracy, that usually is a far away parliament, that removes the elected individual out of touch with the worries of those elected, so he/ she is not reelected.

Fifa should give every fan a vote.
We might say that a party becomes uncool once a certain class of people has joined, or is uncool once a certain class of people have left.

https://www.lesswrong.com/posts/ZQG9cwKbct2LtmL3p/evaporativ...
It's a really limiting view to think your potus is your only leaders.

Also, you can only buy something if somebody has the will to sell it.
That's a stretch. I worked on GDPR compliance and it wasn't because of anything underhanded or nefarious. Just had to comply with new regulations.
We had a simple setup before (and have used spot instances everywhere it made sense). Without getting into the specifics of the use case, the ability to target a specific instance for specific backend processing tasks became necessary. Rest assured, it's not "too complex", only "as complex as necessary". Be careful not to respond with a rote "you're doing it wrong"-ism when discussing technical merits.
I fail to see how that's unscrupulous, as that's a new version of the app, and that's how software was sold for decades.
The part where they stick to assembly while the competition doesn't.
The article was a trip down the memory lane.

Cannot believe how quickly a decade has passed since the debut of the first-gen iPhone and subsequent launch of the App store. I still have faint memories of watching a recap of the launch event keynote by Mr. Jobs on a local news channel (though my preteen brain didn't grasp its significance), while being completely oblivious (not that I was unique) to the revolution it was about to unleash in the lives of MILLIONS of developers in every nook and cranny of the planet -- with Google following suit, of course.

Think of the innumerable industries that have been born, millions (billions?) of lives that have been changed, trillions of dollars of value generated, all as a result of this tectonic shift in software distribution.
>America ended all forms of slavery, including indentured servitude, under the 13th amendment after the civil war.

In reality, most slavery moved to sharecropping and expanding the criminal justice system so that prisons became the new source of cheap black labor. There’s a prison in Mississippi which was converted to a prison after being a plantation became unfashionable and business there has been unchanged since the antebellum.
I disagree. Look at the airplane, it's clearly made in 3D software.
I suspect that even though SF at one point may have been a gathering point for people who didnt want to do things the normal way it has been filled with people who want to make money off the previous group, which makes it less interesting for future trailblazers. i wonder where they will go in the future
In all of these deal announcements, the key words are “up to”. That’s very different from a confirmed 62k order, and in all announcements related to that agreement timing of delivery was unconfirmed [0].
[0] https://mobile.nytimes.com/2018/05/31/business/waymo-chrysle...
Check out waymo's videos; the car detects the flashing lights + reflective fabric on the police officer's uniform, as well as the shoulder and arm gestures. Griefing is as much an issue as is police impersonation right now, that is a very rare occurence that will be punished very severely when occurring.
Having spent a year in the Bay Area after living in Portland for 10, I really do feel there is much more creative energy in places like Portland, and also that it's a far better place to start most companies.

The problem is that the Bay Area might be full of more creative, interesting people, but the quality and cost of life down there is terrible (and nobody's fixing it), and the problems associated with that more than mitigates the benefits.

You meet smart people, but in order to pay their exorbitant rent they need to work long hours for companies that can pay them an enormous salary, and that's usually the big ones you know about already. That financial burdon, combined with chronic transportation issues, prevents the congealing of people and free time required to form the really creative new ideas and companies, and you just end up with whatever the VCs want to throw a pile of money onto.

So naturally, the thing I do find the area useful for is companies that could be considered VC/10x plays. But I feel like most companies, even very successful ones, don't fall under that criteria. And it only works for companies VCs are willing to fund, limiting your ability to get creative and outside the box.

My advice would be to go there for 1-2 years after college, then decide if all the problems with the area are worth it. Anecdotally, most people I've known eventually get sick of it and leave. And infact the problems are so bad now that the area has started losing population.

I think in order for the Bay Area (and California in general) to really become good for creatives, they need to take a deep existential look at themselves and really decide that they're willing to solve the problems. That means making radical, Marshall Plan levels of reform to zoning and property tax laws, establishment of strong regional metropolitan governments, and significant investments in better transportation. I have zero confidence in the idea that the region will continue to be successful unless they aggressively tackle these problems.
Several years ago I attended a distinguished lecture by a researcher with a long history of working with non-human primates. For an entire hour we watched clip after clip of chimps failing at trivially easy human mental tasks mostly involving 2nd order cognition, even after extensive training and guidance by researchers in each task, and videos of research data in which the researcher had anthropomorphized the observed anaimal behavior when the data doesn't actually support it He was trying to undermine the notion that humans are just a better chimpanzee (i.e. quantitative) but are a different order of chimpanzee (i.e. qualitatively different). I was convinced.
Metrics are required platforms (iOS, Android, Web, Windows, macOS, Linux), components (see matrix on https://pspdfkit.com/pdf-sdk/), number of users, number of server nodes (if Docker-based), number of apps/[sub]domains, internal/external access and expected support, to name a few. We're usually quite fast to figure something out that works for both parties, pricing is in the 4 (passion projects) to 6 (enterprise) figure range. Simply drop us a message via https://pspdfkit.com/sales to get things started. We try to make this process as easy as possible.
"America ended all forms of slavery, including indentured servitude, under the 13th amendment after the civil war."

See also "sharecropping" (https://en.wikipedia.org/wiki/Sharecropping) and tenant farming.

"The fact the United States moved so quickly after the ratification of the Constitution to ban slave importation in 1808 shows they had a strong desire to get the issue solved."

I seem to recall that the bill (https://en.wikipedia.org/wiki/Act_Prohibiting_Importation_of...) was supported by some in the slave states because it would increase the value of their existing slaves. On the other hand, the Wikipedia article seems to imply that an important aspect of the issue was that Britain was the largest slave trader of the time.

The idea that "they had a strong desire to get the issue solved" seems clearly false; they kicked that can down the road as long as they could.
Google's DeepMind Taught AI Teamwork by Playing Quake III Arena
> Can they really use that many only in good-weather places with perfect roads?

I would hazard a guess that yes, that's pretty much the Southwestern US year round, if you include Southern CA. There are enough metro areas, and enough new sprawling development, to put those vehicles to work.
Here is a link to the whole proposal:

https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CEL...

You will probably need to search for a few formal definitions, but in my opinion it's relatively easy to understand.
Think construction plans, PowerPoint presentations, contracts - there are many documents that don't fit the classical Word document model.
Tyranny of the majority. And a tiny tiny majority at that, perhaps just statistical variation.
> after witnessing disasters such as Perl 6

OOC, how long ago did you witness this "disaster"? Perl 6 is doing very well, thank you.
We need new business models for new media, which are not based on artificial scarcity. Then new media can afford to outspend old media. It's unfortunate that the current flag-bearers for new media are funded by advertising, rather than creativity and innovation unlocked by fair use and remix culture.
Somebody still uses Tumblr?
This is from 2016. Interesting that the authors most recent post is a tool for finding when your projects are being mentioned on HN or Reddit :)
We did that in https://pspdfkit.com/blog/2017/webassembly-a-new-hope/ - WebAssembly is really fast, the different is less than you'd expect.
Pay zero on it now. You'll be paying on it when you withdraw
An internal wiki for engineers and that has a list of concerns with individual articles for solving known issues/tasks. Plus a culture of encouraging each other to write or update wiki pages when they have an issue.

If you have the time and motivation to write an internal tool, you should also have the time and motivation to write down the how and why.

If you want to make a tool a part of your process, document that process and where to use the tool.
That small region of the US has been, for the last few decades, the global broker of capital and talent working in tech. Silicon Valley is a hub of international activity. The news here is that another place, China, has found ways of brokering the relationship between talent and capital without SV.

Sidenote: The Alipay deal is really more private equity than VC.