Project isolation, social islanding? - Page 1
Forum section: electronics / general

author: paulca
author_rank: Super Contributor
timestamp: August 28, 2024, 08:14:55 pm
content: So, for example sake, you are an experienced, qualified electronics person and you have a project you have been working on for decades. Not a project you haven't finished, but a project that you just keep refining and evolving.Let's say for the strawman it's making hardware devices for automation. Digital actuators of many forms. It's only a strawman for example structure.After decades of development, you find that there are competing "packaged" off the shelf alternatives to many bits of your system, if a little imprecise and inflexible.You occasionally run into a deep problem that takes a lot of thinking and you write out an elaborate article style topic on the subject and publish online... you get back."Why not use, <insert rubbish software here>""Why not use <insert AliExpress module here>""Why do you make it so complicated? Just put a relay on the + of the motor.""Why do you use transistor to separate circuits at all? Just buy this module on Amazon and completely redesign your stuff for less functionality and more cost""I do it with 3 lines of firmware and it's fine for me.""I think you miss-understand how heating control works. If you look at GoogleNest....."^^ This is exactly where I am in home automation software/firmware as a software engineer. Nobody understand me anymore, the 99% majority use HomeAssistant and if something can't be done the HomeAssitant way it can't be done... you are clearly wrong and over complicating your requirements if HomeAssistant doesn't already do it by default. You must be an idiot. Just assimilate the AliExpress modules and HomeAssistant least thy blasphem!Does this analogy ring true for any of you guys since the birth of Arduino and the likes?I have found that trying to explain the only reason they see things as "simple" is because they went the "happy path" and never considered ANY of the reality corner cases. They even confess to stuff consistently breaking. But, yet, I over complicate things.That or they wrap a complicated concept I was trying to explain the how and whys of, but "Dude, you over complicate the WidgetX from <insert smart home garage cloud thingie>", without even the slightest realisation that... someone... somewhere already designed that one.... for you.Anyone? What is the solution? Just shut up and carry on? Try and find a better audience? Abandon my superior bespoke system for a bloated (equally) opinionated "one size fits all" limiting eco-system?

author: SteveThackery
author_rank: Super Contributor
timestamp: August 28, 2024, 09:02:05 pm
content: What an interesting topic!I bet there are loads of stories like yours. There's a lot to be said for standards, even de facto standards like Home Assistant, and for many people the shortcomings are outweighed by the widespread and multi-sourced nature of the standard product. As for functionality, quite often "good enough" really is good enough.There is an element of risk in going for a bespoke solution, or the "Rolls Royce" solution like yours, because it is single-sourced and therefore might become unsupported for a number of reasons.I'm looking forward to hearing other points of view - it'll be an interesting thread.

author: ebastler
author_rank: Super Contributor
timestamp: August 28, 2024, 09:06:20 pm
content: Those casual readers of your post are clearly giving the wrong advice; they have not even understood the specific technical challenge you are facing and the full scope of what you want to achieve. But your response clearly can't be "Hey you guys, shut up and listen closely while I explain the intricacies of my design to you!". They simply won't be interested -- it's not their problem; their problems are simpler; and there are much simpler solutions to their problems.So yes, you could say that you need to "find a better audience". Which audience may or may not exist, depending on how deep the rabbithole is which you have burrowed into.If you should indeed find yourself in a situation where you can't findanykindred spirit interested in your system, maybe that is a reason to declare that project "finished" and turn to something new: While there is a lot of satisfaction in designing something very elegant and "perfect" and seeing it work, there also is the rewarding experience of joint problem solving, showing and explaining a good solution to others, learning from others etc. When that aspect goes missing because your project becomes too specialised/complex/esoteric, you are missing a significant part.On the other hand, if you enjoy the challenge of perfecting your own project, and don't mind the fact that it can be more time-consuming than adapting something off-the-shelf, and are ready to work though the occasional frustrating phases of "lonely" problem-solving -- by all means go for it! It's a hobby, so the value is not measured in productivity.To stick with your example: "I'll cobble something together in HomeAssistant because that will get me a working (though ugly) solution quicker" is not a good reason to abandon your own project in my opinion. "I'll consider my current home automation good enough and focus on a new hobby project, where the exchange with others is more rewarding"maybe a good reason, depending on whether that aspect of the hobby is important to you.

author: coppercone2
author_rank: Super Contributor
timestamp: August 28, 2024, 09:08:54 pm
content: this is why big companies spend an absurd amount of money on marketing R&D to be able to give accurate and truthful answers when comparing products.The more basic it is, and the more generic your audience is, the more effort it takes to convince them... but its basically like a department in a company responsible for this.. and even they get effected by trends, there could be a down turn because of a hot product (that takes a while to get a generally bad reputation). They develop a marketing method, evidence, and reputation and then actively use those things to maintain their user base and have to constantly gather data to maintain relevance and legitimacy.Its like bouer vs milwakee tools. milwakee has a hell of a time convincing the public why their tools are a better investment, but with tools its easier. Home goods... that's mass market stuff.. its a whole different levelIf a major retailer adopts a new low end product from a existing market segment (i.e. something they sell in home depot) your in for some chopMore often, there is disruptive products that initially offer high quality, and then decrease in quality (i.e. amazonbasics batteries). In short, for a period of time someone is basically highly undercutting you (a big guy selling high end stuff for cheap) to build a customer base, then eventually they decrease quality to gain profit. Eventually word gets out, but its disruptive in the industry. in the case of amazonbasics, IIRC they were reselling very high quality cells made by panasonic, and did a swaparoo eventually.... but reputation takes a while to fall...And on the other side of things (rare), sometimes someone does make a better product that is cheaper and then something goes extinct.The big companies are almost.. like epidemiologists... when they fight some viral trend. They are very vigilant about sales data and respond quick.

author: paulca
author_rank: Super Contributor
timestamp: August 28, 2024, 09:44:49 pm
content: Quote from: ebastler on August 28, 2024, 09:06:20 pmAgreed. Found this several times to the determent of my "community karma" scores in various online communities.As my social and "corporate game play mentor/peer" in work says, "There is a difference between being right and communicating it successfully. The later is always harder. The more technical and detailed the harder it will be to justify it to a wider audience".

author: nctnico
author_rank: Super Contributor
timestamp: August 28, 2024, 09:56:32 pm
content: Quote from: paulca on August 28, 2024, 08:14:55 pmMy observation is that lots of people are OK with products not working 100% and/or failing prematurely nowadays. To stay on topic of home automation: take Wifi connected doorbell-cameras for example. People with bad intentions will jam the Wifi before entering so the camera doesn't work when you need it the most.

author: tszaboo
author_rank: Super Contributor
timestamp: August 28, 2024, 10:01:43 pm
content: Quote from: paulca on August 28, 2024, 08:14:55 pmYes. With the Arduinos and easy to cobble together modules, electronics or home assistant reach wider audiences.Which means that more people with very shallow knowledge on the topic will talk about it. One of the reasons it's important to define what sort of solution you are looking for. And ignore some of the answers.

author: paulca
author_rank: Super Contributor
timestamp: August 28, 2024, 10:16:00 pm
content: Quote from: tszaboo on August 28, 2024, 10:01:43 pmYes. It's more the devastating social consequences of doing so in most communities.There are people in this thread whom I have spend weeks in debate with on similar topics in firmware and I came away the winner as they eventually got through to me there WAS a VALID and simpler alternative. I accepted this, embraced it, thanked him for it and moved on with it.I wish for the same in home automation. I want to be told my solution is incomplete or invalid, but I need to be told why.For this to occur I need to communicate the problems I am trying to solve. This is where the problem comes on "wider audiences", especially, or more so on the likes of reddit, than to here.At the same time, "here" is electronics and while there is tolerance for topics like software the audience is limited.

author: paulca
author_rank: Super Contributor
timestamp: August 28, 2024, 10:22:49 pm
content: I felt it necessary to highlight.... that 99% of my system "integrates" via an open message bus. 100% (or close as I can get to it) of state is on the bus.This means that ALL code variables come from the MQTT bus. That means, the entire internals are exposed for any and all third party or second party integrations.In other words, it can work with or without home assistant. Home assistant can work with it.It's actually so well integrated with the likes of home assistant if I install home assistant locally and get it to scan for things it comes back with 100s of things. It has found my service bus topics. It has found the state of my application. It like any other message client is 100% free without restrictions to publish to those same topics. If the messages make sense and are valid they will be actioned.It's not like I am proposing people stop using Home Assistant and use "My emperor's new clothing range". It's not like that at all.

author: exmadscientist
author_rank: Frequent Contributor
timestamp: August 28, 2024, 10:27:03 pm
content: Unfortunately I think this is averygeneral problem these days, and it is only getting worse. Most people who want to talk about a topic do not really understand it. People with deep understanding, who can listen toyouand whatyourgoals are, and then are able to explain things in any kind of useful way... those people are worth their weight in gold. Interestingly, I think the rarest part, or just least obvious skill, is just finding people who can listen well and understand you rather than thinking you said whatever they thought. Listening really is the most powerful "soft" skill!(And yes, I'd say there are a few such people around here.)

author: paulca
author_rank: Super Contributor
timestamp: August 28, 2024, 10:36:31 pm
content: Quote from: exmadscientist on August 28, 2024, 10:27:03 pmWe are all guilty at times. I know I do it. Often however I am presenting my straw man in the hope that someone shows me how easily burnt it is.When people come at your "iron man", something you have re-inforced for thousands of hours with matches and sticks and the general wider concensous supports them... it hurts. No, it leads to your questioning you communications skills, or worse, your entire way of thinking, working and solving problems.At least this thread is on the supportive side. Thanks. Feel free to throw a few I can take them, my ideas are often radically and often radically wrong. I like to endure conversations and debates, in detail as to the "But why".I think these types of debates and all manor of opinions and levels of pendanty are handled on this forum partucularly well. Disagreements happen. Things get heated, things go tagent, but the discussion is open. Ideas are only put down if they are stupid. People that propose them are not. This affords a "safe space" where ideas can be Staw-manned and set ablaze in public without calling anyone an idiot or like wise for proposing the idea.It is better to generate 100 idea a day and find a gem once a month than to hinder yourself for generating ideas at all for fear of failure.(too much beer. I'm out for how)

author: paulca
author_rank: Super Contributor
timestamp: August 28, 2024, 10:48:01 pm
content: Quote from: exmadscientist on August 28, 2024, 10:27:03 pmI would also like to point out. As a credit to this forum and some of it's members, there are those who go out of their way to help and not judge without experience.There are also those who are strong enough, knowledgable enough to "forcefully" and "authoriativeily" say, "No, don't do that, that's stupid. Do this instead."Before ChatGPT revealed it's true colours and was "hobbled" there was a thread on here where someone asked ChatGPT about the forum members and it was able to speak very positively about some of the most frequent here.It's a shame it got "nerfed" (ecause it could dox or dox-ography nearly anyone). It was a very interesting thread asking Chat GPT for it's analysis on forum members and their contribution, helpfulness and like-ability.

author: Geoff-AU
author_rank: Frequent Contributor
timestamp: August 28, 2024, 11:28:26 pm
content: As someone who has inherited projects worked on for decades, sometimes there is very much a better way to do things if you started doing it now. But there are also baked-in lessons that would all have to be re-learned if you ripped it up and started again.But also as someone who tinkers with hobby things until they're just the way I like, the value is primarily realised in my own satisfaction of it working the wayIwant it to work. There's little additional satisfaction derived from telling people about it, because it's a solution that's unique to me.I have a "dumb house" philosophy - I don't want to be updating firmware for light switches or other "Internet of Shit" devices. If I can't turn things on and off without an internet connection I will scream. A light switch lasts for about 30 years, you replace it, and then it lasts another 30 years. The value-add of a phone app to me is zero. But I will do smart "non-essential" things like monitoring energy flows and temperatures. I have 3 temperature sensors on my hot water tank, because I can. Top, middle, and bottom. The middle sensor is the most interesting one because it indicates how much "reserve" hot water we have and shows the mixing behaviour of the tank during a heating cycle.My friends have HA, and they love it, but they also know that I am a weirdo and like to do my own thing so they don't try and persuade me to change.

author: Siwastaja
author_rank: Super Contributor
timestamp: August 31, 2024, 10:51:37 am
content: Interesting discussion, I have encountered this behavior many times and at first it was confusing, but then I realized these who comment have rarely actually done anything. They are asking those questions because they don't know! The problem is that you are reading between the lines and assuming their questions are loaded - and maybe they really are, but try parsing their questionliterallyand the whole game turns around:"Why not just use Aliexpress module and Arduino to do this and that?"You read it as: "This person thinks the task can be achieved by this Aliexpress module and is pushing that view, what an idiot"Try reading it as an honest question: they are asking why not use an Aliexpress modulebecause they do not know.So try answering their question:"Building this system from Aliexpress modules requires countless of hours of integration work and programming, and then maintaining.""Availability of these modules is iffy. Even if you designed around something popular like Raspberry Pi and need to order 10000pcs, you may have to wait for a year.""Specifications of Aliexpress modules can change at any time without a warning."etc.In context of what I have been doing in past and am currently doing, this is always the same case with stuff like ROS (Robot Operating System), or HomeAssistant - they are tools, not ready-to-ship products, yet some people seem tohopethey are the latter kind.They don't know. They can be told about it, many believe, some don't, some people enter some kind of locked-up state and fail to communicate further.We manufacture and sell a kind of home automation (energy management) box. It's absolutely and totally integrated solution, one full custom designed box with one full custom closed firmware, and one full custom server infrastructure. Installers wire it up and customers use it. No complex configuration, no programming work, no wiring up different modules. This is the only way to make something happen in a large scale. This is something >10% of homeownerscouldbuy.There have been some electricians who play around with HomeAssistant and Shelly relay modules etc. but this is like 0.01% stuff, total niche. It's not scalable unless there is another company who abstracts away that work, but even those have been so far unsuccessful: even though the ecosystem has been made simple, the fact you need to buy 5-10 different types of modules, wire them up, configure, buy an optimization cloud service from 3rd party is a total showstopper for 99.99% of sales companies and 99% of electricians. And only 0.01% of homeowners are A) electronics hobbyists B) have time and energy to put into their home automation.Designing a product and/or service which can be sold to masses is a lot of work, and many of the design goals are completely different from a hobby project, or an engineering/science-driven R&D project. Decision makers (customers, installers, sales companies) hate complexity, they want to deal with a single company which can provide everything and have a phone number where to call for support.

author: RJSV
author_rank: Super Contributor
timestamp: August 31, 2024, 04:06:53 pm
content: First thing to come to mind; My (private) rooms lacked capacity to sustain the hardware aspects of a large project. That was after a bit of a 'crash' in resources.Somewhat in isolation, particularly due to IP concerns (intellectual property, on innovation). Not so much on the technology, building mechanical elements for a completely mechanical computer (firstly for teaching and STEM type student introductions).Rather, than IP on the devices, my concern was keeping that business model a bit quiet. The IP stuff actually had an emphasis on covering many 'bases' as they say...to prevent some other larger entity from getting their own Patent(s) on associated devices / methods.In other words, I threw in 'everything but kitchen sink' in my own Patent (applied for) application. Not for eventually IP protection, but simply to clear a setting where I could operate and grow, without some other, big, entity interference.So, I've ended up here, 2024, with issues more along the lines of isolation (an 'isolated project's, like OP stated). Hard to justify years and years of R&D progress, in a vacume with lack of real market involvement.I figure, can't really SELL the hardware with any significant quantity market; maybe best to start an education oriented 'Training' system, as the business profit, while the mechanical computer itself being a tool, along with publishing various manuals and training aidsNot your usual 'IC or transistor' manufacture, for sure.I'm reading your thoughts on how you (OP) present, or attempt to present ideas to others, as you likely need the support. So, it seems a dilemma between needing isolation, meanwhile also needing support.Plus, in my case, the teaching system project is heavily oriented to things old, now; The 8-bit processor types, of 1970's, and engineering culture, of 1980's...that's now 40 years + !Can you (OP) find someone, a partner or investor, that fills in the skills you maybe lack...(A 'spokesperson' or marketing type that can spend the time, to hear your concerns and, uh, TRANSLATE, to a more compact expression.I've encountered others, friends and family, who do the kinds of work that I had anticipated doing, (as a STEM or technical beginners training organization) maybe sometimes just in the similar spirit, which has lots of 'inspiration' or engagement....a difficult item for teachers, often.

author: paulca
author_rank: Super Contributor
timestamp: August 31, 2024, 04:52:12 pm
content: On brevity and compaction. I remember Peter North (I think) on here in the MCU sub having the same issue. If you omit all the details that you are aware of people will either ask about them or assume them. So you need to clarify. If you DO give them all the details it's tldr, wall of text, nobody cares.It's a bit like that fine art of writing a good StackoverFlow question. It's not even funny that there are Stackoverflow questions on how to ask stackoverflow questions.

author: Just_another_Dave
author_rank: Regular Contributor
timestamp: August 31, 2024, 05:01:50 pm
content: I think one of the reasons why you get so many replies recommending things like Arduino or home assistant is that those technologies tend to limit amateurs from gaining more knowledge about electronics. For example, Arduino is quite easy to use for newbies, but it limits significantly the features accessible to the user (many boards don't expose interruptions in Arduino ide). As a result, a broader separation between amateurs and professionals exists nowadays and in order to grow in this field, you need to change the tools and methods you use to others that are less amateur friendly

author: RJSV
author_rank: Super Contributor
timestamp: August 31, 2024, 05:34:35 pm
content: I have to admit, having some difficulty grasping the OP's question(s), involving 'social isolation'. Maybe a bit more explanation what that meant (in post title).Reading posts, and replies, it appears like one single individual (OP) developing product(s), up against some large organized development and marketing entities. I was originally interested, as, in my case, it's just the one engineer, contemplating interfacing with some large group (eventually). Could you please address some of that, 'isolation'....mentioned due to need to keep your product secret, initially ?I have a lot of 'pet peeve' with portions of home automation, that being the video media remote controls, and the layers of complexity being introduced (...pushed), by market. Engineering is one, mostly logical thing. But once the products get into marketer's hands, the agenda swiftly shifts (in my view), virtually IGNORING customer desires, and 'responding' in nearly 100% manipulative ways...In other words, customer complaints usually imply that they get second priority, and the marketers confirm that, by the response (to complaint issues).I cannot necessarily condem the marketers view, as they eventually 'pay the bills'.

author: abeyer
author_rank: Frequent Contributor
timestamp: September 02, 2024, 12:35:26 am
content: Quote from: paulca on August 31, 2024, 04:52:12 pmI feel like some of this is the limitations of the forum format in general. Something like a wiki, or github repo, oranythingthat prescribes a more structured format while still having some form of feedback/discussion can make this more approachable, in my mind. Stackexchange is a step towards that... particularly if you use the community wiki pages and similar, but is maybe too much focused on the q&a format vs something more project focused.

author: vk6zgo
author_rank: Super Contributor
timestamp: September 02, 2024, 01:58:26 am
content: Quote from: paulca on August 28, 2024, 08:14:55 pmMany years ago, the commandment came down from Mt Olympus that we had to convert all the attended TV & Radio Broadcast Transmitter sites to Unattended operation, with all the conversions done "in-house" by the existing staff.We did receive a "suggested" plan, which looked like it had been "slapped together" by someone at the office between Morning Tea & Lunch, which was totally inadequate for the job.At my workplace, initially, in order to meet the ridiculously short timeline, we had a horrible "bodge" in place, which sort of did the job, then we got a few "bits n' pieces" supplied for the final iteration.The core of this was a "Programmable Logic Controller", which received the inputs from various sensors in the form of relay closures, did the "smart stuff" then operated the correct interface control via a relay closure.One bloke who had played with PLCs before got the job of programming the thing, using the little keyboard that came with it, whilst the rest of us tried to sort out the multitude of incompatible interfaces of the transmitters, programme switchers, alarms, etc.One of the first obstacles was that the multi-cored cables supplied with the "conversion kit" were not rated for the control voltages of the (then) 28 year old Vision & Sound transmitters which were 65v DC & AC respectively.The control functions were already available at the control desk, so the easiest way was to use a relay panel to solve the voltage rating problems, plus provide additional relay contacts that we found we needed.As we went through things, we constantly encountered functions that the "auto control" needed that hadn't been anticipated, so when the PLC programming bloke would come out with the latest version of the program & run through it, inevitably, someone would say "what if?"He would exclaim "Arrr, Shit!!!" & go away "Tae think again", like unto "Proud Edward" in the song.For my sins, I was the "Project Leader" & the whole thing was horrifically stressful.I developed a "facial tic", which I always had thought was something fanciful thought up for Inspector Clouseau's superior in "the Pink Panther".

author: John B
author_rank: Frequent Contributor
timestamp: September 02, 2024, 03:06:30 am
content: Not really sure about the specifics of what your setup is, but I gather you don't utilise any of the large automation engines, whether home focused like Home Assistant or OpenHAB, or something generalised like Node Red?I've really enjoyed getting into OpenHAB despite it not being the top dog in (open sourced DIY) home automation.Home automation has attracted people who enjoy tech in their houses, without understanding much about underlying technology or it's pitfalls.Even Home Assistant is very small in comparison to controlled proprietary setups from Google or Amazon.Home Assistant has broadened it's appeal to those wanting to getting a bit more into the DIY aspects, while still having the luxury of equipment that can be connected by installing an addon. While there are many very competent people using HA to an impressive extent, having a large community also opens the floor to more noise.

author: JPortici
author_rank: Super Contributor
timestamp: September 02, 2024, 05:59:46 am
content: Quote from: paulca on August 28, 2024, 08:14:55 pmYes. Trying to make someone understand why arduino or platformio is crap, or why they have failed as educational tools because they have never tried to create a product with them is pointless... because they are not making products, but contraptions.(yes yes yes they helped the masses approach the field but god forbid they try and create something, it's always follow this tutorial and use that library, never create something on your own because they took the shortcut and have no real knowledge of what they are doing)

author: Smokey
author_rank: Super Contributor
timestamp: September 02, 2024, 06:36:47 am
content: Quote from: JPortici on September 02, 2024, 05:59:46 amwhat he said.

author: Jeroen3
author_rank: Super Contributor
timestamp: September 02, 2024, 07:31:09 am
content: Quote from: paulca on August 28, 2024, 08:14:55 pmYes. Just smile and wave when they go to reset and pair some cheap module for the dozenth time into Tuya so that Xi can control their lights.It gives them a feeling of accomplishment because "they did a thing!". Your thing a bit more initial effort, but probably has a much higher WAF and reliability.

author: coppercone2
author_rank: Super Contributor
timestamp: September 02, 2024, 08:08:51 am
content: I had nothing but bad luck trying to make complex systems with arduino in a corporate level because of maybe EMI causing issues. It is supposed to go together using daughter cards and stuff but eventually it starts to crash and when that happens all the time saved on design is wasted... even with the high end modules meant for that sort of thing.It does not scale with 'throw more money at it to solve issues'.

author: watchmaker
author_rank: Unknown
timestamp: September 02, 2024, 12:26:59 pm
content: I have a starter kit sitting on my shelf. My EE/IP attorney son in law told me they were crap for learning and product development. So there it sat for over a year. I may find use for the modules as shortcuts at some point.The issue is they are marketed for learning and users think they understand because they have success. Kind of self reinforcing.But what have they understood other than programming? Thevenin?Arduino and such reveal a pent up demand for true educational kits. When I was a kid, there was a subscription based program (look in 1960s Pop Elect) that provided tube based projects with 120vAC on cardboard bases. It is no surprise these disappeared, (more so that I survived). But they were popular and fed demand for “at home” learning. I think that Arduino takes advantage of that demand.I just started the Elenco AM/FM radio kit. Unlike the Heatkit/Eico kits of the past, this kit does more than provide solder practice. It has sections devoted to theroy of operation and to using a DMM/VOM, sig gen and scope to analyze the various circuits. For example, max power, BW, distortion, etc are analyzed for the audio amp; there are a LM 385 ckt and a discrete AB amp module to compare. It is not complete but it provides a platform for further experimentation and learning.I will write a review of this kit in the sandbox (Beginners) when I am done. (FWIW, I am just finishing capacitors and opamps in my program based on Boylestad and Real Analog). Doing the kit at this stage helps me focus on why the very basics are important to understand.Kits like this motivate the learner to learn about the circuits and device datasheets in more detail using the internet. The kit has given relevance to the subject. It provides a structure for combining theory with analysis.I think there is an opportunity here for one of the more entrepreneurial members of this board.The problem with people thinking they “know” is not unique. It is very common in watchmaking. Electronics seems to be another example of where you cannot know until you actually learn it in your hands.

author: Siwastaja
author_rank: Super Contributor
timestamp: September 02, 2024, 06:15:03 pm
content: Quote from: JPortici on September 02, 2024, 05:59:46 amThere was a time, maybe around decade ago, when some people really implied or point-blank claimed that Arduino is now the pinnacle of professional electronic product engineering. I was actually questioned why I don't want to use Arduino in my designs (many hobby back then, but many also serious) like I owe an explanation. That was weird as fuck.I might have explained it by showing how slow and bloated digitalWrite() is, or how crappy the pin header layout is (impossible to use either soldered or solderless breadboards, quite crucial for actually building hobby stuff or prototypes), or simply that because I had been using very same AVR chips for well over a decade already with no problem why should I downgrade?These arguments had effect of me becoming some kind of "Arduino hater" when in fact I was just responding to so called stupid questions. I gave up. Since then I simply referenced a marketing claim from Arduino's own website:Arduino is meant for artists. Probably that one is gone now, but I think it's a good point. Listen to how they themselves describe their product. Use if you find it useful, and where it suits. Arduino ecosystem is especially suitable when you find the combination of modules or "shields" that is suitable to you. If you are developing something for which no "shields" exist, then the value is pretty limited and the whole Arduino thing only becomes a nuisance. The whole point of it is that it's easy for someone with very little experience in either electronics or programming to use these building blocks and software libraries. But if you don't need it, then you don't.

author: paulca
author_rank: Super Contributor
timestamp: September 05, 2024, 06:58:12 pm
content: Apparently it's not just "hobby projects".Apparently I don't communicate well in my professional domain either.https://www.reddit.com/r/SoftwareEngineering/comments/1f9iuu0/modern_architecture_and_management/It must be me.In work my manager told me that the functional design specification document was not intended for development to use as a blueprint. When I protested that, no, that is exactly what a functional design specification document is for. He told me in paraphrased words.... "Stop bringing your prior experience and knowledge to the team. Accept what the team and doing, embrace it and get in line."Politely, as a qualified software engineer, you provide service to me.KNEEL mortal!and "No". My profession is worth more than my job. This is a government project. I can be subject to negligence charges. The company insurance policy does not cover that, nor does it stop my going to prison.What I did do, which thankfully restored my faith somewhat in humanity is run things by the department head. He confirmed that I am not insane, there are serious issues. So... my balls are well and truely out on the vice.The way I look at is this.Working software and money. It is only when both of those are satisfied and delivered can be talk about bonuses like making friends.If I die on this hill. If I die defending the engineering position, it will be service to my fellow engineers, junior and senior. It will be an honourable death.I'll probably get a pay bump moving to a new company anyway.On a more personal note. I won't lie guys. I am struggling with this. I think deleting my reddit account would be a start. On the career and profression I need to move jobs to somewhere that values engineering values and doesn't suppress them with business objectives. I know they exist. The trouble is... market is dead right now. Nobody is investing, people are being laid off.

author: paulca
author_rank: Super Contributor
timestamp: September 05, 2024, 08:08:24 pm
content: Is it the tendency towards "straw-manning" which is catching me out on communications? Am I exposing too much "tinder" for haters by doing so? Too much analogy leaving open the interpretation such that those who don't want to help, but want to further their own ego can troll you on details and pedantry?My intent is to use analogy to simplify. To create a model that anyone can construct in their head. Such that discussions can proceed on the topic. Actually... this level of thought abstraction challenges many it seems.If I am asking questions, should I refrain from presenting straw-men in fear they be burnt for unexpected and usually irrelevant reasons? In the previous reddit example, I got berated because of something which came down to "granularity", not dividing workstreams outs. I also have no idea what they are talking about 1 service = 1 team. Those are either very large services, very small, non-productive teams or I am completely missing something.In my defence? A straw man, in my opinion is meant to be burnt. People who send business emails with pink, or orange email signatures might disagree. It's a minimal effort attempt at defining or documenting a much more complicated issue.On reddit, if you raise a straw man about a tool or technique, it seems as thou the majority are more interested in burning it and giving comments like, "It was flamable, made of straw, I set fire to it HAHAHAHALOL." or "You shouldn't dress your straw men in plastic shell suits, a single match can burn it in seconds."Maybe I am just being completely and utterly stupid in expecting any kind of sensible answer, discourse and response on places like reddit.I HIGHLY expect that almost ALL successful software engineers avoid the place because they are too busy in the real world and realise that place like Reddit are not the right forum.With virtually ever software engineering project (electronics projects?) being strictly under NDA. Is it any wonder the actual professionals are quiet and the responses I get are not measured or professional?Just like my day job, everyone seems to busy accepting the crap to have time to discuss making it better.It seems to fit the mould I should just join them. Isolate myself and talk only with my project peers. My trouble there is, my project peers are also full of it.

author: ebastler
author_rank: Super Contributor
timestamp: September 05, 2024, 08:44:04 pm
content: Quote from: paulca on September 05, 2024, 08:08:24 pmI have never been active on Reddit or followed any channel there. But whenever I end up on some Reddit page as the result of a Google search, I get exactly the impression you describe. Most users there seem clueless, careless, just there for the trolling, or any combination of the aforementioned. There may be parts of Reddit which are different, but if they exist, I have never found them.StackExchange seems like a much better place for software-related technical questions; maybe others here have further recommendations?

author: nctnico
author_rank: Super Contributor
timestamp: September 05, 2024, 10:20:10 pm
content: Quote from: paulca on September 05, 2024, 06:58:12 pmTo me that would read as "go find another job after dinner". This must be some kind of UK specific thing though as posts with this kind of weird workplace shit have a common demoninator that they are (mostly) posted by people from the UK. Maybe look abroad for a job and move there. No, I'm not trying to be pedantic here; just friendly advice.

author: John B
author_rank: Frequent Contributor
timestamp: September 05, 2024, 11:10:55 pm
content: No specifics on the original post in terms of the project? I would interested to hear about it.I've been dabbling a bit in Home Assistant, also getting a feel for the overall project direction, and the community.I've put a lot of effort into OpenHAB, I think I'm going to continue with that for the time being. But the time investment is not the only reason choosing it over HA.I have some impressions/observations about the HA project and community, but I am curious to hear why/why not HA is/is not suitable for your needs.

author: paulca
author_rank: Super Contributor
timestamp: September 06, 2024, 04:54:57 pm
content: Quote from: John B on September 05, 2024, 11:10:55 pmThe irony it's rather "simple". It's just a "layer down" from HA et. al. It's all code based.To be rude to it, it's a bunch of python scripts hanging off an MQTT bus. More of the work behind it has been in avoiding code by using clearly defined processes, message structure, topic naming. 99% of boiler code has been rolled up in to the library and a "stub" include which even gives you your global access stuff like "mqtt client" and "clock". A minimum deployable service has more lines in it's deployment descriptor (5) than it does code, 2.Code:from home_auto_lib.mqtt import clientclient.blocking_run()The overall architecture is two MQTT buses. I call them "Prod" and "Internal". "Prod" is where any random device connects to the network. "Internal" is the service bus. On this bus all message formats and topics are normalised into a "data fabric". The concept of a "proxy" servers as integration between these two buses. To be accepted onto the internal service bus the translations usually happen in the proxy components. This is probably the hottest area for work when new devices arrive in the flock. It needs a good dose of engineering, it's a mess to be honest.The first thing that jumps out at me when I go to HA is that it's a single process, monolithic python application. It is also a state machine, in memory store. Any and all "in flight" transactions are in memory. If you switch it off and on again it drops everything in flight. It attempts to handle "everyone's" problems. It manages to solve 90% of 90% of peoples problems. If you buy into the eco-system and want something in those other 10%s... Good luck. My own "feeling" is that the added complexities introduced with the intent of solving someone else's problem actually hinder development on the more bespoke and lower levels.It's not "wrong", it's just that by the time HA and others came along I had already seen the problems I will face in the future and decided to side step them before I wrote any much more code. Instead of managing the growing complexity in a big monolithic "state machine", I split everything up across the network such that no code "could" "borrow" or modify state from another. Making all processes expose their guts onto the message bus and making the message bu the source of truth, REALLY simplifies the half of it.It is entirely routine for me to restart services rapidly while making dev changes. The system doesn't care. Very often if a bug does persist and a service fails, all the others just carry on. The specific functionality provided by that service stops though. Usually I notice as a gap on a graph in Grafana. If it's late, I just "restart" the whole stack. Figure it out later. This relies on the project rules of engagement that all services should be stateless and emphemeral. Shooting them in the head and a new one will spawn in it's place and take over unbothered.... unless it continually crashes and restarts.I had a bug with a heating service where it took it's inputs, make it's choice and published it's requests. It then hit a typo in a logging statement and crashed the service completely to console. It got restarted. The heating continued to function absolutely 100% without lose of functionality for 2 months last winter until I fixed it. The poor services was crashing about 2 dozen times a day.It's not perfect, but is it is full control. The libraries were not developed as an entity in its own right. It is code that was already peppering services, so they migrated into a "common" and the services could be split up into several packages, there are a few dozen scripts and another dozen or so support files. This is my complete bespoke setup. The library and core components are about 6 files.There is no persistence internal to the service bus. The "expectation of volatility" rules of engagement just ... side step this entirely. Don't persist. Rebuild. There are caveats of course. For persistence, you probably guessed it. I use services. The only service that exists that qualifies as a persistance "write" service is the "Influx Mqtt Bridge" which takes bus data and publishes it to InfluxDB. The only persistence "read" service is the "Manager" component which is responsible for generating and publishing the master clock pulse and the static config data.In fairness, if you want most of the system to simply "stop". Stopping that clock service will suspend most of it.EDIT:A funny thing we do in software design, sometimes, is to make components of the system "actors" and describe their role as if it was a person describing their role.I as a "Paul's Home Auto" service will subscribe to data changes on the following topics of interest.....If I need config I shall subscribe to the topics that contain it and response to updates.I will patiently wait for that data and not act without it.If I have incomplete or invalid data I should always default to "safe".If I cannot do the "right" thing, I shall do the "safe" thing.If I wish to know what time it is, I shall ask the master clock, I shall NEVER ask the OS.When I have my data I shall make my determination and notify other services by publishing my desires on the bus.If I fail... I shall die to be respawned a new.Where possible, I should "Keep calm, log it, and carry on."

author: abeyer
author_rank: Frequent Contributor
timestamp: September 06, 2024, 09:16:45 pm
content: Quote from: paulca on September 05, 2024, 06:58:12 pmAnd neither of them tend to happen alone in large projects, but by working as part of a team. And they both become harder to achieve at all if your team ignores/circumvents/sabotages you because you're perceived as unfriendly. "Making friends" isn't the goal, but having friendly professional relationships with the rest of your team is firmly on the happy path to get there. At least the way you're responding in the link, you seem to be leaning strongly into the opposite.

author: paulca
author_rank: Super Contributor
timestamp: September 06, 2024, 09:42:16 pm
content: Quote from: abeyer on September 06, 2024, 09:16:45 pmCrossed on the wire, but after a lot of conversations escalating things up stream... Having found some support.I finally remembered my priority. It's not that the project, it's documentation and processes make sense to "ME". It's VERY important it make sense to those more junior to me.I should not be isolating myself. I should be becoming their leader.I can empower myself easily by supporting others through the turbulence as I have the experience and the knowledge to guide.Management can challenge me. Of course they can. But I am a software engineer. I engineer software. That is a 90% people problem. You have deliverables, you have resources.... "people". You typically (as an engineer) have no control over requirements or the people you are given.The way I have previously embraced this and have forgotten, is to ... defend the team. Lead from within. Feel and express the pain the team feel. Become a "gate" for BS. Defend the team. Defend the progress.I don't care who stands in my way, if they stand between me and "working software which meets the customers needs" they stand apposed.My realisation is that, this is exactly what any and all juniors and seniors WANT to do. If you empower them they will flourish. Suffering because they have no voice to speak out. I will be their voice. It's easy to shoot down a 2 year junior. It is not so easy with me.I have done this before. I know how. It's a change of protocol. Focus on the people. Without the people you have nothing, you will achieve nothing. Support the people and they will follow you. If I can "empower" and motivate even two people I can double output.

author: abeyer
author_rank: Frequent Contributor
timestamp: September 07, 2024, 08:26:43 am
content: Quote from: paulca on September 06, 2024, 09:42:16 pm...said Spartacus, shortly before being crucified.

author: The Soulman
author_rank: Super Contributor
timestamp: September 07, 2024, 10:03:52 am
content: Paulca, I believe you are on the right track.A while ago I was reminded here by someone here (can't find it anymore) that all humans are basically black-boxes with an data input and a data output.With no or poor input data most will produce bad or useless output data, or simply near random noise that is hard to filter.But with the correct input ALL black-boxes can produce good output data.Key is finding the required input format, data-size and timing, it is different for each box but a solvable problem.

author: Siwastaja
author_rank: Super Contributor
timestamp: September 07, 2024, 12:11:52 pm
content: Extremely opposite viewpoint: programmers who require carefully crafted exact inputs to produce useful results can be replaced by AI, and said people are quite afraid of losing their jobs, some actually are already. We would need more programmers who are self-managing, have an idea of what is The Right Thing To Do (regarding user experience, software architecture, etc.), big picture and so on.Then again, I also realize this process does not scale. If you are doing something complex and large, you can't find such geniuses and even if you could, they would disagree with each other and are hard to synchronize. So we need accurate descriptions, ticket systems, and hierarchies of leadership and planning.The first extreme is useful in small to mid size projects with right people, but dysfunctional otherwise. The latter is dysfunctional in small projects as it leads to overengineering. Only use the bureaucracy when it's truly needed.The trick is to find the balance between these two extremes.