I'm making the GitHub for Hardware - Page 1
Forum section: electronics / projects

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 09:03:18 am
content: Hey all,I've been working on a project for the last year that I wanted to share with you and get your feedback on - it's called Stemn. It's a website where makers can share the projects they're working on, with specialised support for hardware file formats.The best analogy is a 'GitHub for Hardware'. Unlike existing sites where people upload a zip of their output files (e.g. a forum build thread), each person's project is a real-time, up to date version of the original source files. This way the project stays 'alive', and doesn't end up being a graveyard of outdated files.In the future I want to add the ability for others to contribute to a project by submitting their alterations of the files back to the original project. This way we could achieve the same collaboration that happens on GitHub, but for hardware.Some points of note are:An Arduino rendered natively in the browserThe Arduino project files, hosted live from my DropboxDogfooding Stemn on Stemn - our future roadmap!Comparing changes between different gerber versions(soon to be implemented on our beta website)My vision for Stemn is to enable people to share their hardware designs so others can build upon them, accelerating the state of the art in open hardware, and giving better access to hardware to millions of people.The site is very much in beta - it has only been tested for a few weeks. I'd be really thankful if people could tell me if the site is useful for them in its current state, and how it could be improved for their needs. I'd love to take your ideas and implement them!

author: Monkeh
author_rank: Super Contributor
timestamp: September 25, 2017, 09:24:33 am
content: The rendering's kinda neat, but.. everything else already works on Github. Hardware, software, it's all files, and git works with any pile of files.

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 10:21:28 am
content: Neat, thanks! How do you use GitHub - through the cli or the GitHub desktop client?I forgot to link to ourdesktop clientin my original post. It's a version control system that tracks any files you store in Google Drive or Dropbox. For those not using Git via GitHub already, I think this could be pretty useful.

author: TK
author_rank: Super Contributor
timestamp: September 25, 2017, 10:42:58 am
content: Also ask for permission from Arduino to use their name and original board image. The design is open source, but not the Arduino name and they had a long legal battle to protect it. You can use their design files to create your own board that can look exactly like the Arduino, but without the name on it or you can use the Redboard from Sparkfun or Metro from Adafruit and ask them for permission, I think they will be more flexible and open than the Arduino team.

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 10:43:57 am
content: Great advice - thanks. I hadn't considered that.

author: Monkeh
author_rank: Super Contributor
timestamp: September 25, 2017, 10:52:45 am
content: I use git directly. Github is far from the only place I use it.

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 12:04:18 pm
content: Quote from: Monkeh on September 25, 2017, 10:52:45 amGit level: elite

author: ebclr
author_rank: Super Contributor
timestamp: September 25, 2017, 12:20:02 pm
content: Is this Steamm another kind of circuitmaker with less functionality?https://circuitmaker.com/

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 12:27:30 pm
content: It's different from circuitmaker in that it's not an editor, but rather a version control system. It helps youwithyour work, it's not somewhere youdoyour work.

author: ebclr
author_rank: Super Contributor
timestamp: September 25, 2017, 12:30:49 pm
content: Something like a multivendor Altium Vault, then right?

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 12:36:29 pm
content: Yes, that's right. The difference is Stemn is cloud based, multi platform, and is primarily a version control system, whose client currently has great support for hardware file types.

author: Leo Bodnar
author_rank: Frequent Contributor
timestamp: September 25, 2017, 12:38:44 pm
content: Quote from: sabrehagen on September 25, 2017, 09:03:18 amI think you are overestimating your user base.There are only few tens of thousands of people who really know what they are doing, are capable of using versioning system and go online altogether.My browser is refusing to cooperate. I am still on Safari 5.1

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 12:55:22 pm
content: Quote from: Leo Bodnar on September 25, 2017, 12:38:44 pmI agree, the number is few now, but I hope to increase that number to many more through the community Stemn has the power to create.Yes, Safari 5.1 was released more than 4 years ago and does not support the modern web technologies in use by the Stemn website.

author: TimNJ
author_rank: Super Contributor
timestamp: September 25, 2017, 05:28:25 pm
content: I really like the idea and implementation thus far.I don't know a whole lot about Git so maybe my questions are off-base but...How will changes between schematics (for instance) be tracked? And how will that be visualized? Gerber changes are pretty straightforward, as you've demonstrated. You can easily see the physical differences between different board revs. But schematics with new/deleted connections/parts seems like it would be tough to show. And would the site be able to produce schematic previews?Will everyone on the project have to be using the same engineering toolchain/EDA tool to contribute? (I assume yes?)I know some people will say that hardware design collaboration isn't really something designers are even interested in. (i.e. A single designer will take the project from schematic to gerbers.) But I think if done correctly, then collaboration would be much more attractive. I think a good implementation of version control/collaboration would allow open source projects to become more complex and have a greater impact. Good stuff.

author: sabrehagen
author_rank: Contributor
timestamp: September 25, 2017, 10:15:58 pm
content: Quote from: TimNJ on September 25, 2017, 05:28:25 pmThank you!Quote from: TimNJ on September 25, 2017, 05:28:25 pmHow will changes between schematics be tracked? That's what I'm here to askCurrently the website is like a blank canvas - there's a huge amount of potential to be capitalized on, but finding what's useful and practical is needed. Engineers on eevblog.com likely know exactly what to look for between different versions of schematics. If we can write software to automate/assist in that difference comparison, then it's our duty to best assist engineers in that way.Quote from: TimNJ on September 25, 2017, 05:28:25 pmYes, you're right. Stemn let's engineers version and share their files, however each engineer will need access to the specific software to work on those files.Quote from: TimNJ on September 25, 2017, 05:28:25 pmI think a lot can be learned even for that traditionally one person process. Seeing how another engineer has laid out a board, and observing the transformation from schematic to implementation would be hugely useful for any student of the field. As the best share their work, newbies and pros alike can learn from their design decisions.Like I said, this is only a few weeks old. It's taking an idea and putting it into the wild. Having people excited about the future and how the site might help them gives us a way to refine and progress the site.

author: TwisteR
author_rank: Newbie
timestamp: September 26, 2017, 12:43:58 am
content: QuoteThis problem is solved decades ago in thegEDA/gafby providing theschdiffscript, which may be invoked both directly (to compare two existing files) or as a version control helper diff-utility, e.g.Code:git difftool -x schdiff file.sch

author: sabrehagen
author_rank: Contributor
timestamp: September 26, 2017, 12:51:46 am
content: Awesome - we could runschdiffon a server and present the output to the user in the web browser.One cool thing we've been experimenting with recently is the idea of a tooling/build pipeline. Often a script needs to be run as part of a user's workflow (e.g.CadQuery,schdiff) and this can be a repetitive manual task the user performs. Using Stemn we can automate this so as soon as the user presses save, these scripts are run, ready for human inspection - no more manual operation.Is this automated diffing I'm suggesting already provided by tools you use?

author: TwisteR
author_rank: Newbie
timestamp: September 26, 2017, 01:08:51 am
content: Quote from: sabrehagen on September 26, 2017, 12:51:46 amNot only diffing, gEDA itself is basically a set of tools (framework), tailored to build different pipelines and achieve automation of routine tasks via makefiles, etc. For example, all the data (schematics, pcbs, netlists, BOMs) are stored and transferred in a simple, user-readable and diff-friendly plaintext formats.

author: sleemanj
author_rank: Super Contributor
timestamp: September 26, 2017, 01:24:39 am
content: It seems quite a confused idea to me.It's kinda like github, but not really, there doesn't seem to be a fundamental underpinning of a version control repository (be it git or other), or if there is it's not very visible, there doesn't seem to be an interaction for forking or checking out a repo or...It actually feels closer related to thingiverse with some nascent (adhoc) versioning on files and a look more akin to github.The mentions of dropbox make my eyes open wide in horror... it makes it sound more like "chuck your stuff in random online drive and link to it on our site" which is pretty much the direct opposite of version control.There seems to be a lot of emphasis on the 3d model viewer...which would be good if it was a grabcad type site, but it doesn't seem to be that either.Not to knock the idea, just I'm not really sure WHAT the idea is, maybe it's a good one, but you need to make it clear why people should use stemn instead of github or thingiverse or hackaday.io or bitbucket or ... any of the other similars.

author: sabrehagen
author_rank: Contributor
timestamp: September 26, 2017, 01:27:57 am
content: Quote from: sleemanj on September 26, 2017, 01:24:39 amCool, thanks for the thorough feedback. You'd use Stemn if you use Google Drive or Dropbox to store your files and work on them with others, you work with hardware file formats that are not web-friendly, and you don't use a version control system.Of course, those who are on the fence about using Git currently may find the automated versioning provided by Drive and Dropbox (and presented as a VCS by Stemn) a convenience.

author: stmdude
author_rank: Frequent Contributor
timestamp: September 26, 2017, 05:57:09 am
content: Interesting idea. I like the gerber visual-diff thing. That would be very handy when it's layout design-review time.I downloaded the client, and it ran fine on my somewhat unusual Linux environment, so you did a good job packaging that.However, I don't store my project in Dropbox or GDrive. I store them in git. A local git for the paid stuff, and Github for the free stuff.I couldn't find any obvious way to integrate this with git. Is this in the pipeline?

author: rs20
author_rank: Super Contributor
timestamp: September 26, 2017, 06:07:31 am
content: Yeah, sounds like you need to work on your "marketing"... GitHub is all about git, and git is all about version control, front and center.Ask me: "What's your 'GitHub for Hardware'?" I answer: GitHub.And now you're saying that this is some sort of a VCS overlaid on Google Drive or Dropbox? Why didn't you use Git if you're making the "GitHub for Hardware"? When I hear "GitHub for hardware", I'm imagining a web GUI layer atop a hosted git service, like, um, GitHub. Instead, I've got a downloadable client that hacks a VCS over Google Drive. Very confusing!It sound like you don't are intimidated by GitHub, think all it is a more confusing kind of file hosting site, or straight up don't like it. Probably best, then, not to try to appeal to people who do like Git!

author: xani
author_rank: Frequent Contributor
timestamp: September 26, 2017, 10:01:33 am
content: I'd be interested if it was just "Git backend + sane graphical diffing on few popular schematics/board formats"Hell, even some plugin for GitlabI already put projects in Git because it is great for code that goes with the project and passable for board/schematic stuff (as KiCad, gerber and few others are just text files).Putting "hardware" in one version control system and software that goes with it in another is counterproductive

author: ddavidebor
author_rank: Super Contributor
timestamp: September 26, 2017, 12:52:52 pm
content: I'm fiddling around now.To be sincere i am bit torned right know.For one, I think there is a great need for a version control tool that works well across tools. I use git and submodules, but getting git to work on windows is not easy, and I risked to screw things already a couple of times.Using Gdrive and dropbox to do version control is not a bad idea. It's cheaper storage than github if you can only deal with binary files. easier to get started. Quite realiable.That said, if you think i'm gonna connect my google drive with a one-man-company, you are seriously mistaken.I am absolutely not interested in that job application stuff and curriculum stuff. It's just annoying until you have a big enough codebase. Also, the mac app does not work. Writing everything in javascript with electron will not make you immune to the problems of cross-platform development.

author: rs20
author_rank: Super Contributor
timestamp: September 27, 2017, 07:18:06 am
content: Quote from: ddavidebor on September 26, 2017, 12:52:52 pmI think it might be, given that Gdrive and dropbox are not version control tools...

author: sabrehagen
author_rank: Contributor
timestamp: September 27, 2017, 07:41:13 am
content: Quote from: rs20 on September 27, 2017, 07:18:06 amThey weren't, until now.

author: rs20
author_rank: Super Contributor
timestamp: September 28, 2017, 04:38:07 am
content: Quote from: sabrehagen on September 27, 2017, 07:41:13 amNo, that's not how English works. If you make a version control tool atop GDrive, you've made a version control too atop GDrive. You haven't made GDrive itself a version control tool.

author: sabrehagen
author_rank: Contributor
timestamp: September 28, 2017, 04:40:36 am
content: From the discussion here, it seems those use Git have no desire or need to change to a different version control system. Therefore, it would be of interest to me to speak to people who aren't using a VCS and are using Dropbox or Drive to store their files. Can anybody recommend friends to this thread who fit that use case? Would love to ask some questions of them.

author: ddavidebor
author_rank: Super Contributor
timestamp: September 29, 2017, 06:38:22 pm
content: BTW google drive DOES have basic versioning capabilities. But just simple object versioning, like S3.

author: julian1
author_rank: Frequent Contributor
timestamp: October 01, 2017, 01:48:34 am
content: It's worthwhile to explore git and Github. Github has (https://developer.github.com/webhooks/), so on every commit to the project/branch etc you can generate notification events. That enables third-party websites to do quite sophisticated integration - for example build executables/artifacts and test them, compile documentation, generate and publish entire websites etc. The ecosystem for developers is already pretty extensive.

author: sabrehagen
author_rank: Contributor
timestamp: October 01, 2017, 02:29:22 am
content: Quote from: julian1 on October 01, 2017, 01:48:34 amYeah I think the extensions provided by GitHub via webhooks are great. What I like most is it takes repetitive tasks and runs them remotely, freeing up your personal time and computing power. What were you thinking of with respect to webhook style integrations and Stemn?

author: rs20
author_rank: Super Contributor
timestamp: October 01, 2017, 09:01:58 am
content: Quote from: sabrehagen on October 01, 2017, 02:29:22 amInteractive in-browser Gerber viewing?Interactive in-browser Gerber diff display?Interactive in-browser schematic diff display?Interactive in-browser PCB layout diff display?And, in the "might not be worth the trouble" categories;Interactive graphical schematic merge conflict resolution?Interactive graphical PCB layout merge conflict resolution?But you know, take all the things where naive text-only version control gets awkward, and build graphics around it.

author: sabrehagen
author_rank: Contributor
timestamp: October 01, 2017, 09:10:50 am
content: How do you do PCB layout diff display now?

author: TwisteR
author_rank: Newbie
timestamp: October 01, 2017, 11:48:28 pm
content: Quote from: sabrehagen on October 01, 2017, 09:10:50 amNo problems whatsoeverSame principle as withschdiff— pixel-wise diff (across the layers of interest or the design as a whole).

author: sabrehagen
author_rank: Contributor
timestamp: October 03, 2017, 07:57:56 am
content: Quote from: TwisteR on October 01, 2017, 11:48:28 pmDo you do this at the command line? e.g.git difftool -x schdiff 182110f690c51eb53ea1799fd022e87322d742e3 039e012397f2638d1577dd7c80fb862a5503dbfd cpu.sch

author: TwisteR
author_rank: Newbie
timestamp: October 03, 2017, 12:50:54 pm
content: Yes, that is correct.In fact,schdiffis quite trivial script itself. You may check it yourself by issuing the following command:less $(which schdiff)It just obtains two grayscale images and then uses ImageMagick'scompositetool to overlay one of them over another in 'stereo' mode.As you may see, the only thing worth bothering at this stage is obtaining board/layer images suitable for comparison. That task may be accomplished in a unified way by using 'omnivorous' tools/libraries liketranslate2geda,pcb-rnd,altium2kicad, etc.

author: ehughes
author_rank: Frequent Contributor
timestamp: October 05, 2017, 01:47:13 am
content: What is the business model for the site?Some of my 1st thoughts:1.). There aren't millions of people looking for open hardware to build upon. The hardware community is tiny.2.). Makers will never pay anything for version control. Git is free.3.). A Pro won't put their designs in the hands of a 3rd party.4.). Visual differencing seems to be the primary feature. Even in commercial tools that do this already, It is not a feature that is used a lot.5.). Makers don't document their designs well. It would be easy for the site to turn into a swamp.

author: sabrehagen
author_rank: Contributor
timestamp: October 05, 2017, 02:50:40 am
content: Quote from: ehughes on October 05, 2017, 01:47:13 amThanks for your thoughts! My thoughts on those points are:1. It will grow once there is more open and accessible hardware resources. Right now the barrier to entry is high, but as independent learning becomes easier, more people will move into the space.2. People will pay for what makes their life easier and saves them time.3. This is an old and dying idea that will become moot in future.4. The primary feature is version control for Google Drive and Dropbox. Visual diffing is a useful feature added on top.5. Historically yes, but when communities like GitHub make people take pride in documentation so that others can replicate their work, the quality of documentation improves rapidly.I find it very useful to correlate people's age and experience with their perspective on the industry. How old are you, and how long have you been working in the field?

author: sabrehagen
author_rank: Contributor
timestamp: October 17, 2017, 10:15:24 am
content: Ping ehughes