How database management becomes Nightmare - Page 1
Forum section: electronics / general

author: jismal
author_rank: Contributor
timestamp: November 29, 2018, 01:02:02 am
content: Hello guysHere is a long story.Ok after my college days like all other young Indians I was also forced to getting a JOB ASAP.In India, things are different like my sister says if you throw a bunch of stones at young people out of ten seven will be engineers. But realistically speaking, no one has the talent of engineers they are just byhearting people just to pass their semesters.I was lucky to get a Job at end of the semester, My company was mainly focused on firmware, cloud, Linux apps etc. So as an electronic engineer I went into firmware coding (I hate coding, I am just a regular hardware guy love to built stuff). Anyway after two months of training and writing WPA2(Wifi Security) on a custom SOC (Cortex M0).After some regular Board of directors, meetings company found that it needs a Hardware Team. So they immediately find a project manager who was experienced in building Large server boards but no small IOT devices which were our company focus. We also find 4 others inside the company(who was previously writing firmware and test codes). We 6 Together form a team so we 6 out of 200 become hardware people(The black magic people).But there was a small problem. Our company doesn't have any experience in making a Real product or hardware from scratch to product, they even try to bring an agile model for hardware development (They excel at firmware, cloud, apps you know the drill regular software guys). Our manager gives us some training but really it is just how to deal things on the management side, not from a design point. After the training, we need to set up a lab from scratch where the most costly equipment is a (DSOX2024A Oscilloscope: 200 MHz, 4 Analog Channels). Yes, we don't have any RF equipment or a function generator(yet we do some RF stuffs).Anyway, our tool of choice was Cadence (we don't do PCB Layout tool, we outsource the layout work). So our project manager was insisting on using a database for components. As we five do not have Microsoft office tools(Mistake One). We are forced to used MSSQL(Mistake Two, No one knows how to use MSSQL ) and set up a server system(It's just a LAPTOP sitting on another side of office) Where are all the project are shared by smb windows folder (The Laptop doesn't have a raid so we are in the mercy of Google corporate account of 30Gb for our growing Project size)and The MSSQL Running running as server. So after weeks of learning MSSQL queries and classifying different parameters for different types of components, we finally set up the database and Its starts running in our Cadence capture. The fun fact now we need to go into Digikey and Mouser select a component use LibreOffice base to enter the component parameter into the server then create the schematic models and verify each component (we do have a checklist for each component, even for resistors).So everything was running smoothly, Our first project was nothing but a simple ethernet relay board. God knows how many mistakes we have done, it's like hell for us five, the part symbol is wrong, the design is wrong, most of my mate doesn't really built anything or designed anything. My manager thinking was, if we get all the document right the design will automatically correct(oh boy I wish I could tell him how wrong he was).So the second project starts it has a design with USB, BT/BLE,eMMC,433Mhz Radio, cattery charger, fuel gauge, Anyway we pull that off, God knows how much detuned my RF circuit was.OMG, now that product is going into mass production. You need to know that this board was built by guys who were straight out of college and most of them take Electronic degree because their parent forced them or did not get their first choice.After some time I found that most of our customer base prefer Altium in place of cadence capture. So we buy an Altium license one small problem Altium doesn't want people to get out from their ecosystem they implemented stupid vault and integrated library. Anyway, there is an option to hook external database we use that feature, one small problem the stupid Altium designer doesn't know what is a relational database. All our Cadence capture database was of relational data for vendors and manufacture(this give us the flexibility to add as many of the vendors or manufacturers as an alternative). Because of the Altium now we need to flatten our relational database. This limited our ability to add only 3 Manufacture and 5 vendors we can add more but it will complicate the tables even more. We flatten our database it takes about 3 days. And now we need to redo all the cis configuration in cadence and need to create all the schematic symbol and reverify all the symbol (again we need to create a checklist for each component).Again for Altium project, there was an entire fleet of mistake most of them come from our inexperience of the tool. Some database field doesn't show up, One feature of Altium which allow us to hide some portion of the circuit this feature allow Altium to use same reference designator for circuit hidden under it and outside of it which make us order wrong part number, Thank you Altium for not giving us warning on compiling.Ok after struggling through this we were yet to get into the problem of inventory management of excess components. After five projects excess stock is getting piled up. Our manager says there is software to manage it, but cannot convince the company to buy one. So now we need to check for excess component before going into proto built. I think this was not the right way, so I started looking into various open source software for this, but none of them suite my requirement because I already have a database which works with Altium and cadence together I don't want to mess it. Also, I need to change the data entry into our database through LibreOffice. What I really need is a good web interface where I can add data. I don't want anything to be an online tool which will lock us into one ecosystem.After some time I try to use snipe-it this is its asset management software which is used up by our office to manage Laptops, Licenses etc. Snipe-it is an open source software. It is an excellent software. I hook it up and modify it as it was completely open, But we hit with another problem snipe-it does not support our Corporate MSSQL, it only supports another corporate software MSSQL. Because of easiness, only ii used MSSQL in the first place. Again I need to port my MSSQL to MYSQL anyway this was an easier process. But again snipe it cannot run into our existing database so I modified it and write a 1000 line long trigger such a way that if something comes into snipe it through web interface check the component is a component required in EBOM if it then updates the database containing cadence and Altium data. This workflow is in the early stages. Anyway now I would like to modify some of the snipe-it web interfaces to make it as a true EBOM software which can handle multiple ECAD software BOM and finally integrate it into Octopart, Mouser or Digikey.If someone can also share their nightmare from starting everything from beginning it will be very useful. All industrial experts can give better guidance on how to do it properly.

author: EmmanuelFaure
author_rank: Regular Contributor
timestamp: November 29, 2018, 12:04:37 pm
content: Quotehttps://en.wikipedia.org/wiki/Bullshit_Jobs

author: bd139
author_rank: Super Contributor
timestamp: March 25, 2019, 08:22:55 am
content: LOL this is nothing. One of my first IT jobs was integrating HP CoCreate with ME10, solidworks and Zuken.Step one: buy Oracle box. £1 million later it was installed.Step two: integrations. 5 people and two years later. Mini rebellion from staff who refuse to use it because it means they are accountable so go back to using envelopes and index cards for management.Step three: realise the whole thing was a fucking disaster but are forced to push it through anyway. Thus the only way to get people to use it was sabotage their existing system. Someone went through the factory floor and binned a few envelopes. Then factory goes down on lockdown as defence assets are no longer accountable.Step four: profit.Now I understand the year after I left they had to buy another Oracle box.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: March 25, 2019, 10:13:44 am
content: Thanks for the story, interesting insight (or, more rather: "nothing new under the sun", we have the same problems everywhere eh?..)I'm currently working a job with a large electrical equipment supplier; they use a database for parts management (including libraries), and SVN for design file management. I think the way it works is, library files are linked from the database; these files are shared via SVN, and you search one massive DbLib (well, the DbLib isn't massive, it links into the massive database).Part creation seems a bit onerous, taking maybe half an hour to do a part. Although that doesn't really sound so bad, when you consider spending about as much time reading the datasheet, and drawing, measuring and verifying the symbol and footprint anyway. Less, of course, for reuse cases (resistors, say).When it comes to actual use, at most, you need to: 1. update SVN (you have a local copy of every footprint and symbol in the database), 2. refresh the library, 3. search for part, 4. place part. Nice thing is, at this point, they have thousands of parts, including reasonably diverse resistors, capacitors, diodes, etc., and they've all been purchased and DFM'd and assembled at some point. I'm not sure that the system has paid itself back in terms of design time -- but I can see the back end (alternate parts/sources, inventory management, etc.) is probably more valuable, so may still be worth that cost to us.Take this as an example, I guess, of a system that's more or less working -- I don't know if it's overall more efficient, but at least it's not a cluster of hacked together interfaces.For those just getting started -- by themselves, or at a company that hasn't done much engineering -- I would suggest avoiding terms like "database" that will confuse your managers. Just refer to them as libraries, or graphics or whatever -- don't let them think you need a database, the overhead will lead to no end of grief. For your own sanity, follow consistent naming conventions (pick up IPC-7351B, it's very useful), probably divide libraries by manufacturer, or part type, rather than using one mega library, and if you have an inventory / PN system already, tying that in (with a simple DbLink say) can be easy enough, just don't go out of your way to support things you won't need (like, if you're ordering assemblies from CMs, why make a part for every component on the board?).Tim

author: jismal
author_rank: Contributor
timestamp: August 20, 2019, 07:01:41 am
content: Thank you for you replies.