DBLib clarity - Page 1
Forum section: electronics / general

author: PlainName
author_rank: Super Contributor
timestamp: January 24, 2021, 05:49:26 pm
content: I am pondering a move to dblibs from intlibs but there are a couple of details that I can't find an answer to. Perhaps someone knowledgeable would be kind enough to point me in an appropriate direction... I think initially I would be using an excel sheet on the basis that it's easily editable or otherwise messed around, and similarly easily importable to a real DB once a suitable editing tool is acquired or made.First, For multiple suppliers I note that there are fields like Supplier 1, so presumably the database would need columns for as many Suppplier n as I might find to supply a part. Off-hand that could be 5 or so (RS, LCSC, Mouser, Digikey, Etc). Is it as simple as that?Next, and related, I already have a database of parts with references, suppliers, location, etc. I can get AD to read that database (it's Access) but the suppliers are a linked list rather than numbered field. Can the DBLib cope with that?Perhaps more importantly, the parts database doesn't have AD-specific fields (schematic symbol, footprint). It does allow a limited number of user-defined fields so I can fix it that way, but is it possible to combine more than one database into a master database, so the AD-specific stuff is in one and non-AD (or shared) stuff is in another?

author: PlainName
author_rank: Super Contributor
timestamp: January 25, 2021, 04:35:49 pm
content: And... how does it cope with multiple footprints for a device?This isn't where there may be a SOT-23 or SOT-323, which would have different part numbers, but where the footprint may have the L, N, M (or other, dependent on PCB shop) versions for the same part number.

author: ajb
author_rank: Super Contributor
timestamp: January 25, 2021, 04:48:37 pm
content: Quote from: dunkemhigh on January 24, 2021, 05:49:26 pmyes, although be aware that, afaik, suppliers in the database behind a dblib don't really mesh well with supplier data in ActiveBOM projects--or at least I haven't figured out a good workflow for it.QuoteA linked list in an access database? Do you mean a linked table? AFAIK the backing database needs to appear to Altium as a single table, that has all of the data AD expects right there. If there's a way to make the overall data appear and respond to queries in the same way a single table would, then it should work, but how you get there depends on the database in question. I don't know if that's doable in Access.QuoteAs above, that will depend a lot on the database in use. One option is to define your own part numbers or other identifiers, then use those to link between AD data for a part in one table and whatever other data is in your other table.Quote from: dunkemhigh on January 25, 2021, 04:35:49 pmSame as for suppliers where you have "Supplier", "Supplier 2", "Supplier 3", etc, you can have "Footprint Ref", "Footprint Ref 2", "Footprint Ref 3", etc, and these will all show up in the component panel when you select a part.

author: PlainName
author_rank: Super Contributor
timestamp: January 25, 2021, 05:50:52 pm
content: Thanks for the commentsQuoteMore than likely! Last time I messed with a database is was DBIII, so I no doubt have the terms wrong. It's linked, whatever it is though.QuoteYeah, I hadn't thought through that fully. I can't modify my parts database (in the send of adding extra tables or fields) but it would be OK to process it into something else periodically. At leas for now.QuoteAh! Thank you - obvious now you say it.

author: Pseudobyte
author_rank: Frequent Contributor
timestamp: January 25, 2021, 08:15:56 pm
content: All I have to say is Excel is garbage for databases. The odbc connection is slow and flakey as hell.

author: PlainName
author_rank: Super Contributor
timestamp: January 25, 2021, 09:11:17 pm
content: Yes, I agree with you. But from the user perspective, and particularly for a flat database that Altium will use, the spreadsheet format is a LOT simpler to deal with than a pukka database backend without a frontend.

author: Elasia
author_rank: Frequent Contributor
timestamp: January 25, 2021, 09:55:17 pm
content: Mostly just comes down to how skilled you are with working sql statements if you want the raw of itEasiest to just start at where you will end up either way instead of mucking around with excel etc and making those work.. thing is with sql you can use it to make a composite view for altium to interact with vs all your relational data you are trying to work withFeranec has a decent video on basic setup with a mysql dbI use MSSQL myself but potato potatoe

author: PlainName
author_rank: Super Contributor
timestamp: January 25, 2021, 10:20:16 pm
content: QuoteZero skill. I can learn, but without ongoing use it's going to be forgotten, so next time I need to mess with it I'll have to learn all over again. BTDT and, for once, I'd like to use a tool rather than perpetually make sub-tools to support itQuoteYes, I agree. Where do I want to end up, though? I don't know at this point, so something that is easy to get going, change, and figure out what I really wanted to do, then really doing it. I am sure I will go the proper database route eventually, but I don't know at this point what I want to see when I get there.

author: PlainName
author_rank: Super Contributor
timestamp: February 08, 2021, 04:11:12 am
content: Just in case anyone wondered where this went...Figured I'd try the Excel route because:* No server to maintain* No fancy database language to learn* Everything exists in a single folder (well, under a single folder) for backups, etc.* Easy to manipulate the dataSpent some time reorganising the intlibs since that would be a useful side effect even if it don't go any further. Then used the Altium lib list facility to get a .csv per .schlib. Read into Excel, changed the headings as appropriate, dumped unneeded columns, etc. Created dblibs from the .xls.Magic. Worked well and made things like changing all caps to use a generic cap symbol a piece of cake. What it doesn't do is allow one to make changes when Altium is running since the Excel sheets are locked all the time. Grrrr.Really didn't want to go the Access route because, well, it's Access but also because it is 32-bit Access and Altium wants 64-bits. There is a way to achieve this but it's a knife-edge - a seemingly innocuous Office update can upset the balance, and that means starting again with a complete uninstall of Office, installing the 32-bit version, doing the 64-bit hack, thowing in a few sacrificial chickens... Already hit that (but noticed quickly enough to be able to restore from backup), don't want to be forever worried about doing things on my PC>Midway solution seems to be sqlite, which has the advantages of a pukka database but doesn't need a database server. Unfortunately, it turns out that that also suffers from Altium holding the files open. Grrr. But there is a fix in the recent squlite3 which works around that, so that was the next thing to try.And try again. Had to use DB Browser to import the data and that removes spaces from strings when the string is column header. No, really! So none of the columns were recognisable in Altium without mapping, which I didn't want to do. Also put the kibosh on my clever scheme to do mass updates: export to csv, use Excel to do the biz, import back to sqlite3. However, it turns out that if the table column headers have the spaces (by manual editing), the imported headers with spaces which have the spaces removed on import, still match so the data ends up in the right place.A database browser other than DB Browser might be better behaved. Of course, a proper database tool that allowed the sort of manipulation Excel does but without learning SQL (and without having to type in loads of arcane and lengthy lines) might exist, but I'm blowed if I can find one for a reasonable price. That works, anyway.So the current situation is sqlite database with Excel export/import for bulk operations. However, I am aware that there is no backward route from here, so not sure if it will stay when I need to add more parts, or if I'll do a parallel update of the intlibs for a while.

author: ajb
author_rank: Super Contributor
timestamp: February 08, 2021, 04:20:59 am
content: Glad you got it working! Once you go dblibs I don't think you'll want to go back, unless you get to the point when you're willing to pay for a true library management solution.I've always used access and it's been perfectly fine. I just have a single table, so it's effectively just a multi-user excel sheet. I do remember needing to jump through a hoop or two at the 32->64 but switch, but no trouble since then and it's a perfectly serviceable all-in-one solution for this.

author: PlainName
author_rank: Super Contributor
timestamp: February 08, 2021, 04:48:47 am
content: Thanks. And I appreciate the gentle nudges along the wayThe 32-64 thing shouldn't be a problem with current Office, but we stick with 2010 here and that's 32-bit because Microsoft recommended that, and some of the addons are 32-bit only. Previously I might have been tempted to try migrating to 64-bit, but now that sqlite is working there doesn't seem to be the need.

author: PlainName
author_rank: Super Contributor
timestamp: February 08, 2021, 05:39:18 am
content: Hmmm. Just noticed comments are being vaped on update.Thought I would replace already placed components in a current design with the new library parts, which is easy enough. However, the comment field on the schematic symbol is then deleted. Doesn't matter if the part to be changed has anything in the comment, or if the database part has anything (same or different), on updating the comment field is blank.Is this expected? If I place a new part from the new library, the comment field contains the expected text. It is just when updating that it's lost.

author: radar_macgyver
author_rank: Frequent Contributor
timestamp: February 08, 2021, 05:52:12 am
content: Have not seen the disappearing comments issue (I'm on 21.0.9).Just curious why you'd want multiple footprints if you have supplier part numbers in your db. The supplier p/n is linked to a single footprint only, so you wouldn't want more than one footprint unless you do something like have one footprint for hand soldering and another for PnP.My way of dealing with this was to set up a mysql instance, and use Access as a front-end to edit the tables (access can view remote tables).

author: PlainName
author_rank: Super Contributor
timestamp: February 08, 2021, 06:52:11 am
content: QuoteGenerics. You might not have a specific part, never mind a specific manufacturer's part, in mind when designing a circuit. You just know it will be a resistor of roughly this value. For a simple project, you might even know it will be 0603 or something, and you can source it from the cheapest or quickest vendor when you order. Or course, there's a risk that you 15-year-old stockpile, which is the last remaining source on the planet, ran out with the previous project...Strangely enough, I really needed multiple schematic symbols for a specific part, which Altium doesn't allow. It does allow alternative symbols, but that's not quite the same thing. In my specific case I'm using a relay so with the alternate symbol I can show the coil as a coil or box, say. However, what I can't do is have the coil as part A and the contacts as part B (so they can be arranged more appropriately on the schematic). So I end up with two different parts that have the same manufacturer's part number. Fortunately, the manufacturer's part number is not used to identify parts in my database, although this isn't the reason for that

author: PlainName
author_rank: Super Contributor
timestamp: February 08, 2021, 08:57:11 am
content: Forgot to add:Most SMD footprints have more than one variation depending on manufacturing process and packing density. Typical passives from Altium, for instance, have three footprints. In the old days, PTH footprints might have the little sticky-out bit for bending the leads over and fixing, or might have vertical mounts, etc. So having more than one footprint for a specific manufacturer's part number is actually the norm rather than some strange quirk.

author: ajb
author_rank: Super Contributor
timestamp: February 08, 2021, 07:47:57 pm
content: How are you updating the parts from the old library with parts from the new library? If you do "Update from libraries" (which I guess isn't what you're doing because you're switching libraries) there is an opportunity to choose which parameters get updated. There have been bugs in this in the past, I don't know if they have been corrected or not. If it's just getting blanked out that's strange and probably a bug. As a rule I try to avoid relying on making any manual changes to component parameters in the schematic for reasons like this. Also it's arguably better to have the library be the authority on component data like that rather than having data embedded in the schematic where it may or may not be visible. It would be really nice if you could attach directives to components for situations where you need to assign data in the schematic, that way it's not subject to synchronization with the component library at all.Needing to keep an older version of office makes sense, so yeah, if Access doesn't work in your environment then it doesn't work.

author: PlainName
author_rank: Super Contributor
timestamp: February 08, 2021, 11:47:23 pm
content: In the component property, for the source field, I hit the down arrow and select the appropriate library. I then get a 'Component source' popup from which I can select 'Clean' (remove existing ID to select manually later), 'Do not change', and 'Select' (select new component manually in the modal dialog). I go for the latter and, surprise, get a modal dialog from which to pick the library and component. Select the right one and it updates in place perfectly, except that it loses the comment field.I also noticed in playing around that placing a new component has the comment field invisible regardless of whether something is in it - for a relay I might have 'SPNO' in the comment, and on placing it the comment still retains that but is hidden. The schematic symbol that is linked to has the comment field visible.This is 21.0.9. Not a show-stopper but quite irritating.

author: radar_macgyver
author_rank: Frequent Contributor
timestamp: February 09, 2021, 05:10:49 am
content: I think the recommended way to do it is click the ellipsis button "..." next to Design Item ID, and you get a chooser to pick another part in the same library. At this point, you can also expand the "categories" tab to choose a part from a different library.How I deal with generics is to place a part with an approximate value, or similar functionality, and a text note indicating I need to change it later. Once the design is close to finalization, I either make a new part if needed, or choose an existing one.Quote from: dunkemhigh on February 08, 2021, 08:57:11 amMy mysql db uses the manufacturer P/N as the key for the tables, so the db won't let me enter non-unique values. It was my way of ensuring no duplicate parts, but it did mean no secondary footprints to address manufacturing concerns. I used to do this by replacing the footprint on the PCB - I'll consider using the multiple footprint option now. Also, it was an honest question, didn't mean to pass any judgements.Quote from: dunkemhigh on February 08, 2021, 11:47:23 pmCheck Tools->Preferences->Schematic->Defaults, select "Part" from the primitive list and see if the comment is hidden.

author: PlainName
author_rank: Super Contributor
timestamp: February 09, 2021, 10:32:59 am
content: QuoteNo problem - I took it at face valueQuoteNot hidden, and not hidden under 'comment' primitive either. Same applies for the Excel-based database, but the original intlib does show the comment field. All of these use the same .schlib so it seems to be something about the database aspect that is causing it.Not sure if it's' relevant, but when selecting the part in the component panel, the schematic symbol shown in the models pane does show the comment field contents.

author: ajb
author_rank: Super Contributor
timestamp: February 09, 2021, 04:33:52 pm
content: Hmmm. Do you have a "Comment" field in your database that you're wanting to use to drive what appears on the schematic? I have the comment field in my schlib symbols set to "=Value" which links it to the database field called "Value". I don't remember how I got to that arrangement, but that may make a difference. I think the comment field is not handled the same way as a generic parameter because of the way it's used in the PCB. It could be that there's some sort of special handling of the comment field that fails if there's a disagreement between the comment value in the database vs the part?

author: PlainName
author_rank: Super Contributor
timestamp: February 09, 2021, 10:10:03 pm
content: Not in the intlibs - the comment field holds the value (or chip ID, etc). In the database version, some libs I left like that and some I changed to have an additional 'value' field. Makes no difference - the intlib shows the comment on place, the databases don't. I also tried setting the database to update and add on placement. I don't think that's an issue because it does update the comment field, just also sets it to hidden.I think I might take this is up with Altium support one night. It's not a biggie in itself, but might point to a bigger issue I'm overlooking.

author: jc101
author_rank: Frequent Contributor
timestamp: February 10, 2021, 05:49:21 pm
content: Quote from: dunkemhigh on February 09, 2021, 10:10:03 pmIn the database library table view, has the comment been set to be displayed by default? Any of the fields can be set to hide or show on placement, by default it is set to off.

author: PlainName
author_rank: Super Contributor
timestamp: February 10, 2021, 06:31:59 pm
content: Yes, in every table the comment is visible.I might be getting closer to the cause. If I make the comment visible it is overlayed on the schematic symbol instead of being just below. So perhaps it is being hidden because it's conflicting with the symbol body. I note that the designator is at the bottom, instead of being at the top, as well. I could go with this being the cause and the fix being to explicitly set these strings, but then the intlib - which uses exactly the same symbol - should be affected too, and it's not.But, anyway, I explicitly set the positions in the library (even though the 'models' pane of the component panel shows them in the right place), but it made no difference.

author: PlainName
author_rank: Super Contributor
timestamp: February 10, 2021, 06:38:10 pm
content: Ah! Ah-ha! Found itIn the dblib there is a column "Visible on add" for comment, which was unticked. Ticked it and now it works. Annoyingly, there isn't a global setting for this so every table has to have it set individually.

author: jc101
author_rank: Frequent Contributor
timestamp: February 10, 2021, 06:39:51 pm
content: Quote from: dunkemhigh on February 10, 2021, 06:38:10 pmAh that’s the thing I was referring to, tried to remember the name without firing up a copy of Altium to get the exact name.

author: PlainName
author_rank: Super Contributor
timestamp: February 10, 2021, 06:59:33 pm
content: Oh, well thanks for pointing me in the right direction

author: plazma
author_rank: Frequent Contributor
timestamp: February 14, 2021, 11:49:23 am
content: I built a component database library to my workplace. The unique identifier is a company internal code received from the purchase department. Each manufacturer part number got its own internal code. In practice I could have used the manufacturer part number as the unique identifier.I'm building my hobby database library on exel. There is not so many codes so excel sheet works fine. For this database library I use the manufacturer part number as the unique identifier.Each component can have multiple footprints. Most have three: IPC Least, Nominal and Most. Some have a fourth footprint which is usually manufacturer specified.So for example a part with QFN and SOIC footprints appears twice in the library. That's because the QFN and SOIC parts have their own manufacturer part numbers. And both of them can have multiple footprints. Like QFN can have IPC L, N and M. Also the SOIC got the different IPC densities.

author: PlainName
author_rank: Super Contributor
timestamp: February 14, 2021, 12:15:34 pm
content: QuoteSeems to something of a favorite, but I wonder how appropriate it is. A major issue is that there is no guarantee that manufacturers will have unique numbers between them, but another is that the format varies wildly.Instead, why not use a guaranteed unique to you number? The formatting will be consistent and appropriate (if you wanted, you could ensure no 0 or 1, so you don't confuse them with O and l), so you know they will fit in the space provided, and you won't end up with 40-character-wide columns showing 6-character numbers.

author: Pseudobyte
author_rank: Frequent Contributor
timestamp: February 15, 2021, 03:57:35 pm
content: Most people use an incrementing index for the unique identifier...I think?Would anyone be interested in seeing how to setup an AWS RDS (postgresql) deployment for Altium DBlib? If that is valuable even to a few people I can write up a post.