Using an Excel file as library database - Page 1
Forum section: electronics / general

author: hkBattousai
author_rank: Regular Contributor
timestamp: June 26, 2016, 08:06:00 pm
content: I had been using discreet schematic and footprint libraries for every component. Recently, I decided to start using a database system for some advantages it brings. I watched thisand readthis official documentation article. The nicely explains how to do it with Access based database. However, I don't have an Access license, so I want to do the same thing with an Excel based one.As far as I understood, Altium doesn't create the Excel file itself, but requires the user to initialize the database with an existing Excel one. The problem is, I don't know what Excel file format Altium expects to see. I prepared the simple Excel file in the attachment; which is nothing but an empty file with only some headers on its first row. Altium didn't accept it and give the error message "Connection Failed. Check you connection settings.".I searched Google results for an hour, but couldn't find how to use an Excel file as library database. Every search result says that it can be done with either Access or Excel, but none of them explains how to do it with Excel. The official documentation doesn't help at all.Can you please tell me how to use Excel as library database?

author: nctnico
author_rank: Super Contributor
timestamp: June 26, 2016, 08:16:41 pm
content: Get an access license (cheap on Ebay for older Office versions) and don't mess with Excel.

author: electricar
author_rank: Regular Contributor
timestamp: June 26, 2016, 08:29:21 pm
content: You have to save it as an .xls-file.

author: Gribo
author_rank: Frequent Contributor
timestamp: June 26, 2016, 09:03:05 pm
content: SQL server express edition is free and much better than Access. Also, with a little bit more work, you can get MariaDB to cooperate with Altium.Edit: To use Excel, you need to setup an ODBC data source, and choose the right Excel file.In the DB lib (IIRC) you select the ODBC source and it should show you your table.Then you can map component field to the table's columns.

author: nctnico
author_rank: Super Contributor
timestamp: June 27, 2016, 01:04:52 am
content: Quote from: Gribo on June 26, 2016, 09:03:05 pmMaybe as a database but Access has easier data editing build in.

author: Randy_Clemmons
author_rank: Newbie
timestamp: December 31, 2016, 04:54:36 am
content: Purchasing Access is not required to use an Access database.You can install the free Microsoft Access Runtimehttp://pcbparts.blogspot.com/2014/12/download-access-runtime-microsoft-site.htmlYou can download a free complete working Access DbLib at:https://pcbparts.blogspot.com/p/downloads.htmlYou may need to install "Microsoft Access Database Engine 2010 Redistributable" to get Altium connected to the database.https://www.microsoft.com/en-us/download/details.aspx?id=13255

author: slicendice
author_rank: Frequent Contributor
timestamp: December 31, 2016, 09:59:30 am
content: Here is a few links to get you started with Access DB in AltiumThe main page:http://techdocs.altium.com/display/ADOH/Linking+Existing+Components+to+Your+Company+Database#LinkingExistingComponentstoYourCompanyDatabase-ConnectingtoanExternalDatabaseThe sub page:http://techdocs.altium.com/display/ADOH/Using+Components+Directly+from+Your+Company+DatabaseThese should help you get started. Personally I would go for other DB solution than Access as it sucks big time and is really slow. When working with databases, just following guides might not be enough. One has to have some form of understanding about how databases really work. ( I studied databases, how set them up and how to program them, for 2 years. There are so many pitfalls one has to aware of before even creating a database server/service. )If you get in trouble, a better way would be to pay someone to setup the thing for you, so that it works great.

author: technotronix
author_rank: Regular Contributor
timestamp: January 02, 2017, 02:38:33 pm
content: I don't know much about it, but here may a way to use excel file as a library database.https://www.reddit.com/r/ECE/comments/11z15e/does_anyone_here_use_altium_with_database/

author: mrtn
author_rank: Contributor
timestamp: January 03, 2017, 04:55:36 pm
content: Quote from: Gribo on June 26, 2016, 09:03:05 pmAltium 15 & 16 (and I'm sure 17) will read XLS excel files directly. (Not the newer XLSX). No need to set up your own ODBC source.Quote from: slicendice on December 31, 2016, 09:59:30 amThat has been my experience (which I outlined in a previous thread/blog post.) Also Altium will not easily access the newest version of Access databases. So if you buy Access 2016 do not expect it to work with Altium yet. To further complicate the matter, Access 2016 cannot create the old MDB access database format so you're going to waste many hours if you go this route.

author: slicendice
author_rank: Frequent Contributor
timestamp: January 03, 2017, 05:30:39 pm
content: As I stated before Access DB solution suck big time.

author: kripton2035
author_rank: Super Contributor
timestamp: January 03, 2017, 05:47:57 pm
content: excel is everything BUT a database. dont loose time using it like a database.

author: slicendice
author_rank: Frequent Contributor
timestamp: January 03, 2017, 07:07:05 pm
content: Quote from: kripton2035 on January 03, 2017, 05:47:57 pmYep! :-)

author: Randy_Clemmons
author_rank: Newbie
timestamp: January 05, 2017, 01:33:32 am
content: Myself and several users I personally know are using Access 2016 databases for DBLIB and SVNDBLIB libraries with Altium and we are having NO problems.The catch is you need to install 32bit versions of Access. If you read the fine print Microsoft recommends installing 32bit versions of the Microsoft Office applications.The 64bit versions of Microsoft Office have compatibly issues with 32bit apps, like Altium and many other 32bit apps installed on your PC.

author: slicendice
author_rank: Frequent Contributor
timestamp: January 05, 2017, 08:39:14 am
content: Hahhah, 64-bit Office has issues everywhere, while 32-bit works great. Only reason to use 64-bit office is when you need more addressable RAM. But there are much better solutions than Office in cases where RAM becomes an issue.

author: julianhigginson
author_rank: Frequent Contributor
timestamp: January 06, 2017, 04:20:01 am
content: I've used excel database to build a parts library a few times now.Actually it's very simple to manage, and nice to just have a file you can open up and investigate.And you don't even need excel. you can use libre office as long as you make sure to always save in xls format.(but full excel also lets you have BOM templates in AD... so maybe worth buying)Biggest issue with using an xls library isif you want to use it in AD and edit it in excel at the same time, you sometimes get file lock complaints.. then you have to close/reopen, which is frustrating, but easy to do, and pretty fast. Especially considering you don't have to do it that often.I don't understand the issue complaining about excel being not a real database... it's a 2D grid, that lets you have a basic set of parameters for each part, EXACTLY THE SAME as a relational database allows... but without having to design and manage a relational database. (and with the requirement to look after the file a little bit when you have multiple users accessing it... then again, with the proper database you have exactly the same problem to deal with for the actual footprint and schematic libraries.)I'm a bit dodgy on access. issues mentioned by people above, plus, well.. it looks like it's on it's last legs for the last 5 years, but there's always going to be excel spreadsheets.If I wanted to go past excel file parts db, I think I'd go for the SQL database. Have used that before, and liked it.. But for small scale use, it needs a VM on the local network to host at a minimum, or it needs an internet-accessible SQL server.... And you have to get a specific version of the database connector (altium won't work with the newest) But when you make it work, it works great.

author: kripton2035
author_rank: Super Contributor
timestamp: January 06, 2017, 07:21:17 am
content: Quote from: julianhigginson on January 06, 2017, 04:20:01 amrelational databases can implement constraint on the relations, and on the data type. excel can do that but it's a huge macro work.the main concern IMHO is that in excel you can put any data in a cell, and most of the time you don"t verify the type of it.if it's for your own usage, it's ok, but if others use your excel spreadsheet, then the mess begins, as they will type something that was not predicted in a cell and mess the rest of the sheet.theh when you will search for your datas, using a "Simple Query Language" searching for datas around multiples sheets, you will learn that you have written a lot of datas but that you will almost never be able to find the one you're searching for.but for that, you first need to have a lot of datas in your excel sheet, then only you will understand !

author: slicendice
author_rank: Frequent Contributor
timestamp: January 06, 2017, 08:46:13 am
content: If you want something slow, unreliable, confusing and complicated, difficult to manage, that runs out of memory sooner than you think and corrupts data often then go ahead with Excel as a database (I would NOT).If you want something that is easy to manage, is fast, is reliable, can be configured to be accessed from anywhere and is simple, then go with a real relational database.Excel is not designed to work as a database. It's designed to use a table of data (could also come from a database) and do a lot of calculations from that data and then generate a report view with new numbers and maybe some graphs.I've worked for a company that use Office (not just Excel, for everything), and boy did we have problems all the time.Don't get me wrong though, Excel is great tool and I love it, but using it as a database is out of the question for me. I'd rather do for a real database, whether it's MSSQL or SQLITE does not really matter. They all have their pros and cons.

author: julianhigginson
author_rank: Frequent Contributor
timestamp: January 07, 2017, 10:27:40 am
content: what is it with altium users in the industry being so much like superstitious old ladies? you really do hear the most ridiculous un-evaluated fear based garbage from altium users.I've already said I have used mySQL database for the parts database before, and don't mind it (but having to host the database somewhere is something to think about! and you still need to deal with the library files some other way!) and also - if you want a DB management app for adding and maintaining parts, and providing a nice handy search interface, well, you've got to build that and maintain that too( easy enough, but someone has to do it)compare to an excel sheet, that really is NOT big for a several hundred parts, and you can be up and running with for a parts database in like 10 minutes.yes if you have 100k parts to maintain and 345 users and a small team of dedicated librarians to maintain everything then OF COURSE a proper database is the way to go (or Altium vault.... hmm.)But I can tell you that an xls I looked after with 3 other users with around 800 parts (which depending on what you're designing and how you reuse parts can be many years worth of projects!) was only around 300k on disk and loaded fine in excel and worked fine on a basic computer. And having the database stored in folder alongside the lib files in a git repo was also a really handy way to keep track of changes all in one place... plus deal with any dramas that someone might cause - though nobody did.Any you know what?? even if you start with a 10 minute setup excel file parts database, which eventually gets outgrown as your company grows... it's trivial to export it into a new SQL database when you eventually decide you need to make and host one!The only parts database solution I'm dodgy on is Access. it's just so 90s and hokey, and, well, I keep expecting it to get EOL'd any year now.

author: slicendice
author_rank: Frequent Contributor
timestamp: January 07, 2017, 10:54:11 am
content: MS has tried to EOL Access for years, but too many companies still use it so MS had no choice but to keep supporting it. MSSQL is so much better and cheaper for MS to maintain in the long run (one app less to keep track of).A simple SQLite DB can be hosted anywhere, locally, remotely, you name it. Does not take up much space and is portable. Best of all it's free. There are a lot of DB management GUI tools out there that are free.Excel is easy to get started with a few hundred components, for that amount of data even a txt-file would be enough, but in the long run you will need that DB, whatever the DB format is does not really matter as long as you know the difference.I fully understand that users like to use Excel as their DB as it does not involve much or no programming knowledge to get started, while DB creation might need at least basic understanding of how one should talk to (program the) the DB.

author: julianhigginson
author_rank: Frequent Contributor
timestamp: January 07, 2017, 11:58:32 am
content: ooh.. I didn't realise I could have an on-disk SQL database running an altium parts database...... I have only seen hosted mySQL... And if there's some basic db management gui tools that will do the job I'd be very happy to know for next time I want an SQL parts database.What database solution and tools do you use for your SQL parts database?

author: slicendice
author_rank: Frequent Contributor
timestamp: January 07, 2017, 02:35:37 pm
content: You can have virtually any DBMS connected to Altium Designer using Microsoft libraries and it can be local, on a different computer on your network or remote on some host on the other side of the globe. Altium does not care nor know where the physical DB is located.I don't personally own Altium Designer so it is a bit difficult for me to help setting up the DB and the connection for it. Someone who actually owns the application and has this setup locally could be of more use to give a step by step guide to setting things up.For databases in general I usually use MSSQL and SQLITE, both locally and remotely. I use those for private version control for my programming and to store links to documents, design files and pictures or other information related to my current project I'm working on. Mostly it is software development related, not PCB design nor electronics components related, but the basic principle is still the same for any relational data one want to store.HERE'sa link to Altium that describes the process. It is a bit difficult to understand because they focus on so many different no related things at the same time that it becomes a bit confusing, even for an expert. But pretty much all required setup information for the Altium Designer part is there.EDIT: I now realized that the link I sent is the same as in the original post for this thread. Sorry for the duplicate.

author: mrtn
author_rank: Contributor
timestamp: January 08, 2017, 10:39:05 pm
content: Quote from: slicendice on January 07, 2017, 02:35:37 pmI've had plenty of experience with real databases as well. The truth is that Excel is an option that some people might want to use.I made static libraries that I don't intend to update. Seems completely appropriate to me to use an Excel data source. I used the least amount of effort to achieve the goal.

author: slicendice
author_rank: Frequent Contributor
timestamp: January 09, 2017, 07:45:30 am
content: Quote from: mrtn on January 08, 2017, 10:39:05 pmIn the case of small library, that will not get updated, then any DB source is great. If it's gonna get big, then switch to a real DB.

author: julianhigginson
author_rank: Frequent Contributor
timestamp: January 09, 2017, 10:20:44 am
content: Quote from: slicendice on January 09, 2017, 07:45:30 amMy point too, is, if excel works while it's small enough, then why not use something that suits your purpose upfront, then switch once you start to see a real need to spend the extra effort doing so?

author: Miles Teg
author_rank: Regular Contributor
timestamp: January 11, 2017, 09:00:45 am
content: Yeah!In 2015 I started building my altium "database" using Excel too. Was nice for a start.But then I needed to work with another colleague or trainee working on another Altium desktop.And pushed by needs of integrating internal purchase price of components we decide to go on real database.Needed also that we want to access it from outside the company, from my home or factories.I did a - I think - simple explanation of how build a DBLib with PostGreSQL, in answer of Robert Feranec tips video of having specific schematic symbol :The doc:https://docs.google.com/document/d/1-B35fDdS_LL7PVdWb8YbCH_7kQBTdq51whVv366jwJw/edit?usp=sharingNothing really new, but nice to avoid some annoying and time consuming mistakes

author: slicendice
author_rank: Frequent Contributor
timestamp: January 11, 2017, 11:16:14 am
content: Quote from: Miles Teg on January 11, 2017, 09:00:45 amAwesome post, thanks a lot, it will help others for sure.