Package: pimdb
pimdbPimdb is a python package and command line utility to maintain a local copy of
the essential parts of theInternet Movie Database(IMDb) based in the TSV files
available fromIMDb datasets.LicenseTheIMDb datasetsare only available for
personal and non-commercial use. For details refer to the previous link.Pimdb is open source and distributed under theBSD license. The source
code is available fromhttps://github.com/roskakori/pimdb.InstallationPimdb is available fromPyPIand can be
installed using:$pipinstallpimdbQuick startDownloading datasetsTo download the current IMDb datsets to the current folder, run:pimdbdownloadall(This downloads about 1 GB of data and might take a couple of minutes).Transferring datasets into tablesTo import them in a local SQLite databasepimdb.dblocated in the current
folder, run:pimdbtransferall(This will take a while. On a reasonably modern laptop with a local database
you can expect about 2 hours).The resulting database contains one tables for each dataset. The table names
are PascalCase variants of the dataset name. For example, the date from the
datasettitle.basicsare stored in the tableTitleBasics. The column names
in the table match the names from the datasets, for exampleTitleBasics.primaryTitle. A short description of all the datasets and
columns can be found at the download page for theIMDb datasets.Optionally you can specify a different database using the--databaseoption
with anSQLAlchemy engine configuration.Querying tablesTo query the tables, you can use any database tool that supports SQLite, for
example the freely available and platform independent community edition ofDBeaveror thecommand line shell for SQLite.For simple queries you can also usepimdband look at the result as
UTF-8 encoded TSV. For example, here are the details of the top 10 oldest
people alive according to IMDb:pimdbquery"select * from NameBasics where birthYear is not null and deathYear is null order by birthYear limit 10">oldest_people_alive.tsvYou can also run an SQL statement stored in a file:pimdbquery--filesome.sqlBuilding normalized tablesThe tables so far are almost verbatim copies of the IMDb datasets with the
exception that possible duplicate rows have been removed. This data model
already allows to perform several kinds of queries quite easily and
efficiently.However, the IMDb datasets do not offer a simple way to query N:M relations.
For example, the columnNameBasics.knownForTitlescontains a comma separated
list of tconsts like "tt2076794,tt0116514,tt0118577,tt0086491".To perform such queries efficiently you can build strictly normalized tables
derived from the dataset tables by running:pimdbbuildIf you did specify a--databasefor thetransfercommand before, you have to
specify the same value forbuildin order to find the source data. These tables
generally use snake_case names for both tables and columns, for exampletitle_allias.is_original.Querying normalized tablesN:M relations are stored in tables using the naming templatesome_to_other,
for examplename_to_known_for_title. These relation tables contain only the
numeric ID's to the respective actual data and a numeric columnorderingto
remember the sort order of the comma separated list in the IMDb dataset column.For example, here is an SQL query to list the titles Alan Smithee is known
for:selecttitle.primary_title,title.start_yearfromname_to_known_for_titlejoinnameonname.id=name_to_known_for_title.name_idjointitleontitle.id=name_to_known_for_title.title_idwherename.primary_name='Alan Smithee'For more information on which tables are available on how they are related
read the chapter about thepimdb data model.Where to go from herePimdb'sonline documentationdescribes all
aspects in further detail. You might find the following chapters of particular
interest:Usage: all command line
options explainedData model:
available tables and example SQL queriesContributing:
obtaining the source code and building the project locally