Package: zowie
ZowieZowie ("Zotero linkwriter") is a command-line program for macOS that writes Zoteroselectlinks into the file attachments contained in a Zotero database.Table of contentsIntroductionInstallationUsageKnown issues and limitationsAdditional tipsGetting helpContributingLicenseAcknowledgmentsIntroductionWhen usingZotero, you may on occasion want to work with PDF files and other attachment files outside of Zotero.  For example, if you're aDEVONthinkuser, you will at some point discover the power of indexing your local Zotero database from DEVONthink.  However, when viewing or manipulating the attachments from outside of Zotero, you may run into the following problem: when looking at a given file,how do you find out which Zotero entry it belongs to?Enter Zowie (a loose acronym for"Zotero linkwriter", and pronounced likethe interjection).  Zowie scans through the files on your disk in a local Zotero database, looks up the Zotero bibliographic record corresponding to each file found, and writes aZotero select linkinto the file and/or certain macOS Finder/Spotlight metadata fields (depending on the user's choice).  A Zotero select link has the formzotero://select/...and when opened on macOS, causes the Zotero desktop application to open that item in your database.  Zowie thus makes it possible to go from a file opened in an application other than Zotero (e.g., DEVONthink, Adobe Acrobat), to the Zotero record corresponding to that file.Regretfully, Zowie canonlywork with Zotero libraries that use normal/local data storage;it cannot work when Zotero is configured to use linked attachments.InstallationThere are multiple ways of installing Zowie, ranging from downloading a self-contained, single-file, ready-to-run program, to installing it as a typical Python program usingpip.  Please choose the alternative that suits you and your Mac environment.Alternative 1: downloading the ready-to-run programOn macOS Catalina (10.15) or later, you can use a ready-to-run version of Zowie that only needs a Python interpreter version 3.8 or higher on your computer. That's the case for macOS 10.15 and later, but before you can use it, you may need to let macOS install some additional software components from Apple. To test it, run the following command in a terminal andtake note of the version of Pythonthat it prints:python3--versionIf this is the first timeyou've runpython3on your system, macOS will either ask you if you want to install certain additional software components, or it may produce an error aboutxcrun: error: invalid active developer path (/Library/Developer/CommandLineTools) .... In either case, the solution is to run the following command in the terminal:xcode-select --installIn the pop-up dialog box that this brings up,click theInstallbuttonand agree to let it install theCommand Line Toolspackage from Apple.Next,Go to thelatest release on GitHuband find theAssetsDownloadthe ZIP file whose name contains the version of Python on your computer (which you determined by runningpython3 --versionabove)Unzipthe file (if your browser didn't unzip it)Open the folderthat gets created (it will have a name likezowie-1.2.0-macos-python3.8)Look inside forzowieandmove itto a location where you put other command-line programs (such as/usr/local/bin).If you want to put it in/usr/local/binbut that folder does not exist on your computer yet, you can create it by opening a terminal window and running the following command (priorto movingzowieinto/usr/local/bin):sudomkdir/usr/local/binThe following is an example command that you can type in a terminal to move Zowie there:sudomvzowie/usr/local/binAlternative 2: installing Zowie usingpipxYou can usepipxto install Zowie. Pipx will install it into a separate Python environment that isolates the dependencies needed by Zowie from other Python programs on your system, and yet the resultingzowiecommand wil be executable from any shell – like any normal program on your computer. If you do not already havepipxon your system, it can be installed in a variety of easy ways and it is best to consultPipx's installation guidefor instructions. Once you have pipx on your system, you can install Zowie with the following command:pipxinstallzowiePipx can also let you run Zowie directly usingpipx run zowie, although in that case, you must always prefix every Zowie command withpipx run.  Consult thedocumentation forpipx runfor more information.Alternative 3: installing Zowie usingpipThe instructions below assume you have a Python 3 interpreter installed on your computer.  Note that the default on macOS at least through 10.14 (Mojave) is Python2– please first install Python version 3 and familiarize yourself with running Python programs on your system before proceeding further.You should be able to installzowiewithpipfor Python 3.  To installzowiefrom thePython package repository (PyPI), run the following command:python3-mpipinstallzowieAs an alternative to getting it fromPyPI, you can usepipto installzowiedirectly from GitHub:python3-mpipinstallgit+https://github.com/mhucka/zowie.gitIf you already installed Zowie once before, and want to update to the latest version, add--upgradeto the end of either command line above.Alternative 4: installing Zowie from sourcesIf  you prefer to install Zowie directly from the source code, you can do that too. To get a copy of the files, you can clone the GitHub repository:gitclonehttps://github.com/mhucka/zowieAlternatively, you can download the files as a ZIP archive using this link directly from your browser using this link:https://github.com/mhucka/zowie/archive/refs/heads/main.zipNext, after getting a copy of the files,  runsetup.pyinside the code directory:cdzowie
python3setup.pyinstallUsageFor help with usage at any time, runzowiewith the option-h.Thezowiecommand-line program should end up installed in a location where software is normally installed on your computer, if the installation steps described in the previous section proceeded successfully.  Running Zowie from a terminal shell then should be as simple as running any other shell command on your system:zowie-hIf you installed it as a Python package, then an alternative method is available to run Zowie from anywhere, namely to use the normal approach for running Python modules:python3-mzowie-hCredentials for Zotero accessZowie relies on theZotero sync APIto get information about your references.  This allows it to look up Zotero item URIs for files regardless of whether they belong to your personal library or shared libraries, and from there, construct the appropriate Zotero select link for the files.  If you do not already have aZotero sync account, it will be necessary to create one before going any further.To use Zowie, you will also need both an API user identifier (also known as theuserID) and anAPI key.  To find out your Zotero userID and create a new API key, log in to your Zotero account atZotero.organd visit theFeeds/APItab of the yourSettingspage.  On that page you can find your userID and create a new API key for Zowie.The first time you run Zowie, it will ask for this information and (unless the-Koption is given) store it in your macOS keychain so that it does not have to ask for it again on future occasions.  It is also possible to supply the identifier and API key directly on the command line using the-iand-aoptions, respectively; the given values will then override any values stored in the keychain and (unless the-Koption is also given) will be used to update the keychain for the next time.Basic usageZowie can operate on a folder, or one or more individual files, or a mix of both. Suppose your local Zotero database is located in~/Zotero/. Perhaps the simplest way to run Zowie is the following command:zowie~/ZoteroIf this is your first run of Zowie, it will ask you for your userID and API key, then search for files recursively under~/Zotero/.  For each file found, Zowie will contact the Zotero servers over the network and determine the Zotero select link for the bibliographic entry containing that file. Finally, it will use the default method of recording the link, which is to write it into the macOS Finder comments for the file.  It will also store your Zotero userID and API key into the system keychain so that it does not have to ask for them in the future.If you are a user ofDEVONthink, you will probably want to add the-soption (see theexplanation belowfor the details):zowie-s~/ZoteroInstead of a folder, you can also invoke Zowie on one or more individual files (but be careful to put quotes around pathnames with spaces in them, such as in this example):zowie-s"~/Zotero/storage/26GS7CZL/Smith 2020 Paper.pdf"Available methods of writing Zotero linksZowie supports multiple methods of writing the Zotero select link.  The option-lwill cause Zowie to print a list of all the methods available, then exit.The option-mcan be used to select one or more methods when running Zowie.  Write the method names separated with commas without spaces.  For example, the following command will make Zowie write the Zotero select link into the Finder comments as well as the PDF metadata attributeSubject:zowie-mfindercomment,pdfsubject~/Zotero/storageAt this time, the following methods are available:findercomment: (The default method.) Writes the Zotero select link into the Finder comments of each file, attempting to preserve other parts of the comments. If Zowie finds an existing Zotero select link in the text of the Finder comments attribute, it only updates the link portion and tries to leave the rest of the comment text untouched. Otherwise, Zowieonlywrites into the comments attribute if either the attribute value is empty or Zowie is given the overwrite (-o) option. (Note that updating the link text requires rewriting the entire Finder comments attribute on a given file. Finder comments have a reputation for being easy to get into inconsistent states, so if you have existing Finder comments that you absolutely don't want to lose, it may be safest to avoid this method.)pdfproducer: (Only applicable to PDF files.) Writes the Zotero select link into the "Producer" metadata field of each PDF file. If the "Producer" field is not empty on a given file, Zowie looks for an existing Zotero link within the value and updates the link if one is found; otherwise, Zowie leaves the field untouched unless given the overwrite flag (-o), in which case, it replaces the entire contents of the field with the Zotero select link.  For some users, the "Producer" field has not utility, and thus can be usefully hijacked for the purpose of storing the Zotero select link. The value is accessible from macOS Preview, Adobe Acrobat, DEVONthink, and presumably any other application that can display the PDF metadata fields.  However, note that some users (archivists, forensics investigators, possibly others) do use the "Producer" field, and overwriting it may be undesirable.pdfsubject: (Only applicable to PDF files.) Writes the Zotero select link into the "Subject" metadata field of each PDF file. If the "Subject" field is not empty on a given file, Zowie looks for an existing Zotero link within the value and updates the link if one is found; otherwise, Zowie leaves the field untouched unless given the overwrite flag (-o), in which case, it replaces the entire contents of the field with the Zotero select link.  Note that the PDF "Subject" field is not the same as the "Title" field. For some users, the "Subject" field is not used for any purpose and thus can be usefully hijacked for storing the Zotero select link. The value is accessible from macOS Preview, Adobe Acrobat, DEVONthink, and presumably any other application that can display the PDF metadata fields.wherefrom: Writes the Zotero select link to the "Where from" metadata field of each file (thecom.apple.metadata:kMDItemWhereFromsextended attribute). This field is displayed as "Where from" in Finder "Get Info" panels; it is typically used by web browsers to store a file's download origin. The field is a list. If Zowie finds a Zotero select link as the first item in the list, it updates that value; otherwise, Zowie prepends the Zotero select link to the list of existing values, keeping the other values unless the overwrite option (-o) is used. When the overwrite option is used, Zowie deletes the existing list of values and writes only the Zotero select link. Note that if macOS Spotlight indexing is turned on for the volume containing the file, the macOS Finder will display the updated "Where from" values in the Get Info panel of the file; if Spotlight is not turned on, the Get info panel will not be updated, but other applications will still be able to read the updated value.Note that, depending on the attribute, it is possible that a file has an attribute value that is not visible in the Finder or other applications.  This is especially true for "Where from" values and Finder comments.  The implication is that it may not be apparent when a file has a value for a given attribute, which can lead to confusion if Zowie thinks there is a value and refuses to change it without the-ooption.Filtering by file typeBy default, Zowie acts on all files it finds on the command line, except for certain files that it always ignores: hidden files and files with extensions.sqlite,.bak,.csl,.css,.js,.json,.pl, and a few others.  If the-moption is used to select methods that only apply to specific file types, Zowie will examine each file it finds in turn and only apply the methods that match that particular file's type, but it will still consider every file it finds in the directories it scans and apply the methods that are not limited to specific types.You can use the option-fto make Zowie filter the files it finds based on file name extensions.  This is useful if you want it to concentrate only on particular file types and ignore other files it might find while scanning folders. Here is an example (this also using the-soption forreasons given below):zowie-s-fpdf,mp4,mov~/Zoterowill cause it to only work on PDF, MP4, and QuickTime format files.  You can provide multiple file extensions separated by commas, without spaces and without the leading periods.Filtering by dateIf the-doption is given, the files will be filtered to use only those whose last-modified date/time stamp is no older than the given date/time description. Valid descriptors are those accepted by the Python dateparser library. Make sure to enclose descriptions within single or double quotes. Examples:zowie-d"2 weeks ago"....
zowie-d"2014-08-29"....
zowie-d"12 Dec 2014"....
zowie-d"July 4, 2013"....Special-case behaviorAlthough Zowie is not aimed solely at DEVONthink users, its development was motivated by the author's desire to use Zotero with that software.  A complication arose due to an undocumented feature in DEVONthink:it ignores a Finder comment if it is identical to the value of the "URL" attribute(which is the name it gives to thecom.apple.metadata:kMDItemWhereFromsattributediscussed above).  In practical terms, if you do something like write the Zotero select link into the Finder comment of a file and then have a DEVONthink smart rule copy the value to the URL field, the Finder comment will subsequentlyappear blank in DEVONthink(even though it exists on the actual file).  This can be unexpected and confusing, and has caught people (including the author of Zowie) unaware.  To compensate, Zowie 1.2 introduced a new option: it can add a trailing space character to the end of the value it writes into the Finder comment when using thefindercommentmethod.  Since approaches to copy the Zotero link from the Finder comment to the URL field in DEVONthink will typically strip whitespace around the URL value, the net effect is to make the value in the Finder comment just different enough from the URL field value to prevent DEVONthink from ignoring the Finder comment.  Use the option-sto  make Zowie to add the trailing space character.Additional command-line argumentsTo make Zowie only print what it would do without actually doing it, use the-n"dry run" option.If given the-qoption, Zowie will not print its usual informational messages while it is working. It will only print messages for warnings or errors.  By default, messages printed by Zowie are also color-coded. If given the option-C, Zowie will not color the text of messages it prints. (This latter option is useful when running Zowie within subshells inside other environments such as Emacs.)If given the-Voption, this program will print the version and other information, and exit without doing anything else.If given the-@argument, this program will output a detailed trace of what it is doing.  The debug trace will be sent to the given destination, which can be-to indicate console output, or a file path to send the output to a file.When-@has been given, Zowie also installs a signal handler on signalSIGUSR1that will drop Zowie into the pdb debugger if the signal is sent to the running process.Summary of command-line optionsThe following table summarizes all the command line options available.ShortLong form optMeaningDefault-aA--api-keyAAPI key to access the Zotero API service-C--no-colorDon't color-code the outputUse colors in the terminal-d--after-dateDOnly act on files modified after date "D"Act on all files found-f--file-extFOnly act on files with extensions in "F"Act on all files found⚑-h--helpDisplay help text and exit-i--identifierIZotero user ID for API calls-K--no-keyringDon't use a keyring/keychainStore login info in keyring-l--listDisplay known services and exit-m--methodMSelect how Zotero select links are writtenfindercomment-n--dry-runSay what would be done, but don't do itDo it-o--overwriteOverwrite previous metadata contentDon't write if already present-q--quietDon't print messages while workingBe chatty while working-s--spaceAppend trailing space to Finder commentsDon't add a space★-V--versionDisplay program version info and exit-@OUT--debugOUTDebugging mode; write trace toOUTNormal mode⬥⚑   Certain files are always ignored: hidden files, macOS aliases, and files with extensions.sqlite,.sqlite-journal,.bak,.csl,.css,.js,.json,.pl, and.config_resp.⬥   To write to the console, use the character-as the value ofOUT; otherwise,OUTmust be the name of a file where the output should be written.★   See the explanation in the section onspecial-case behavior.Return valuesThis program exits with a return code of 0 if no problems are encountered.  It returns a nonzero value otherwise. The following table lists the possible return values:CodeMeaning0success – program completed normally1the user interrupted the program's execution2encountered a bad or missing value for an option3no network detected – cannot proceed4file error – encountered a problem with a file5server error – encountered a problem with a server6an exception or fatal error occurredKnown issues and limitationsThe following is a list of currently-known issues and limitations:Zowie can only work when Zotero is set to use direct data storage; i.e., where attached files are stored in Zotero.  Itcannot work if you uselinked attachments, that is, if you setLinked Attachment Base Directoryin your Zotero Preferences'Advanced→Files and Folderspanel.If you useDEVONthinkin a scheme in which you index your Zotero folder and use Zowie to write the Zotero select link into the Finder comments of files, beware of the following situation. If you use a DEVONthink smart rule to copy the comment string into the "URL" field, DEVONthink will (after reindexing the file) suddenly display anemptyFinder comment, even though the comment is still there. This is due to adeliberate behavior in DEVONthinkand not a problem with Zowie, as discussed in thesection on special-case behavior. Using the-soption will avoid this, but at the cost of adding an extra character to the Finder comment, so make sure to account for the added space character in any scripts or other actions you take on the Finder comment.DEVONthinkbases the "URL" value of a file on the file'scom.apple.metadata:kMDItemWhereFromsextended attribute.  The original hope behind Zowie was to make it write Zotero select links directly into that attribute value. Unfortunately, it turns out that if a file has already been indexed by DEVONthink, thenit willnotdetect any changes to thecom.apple.metadata:kMDItemWhereFromsattributemade by an external program. Thus, if you index your Zotero folder within DEVONthink, you cannot use Zowie'swherefromsmethod to update the "URL" field directly. You are advised instead to use Zowie'sfindercommentmethod (the default) in combination with smart rules in DEVONthink, as discussed inthe wiki. I share your frustration.For reasons I have not had time to investigate, the binary version ofzowietakes a very long time to start up on macOS 10.15 (Catalina) and 11.1 (Big Sur).  On my test system inside a virtual machine running on a fast iMac, it takes 10 seconds or more before the first output fromzowieappears.Additional tipsIn thewiki associated with the Zowie project in GitHub, I have started writing some notes about how I personally use Zowie to combine Zotero with DEVONthink.Getting helpIf you find an issue, please submit it inthe GitHub issue trackerfor this repository.ContributingI would be happy to receive your help and participation if you are interested.  Everyone is asked to read and respect thecode of conductwhen participating in this project.  Development generally takes place on thedevelopmentbranch.LicenseThis software is Copyright (C) 2020–2023, by Michael Hucka and the California Institute of Technology (Pasadena, California, USA).  This software is freely distributed under a 3-clause BSD type license.  Please see theLICENSEfile for more information.AcknowledgmentsThis work is a personal project developed by the author, using computing facilities and other resources of theCalifornia Institute of Technology Library.Thevector artworkof an exclamation point circled by a zigzag, used as the icon for this repository, was created byAlfredo @ IconsAlfredo.comfrom the Noun Project.  It is licensed under the Creative CommonsCC-BY 3.0license.Zowie makes use of numerous open-source packages, without which Zowie could not have been developed.  I want to acknowledge this debt.  In alphabetical order, the packages are:aenum– advanced enumerations for Pythonbiplist– A binary plist parser/writer for Pythonboltons– package of miscellaneous Python utilitiesbun– a set of basic user interface classes and functionsCommonPy– a collection of commonly-useful Python functionsfastnumbers– number testing and conversion functionsipdb– the IPython debuggerkeyring– access the system keyring service from Pythonpdfrw– a pure Python library for reading and writing PDFsplac– a command line argument parserpy-applescript– a Python interface to AppleScriptPyInstaller– a packaging program that creates standalone applications from Python programspyobjc– Python ⇌ Objective-C and macOS frameworks bridgepyxattr– access extended file attributes from Pythonpyzotero– a Python API client for Zoterosetuptools– library forsetup.pyShiv– command-line utility for creating self-contained Python zipappsSidetrack– simple debug logging/tracing packagewheel– setuptools extension for building wheelsThedevelopers of DEVONthink, especially Jim Neumann and Christian Grunenberg, quickly and consistently replied to my many questions on theDEVONtechnologies forums.