Package: ames
amesAutomated Metadata ServiceManage metadata from different sources.  The examples in the package are
specific to Caltech repositories, but could be generalized.  This package
is currently in development and will have additional sources and matchers
added over time.Basic InstallYou need to have Python 3.7 on your machine
(Minicondais a great
installation option).If you just need the python functions to write your own code
(like codemeta_to_datacite) open a terminal and typepip install amesFull InstallYou need to have Python 3.7 on your machine
(Minicondais a great
installation option).  Test whether you have python installed by opening a terminal or
anaconda prompt window and typingpython -V, which should print version 3.7
or greater.Clone amesIt's best to download this software using git.  To install git, typeconda install gitin your terminal or anaconda prompt window.  Then find where you
want the ames folder to live on your computer in File Explorer or Finder
(This could be the Desktop or Documents folder, for example).  Typecdin anaconda prompt or terminal and drag the location from the file browser into
the terminal window.  The path to the location
will show up, so your terminal will show a command likecd /Users/tmorrell/Desktop.  Hit enter.  Then typegit clone https://github.com/caltechlibrary/ames.git. Once you
hit enter you'll see an epxml_to_datacite folder.  Typecd amesInstallNow that you're in the ames folder, typepython setup.py install.  You can
now run all the different operations described below.UpdatingWhen there is a new version of the software, go to the ames
folder in anaconda prompt or terminal and typegit pull.  You shouldn't need to re-do
the installation steps unless there are major updates.OrganizationHarvesterscrossref_refs - Harvest references in datacite metadata from crossref event datacaltechdata - Harvest metadata from CaltechDATAcd_github - Harvest GitHub repos and codemeta files from CaltechDATAmatomo - Harvest web statistics from matomocaltechfeeds - Harvest Caltech Library metadata from feeds.library.caltech.eduMatcherscaltechdata - Match content in CaltechDATAupdate_datacite - Match content in DataCiteExample OperationsThe run scripts show examples of using ames to perform a specific update
operation.CodeMeta managementIn the test directory these is an example of using the codemeta_to_datacite
function to convert a codemeta file to DataCite standard metdataCodeMeta UpdatingCollect GitHub records in CaltechDATA, search for a codemeta.json file, and
update CaltechDATA with new metadata.CodeMeta SetupYou need to set an environmental variable with your token to access
CaltechDATAexport TINDTOK=CodeMeta UsageTypepython run_codemeta.py.CaltechDATA Citation AlertsHarvest citation data from the Crossref Event Data API, records in
CaltechDATA, match records, update metadata in CaltechDATA, and send email to
user.Citation Alerts SetupYou need to set environmental variables with your token to access
CaltechDATAexport TINDTOK=and Mailgunexport MAILTOK=.Citation Alerts UsageTypepython run_event_data.py. You'll be prompted for confirmation if any
new citations are found.Media UpdatesUpdate media records in DataCite that indicate the files associated with a DOI.Media SetupYou need to set an environmental variable with your password for your DataCite
account usingexport DATACITE=Media UsageTypepython run_media_update.py.CaltechDATA metadata checksThis will run checks on the quality of metadata in CaltechDATA.  Currently this
verifies whether redundent links are present in the related identifier section.It also can update metadata with DataCite.Metadata Checks SetupYou need to set environmental variables with your token to access
CaltechDATAexport TINDTOK=Metadata Checks UsageTypepython run_caltechdata_checks.py.CaltechDATA Metadata UpdatesThis will improve the quality of metadata in CaltechDATA.  This option is
broken up into updates that should run frequently (currently every 10 minutes)
and daily. Frequent updates include adding a recommended citation to the
descriptions, and daily updates include adding CaltechTHESIS DOIs to
CaltechDATA.Metadata Updates SetupYou need to set environmental variables with your token to access
CaltechDATAexport TINDTOK=Metadata Updates UsageTypepython run_caltechdata_updates.pyorpython run_caltechdata_daily.py.CaltechDATA COUNTER Usage ReportsThis will harvest download and view information from matomo and format it into
a COUNTER report.  This feature is still being tested.Usage Report SetupYou need to set environmental variables with your token to access
Matomoexport MATTOK=Usage Report UsageTypepython run_usage.py.Archives ReportsRuns reports on ArchivesSpace.  Current reports:accession_report: Returns accession records that match a certain subjectformat_report: Returns large report on accessions with certain media formatsExample usage:python run_archives_report.py accession_report accession.csv -subject "Manuscript Collection"Update EprintsPerform update options using the Eprints API. Supports url updates to https for
resolver field, special character updates, and adjusting the item modified date
(which also regenerates the public view of the page).Example usage:python run_eprints_updates.py update_date authors -recid 83420 -user tmorrell -passwordCODA ReportsRuns reports on Caltech Library repositories.  Current reports:doi_report: Records (optionally filtered by year) and their DOIs.thesis_report: Matches Eprints tsv export for CaltechTHESISthesis_metadata: Matches Eprints metadata tsv export for CaltechTHESIScreator_report: Finds records where an Eprints Creator ID has an ORCID
but it is not included on all records.  Also lists cases where an author has
two ORCIDS.creator_search: Export a google sheet with the author lists of all
publications associated with an author id. Requires -creator argumentpeople_search: Search across the CaltechPEOPLE collection by divisionfile_report: Records that have potential problems with the attached filesstatus_report: Reports on any records with an incorrect status in feedsrecord_number_report: Reports on records where the record number and resolver
URL don't matchalt_url_report: Reports on records with discontinure alt_url fieldlicense_report: Report out the license types in CaltechDATAReport UsageType something likepython run_coda_report.py doi_report thesis report.tsv -year 1977-1978The first option is the report typeNext is the repository (thesis or authors)Next is the output file name (include .csv or .tsv extension, will show up in current directory)Report OptionsSome reports include a -year option to return just the records from a specific year (1977) or a
range (1977-1978)Some reports include a -group option to return just the records with a
specific group name.  Surround long names with quotes (e.g. "Keck Institute for Space Studies")Some reports include a -item option to return just records with a
specific item type.  Supported types include:CaltechDATA item types (Dataset, Software, ...)CaltechAUTHORS item types (article, monograph, ...)CaltechAUTHORS monograph sub-typesdiscussion_paperdocumentationmanualotherproject_reportreporttechnical_reportwhite_paperworking_paperThere are some additional technical arguments if you want to change the default behavior.Adding-source eprintswill pull report data from Eprints instead of feeds.  This is
very slow.  You may need to add -username and -password to provide login
credentialsAdding-sample XXXallows you to select a number of randomly selected records.  This makes it
more reasonable to pull data directly from Eprints.You can combine multiple options to build more complex queries, such as this
request for reports from a group:python run_coda_report.py doi_report authors keck_tech_reports.csv -group "Keck Institute for Space Studies" -item technical_report project_report discussion_paperpython run_coda_report.py people_search people chem.csv -search "Chemistry and Chemical Engineering Division"