Package: archivenow
A Tool To Push Web Resources Into Web ArchivesArchive Now (archivenow) currently is configured to push resources into six public web archives. You can easily add more archives by writing a new archive handler (e.g., myarchive_handler.py) and place it inside the folder “handlers”.As explained below, this library can be used through:Command Line Interface (CLI)A Web ServiceA Docker ContainerPythonInstallingThe latest release ofarchivenowcan be installed using pip:$pipinstallarchivenowThe latest development version containing changes not yet released can be installed from source:$gitclonegit@github.com:oduwsdl/archivenow.git$cdarchivenow$pipinstall-rrequirements.txt$pipinstall./CLI USAGEUsage of sub-commands inarchivenowcan be accessed through providing the-hor–helpflag, like any of the below.$archivenow-husage:archivenow.py[-h][--mg][--wc][--cc][--cc_api_key[CC_API_KEY]][--is][--st][--ia][--warc[WARC]][-v][--all][--server][--host[HOST]][--agent[AGENT]][--port[PORT]][URI]positionalarguments:URIURIofawebresourceoptionalarguments:-h,--helpshowthishelpmessageandexit--mgUseMegalodon.jp--wcUseTheWebCiteArchive--ccUseThePerma.ccArchive--cc_api_key[CC_API_KEY]AnAPIKEYisrequiredbyThePerma.ccArchive--isUseTheArchive.is--stUseTheArchive.st--iaUseTheInternetArchive--warc[WARC]GenerateWARCfile-v,--versionReporttheversionofarchivenow--allUseallpossiblearchives--serverRunarchiveNowasaWebService--host[HOST]Aserveraddress--agent[AGENT]Use"wget"or"squidwarc"forWARCgeneration--port[PORT]AportnumbertorunaWebServiceExamplesExample 1To save the web page (www.foxnews.com) in the Internet Archive:$archivenow--iawww.foxnews.comhttps://web.archive.org/web/20170209135625/http://www.foxnews.comExample 2By default, the web page (e.g., www.foxnews.com) will be saved in the Internet Archive if no optional arguments are provided:$archivenowwww.foxnews.comhttps://web.archive.org/web/20170215164835/http://www.foxnews.comExample 3To save the web page (www.foxnews.com) in the Internet Archive (archive.org) and Archive.is:$archivenow--ia--iswww.foxnews.comhttps://web.archive.org/web/20170209140345/http://www.foxnews.comhttp://archive.is/fPVycExample 4To save the web page (https://nypost.com/) in all configured web archives. In addition to preserving the page in all configured archives, this command will also locally create a WARC file:$archivenow--allhttps://nypost.com/--cc_api_key$Your-Perma-CC-API-Keyhttp://archive.is/dcnanhttps://perma.cc/53CC-5ST8https://web.archive.org/web/20181002081445/https://nypost.com/https://megalodon.jp/2018-1002-1714-24/https://nypost.com:443/http://www.webcitation.org/72ramyxT2https://Archive.st/archive/2018/10/nypost.com/h5m1/nypost.com/index.htmlhttps_nypost.com__96ec2300.warcExample 5To download the web page (https://nypost.com/) and create a WARC file:$archivenow--warc=mypage--agent=wgethttps://nypost.com/mypage.warcServerYou can runarchivenowas a web service. You can specify the server address and/or the port number (e.g., –host localhost  –port 12345)$archivenow--serverRunningonhttp://0.0.0.0:12345/(PressCTRL+Ctoquit)Example 6To save the web page (www.foxnews.com) in The Internet Archive through the web service:$curl-ihttp://0.0.0.0:12345/ia/www.foxnews.comHTTP/1.0200OKContent-Type:application/jsonContent-Length:95Server:Werkzeug/0.11.15Python/2.7.10Date:Tue,02Oct201808:20:18GMT{"results":["https://web.archive.org/web/20181002082007/http://www.foxnews.com"]}Example 7To save the web page (www.foxnews.com) in all configured archives though the web service:$curl-ihttp://0.0.0.0:12345/all/www.foxnews.comHTTP/1.0200OKContent-Type:application/jsonContent-Length:385Server:Werkzeug/0.11.15Python/2.7.10Date:Tue,02Oct201808:23:53GMT{"results":["Error (The Perma.cc Archive): An API Key is required ","http://archive.is/ukads","https://web.archive.org/web/20181002082007/http://www.foxnews.com","http://Archive.st/ikxq","Error (Megalodon.jp): We can not obtain this page because the time limit has been reached or for technical ... ","http://www.webcitation.org/72rbKsX8B"]}Example 8Because an API Key is required by Perma.cc, the HTTP request should be as follows:$curl-ihttp://127.0.0.1:12345/all/https://nypost.com/?cc_api_key=$Your-Perma-CC-API-KeyOr use only Perma.cc:$curl-ihttp://127.0.0.1:12345/cc/https://nypost.com/?cc_api_key=$Your-Perma-CC-API-KeyRunning as a Docker Container$dockerimagepulloduwsdl/archivenowDifferent ways to run archivenow$dockercontainerrun-it--rmoduwsdl/archivenow-hAccessible at 127.0.0.1:12345:$dockercontainerrun-p12345:12345-it--rmoduwsdl/archivenow--server--host0.0.0.0Accessible at 127.0.0.1:22222:$dockercontainerrun-p22222:11111-it--rmoduwsdl/archivenow--server--port11111--host0.0.0.0To save the web page (http://www.cnn.com) in The Internet Archive$dockercontainerrun-it--rmoduwsdl/archivenow--iahttp://www.cnn.comPython Usage>>>fromarchivenowimportarchivenowExample 9To save the web page (www.foxnews.com) in The WebCite Archive:>>>archivenow.push("www.foxnews.com","wc")['http://www.webcitation.org/6o9LTiDz3']Example 10To save the web page (www.foxnews.com) in all configured archives:>>>archivenow.push("www.foxnews.com","all")['https://web.archive.org/web/20170209145930/http://www.foxnews.com','http://archive.is/oAjuM','http://www.webcitation.org/6o9LcQoVV','Error(ThePerma.ccArchive):AnAPIKEYisrequired]Example 11To save the web page (www.foxnews.com) in The Perma.cc:>>>archivenow.push("www.foxnews.com","cc",{"cc_api_key":"$YOUR-Perma-cc-API-KEY"})['https://perma.cc/8YYC-C7RM']Example 12To start the server from Python do the following. The server/port number can be passed (e.g, start(port=1111, host=’localhost’)):>>>archivenow.start()2017-02-0915:02:37Runningonhttp://127.0.0.1:12345(PressCTRL+Ctoquit)Configuring a new archive or removing existing oneAdditional archives may be added by creating a handler file in the “handlers” directory.For example, if I want to add a new archive named “My Archive”, I would create a file “ma_handler.py” and store it in the folder “handlers”. The “ma” will be the archive identifier, so to push a web page (e.g., www.cnn.com) to this archive through the Python code, I should write:archivenow.push("www.cnn.com","ma")In the file “ma_handler.py”, the name of the class must be “MA_handler”. This class must have at least one function called “push” which has one argument. See the existinghandler filesfor examples on how to organized a newly configured archive handler.Removing an archive can be done by one of the following options:Removing the archive handler file from the folder “handlers”Renaming the archive handler file to other name that does not end with “_handler.py”Setting the variable “enabled” to “False” inside the handler fileNotesThe Internet Archive (IA) sets a time gap of at least two minutes between creating different copies of the “same” resource.For example, if you send a request to IA to capture (www.cnn.com) at 10:00pm, IA will create a new copy (C) of this URI. IA will then returnCfor all requests to the archive for this URI received until 10:02pm. Using this same submission procedure for Archive.is requires a time gap of five minutes.Citing Project@INPROCEEDINGS{archivenow-jcdl2018,
  AUTHOR    ={Mohamed Aturban and
               Mat Kelly and
               Sawood Alam and
               John A. Berlin and
               Michael L. Nelson and
               Michele C. Weigle},
  TITLE     ={{ArchiveNow}: Simplified, Extensible, Multi-Archive Preservation},
  BOOKTITLE ={Proceedings of the 18th{ACM/IEEE-CS}Joint Conference on Digital Libraries},
  SERIES    ={{JCDL}'18},
  PAGES     ={321--322},
  MONTH     ={June},
  YEAR      ={2018},
  ADDRESS   ={Fort Worth, Texas, USA},
  URL       ={https://doi.org/10.1145/3197026.3203880},
  DOI       ={10.1145/3197026.3203880}}