MS Paint Fan Adventures

Stub to write down notes about project in progress
Archiving the contents of https://mspfa.com/ - uses a JS app to read text-based stories with embedded images and Flash content. Often link to YouTube alternates for flashes.
Custom archiver code is at https://github.com/riking/mspfa-archiver . Giant tangled mess of Go scripts.
Contact @riking to get upload permissions for the Archive collection
== Operating the custom archiver ==
=== Prep ===
# Clone into $GOPATH - there will probably be updates to the script
# Get wpull: pip3 install wpull==1.2.3; pip3 install -r https://github.com/chfoo/wpull/blob/v1.2.3/requirements.txt
# Symlink it into the current directory: ln -s $(which wpull)
# Get yt-dlp, yt-dlp -U as needed
# If needed, symlink ./target to a bigger drive or specify -o=folder every time you run it
# Build the code (TODO: patches to datatogether/warc) go build -v .
# Put IAS3 credentials into ./ias3.json - {"AccessKeyID": "...", "SecretAccessKey": "..."}
=== Running ===
While testing, make sure to include both -test -ident MSPFA_Test_12345 so that the uploaded Archive items go into test_collection.
When testing changes to the script, use -devScript to load the script from the local folder.
Basic usage: ./mspfa-archiver -dl -ident auto -s 1234
TODO - Script to automatically run on each story ID and save a list of failures
=== Recovering from Errors ==
If a download step fails (e.g. broken URL passed to photobucket step) run the archiver again with the -fu ("F"orce "U"pload) flag.
If you encounter a dead domain and don't want to wait for wpull, include -wpullArgs '--exclude-domains majhost.com,g0m.yore.ma' etc etc.
If the download works fine but the IA upload fails, run the script again with just -ident and no -dl flag. That will avoid modifying the WARC and just upload what's already downloaded.
== Work Division ==
=== 1-999 ===
Archiver: riking
Stage: Initial archiving
Problematic story IDs:
EDIT: using this google sheet instead for now https://docs.google.com/spreadsheets/d/1DQ52iRQ3w9M6hARTFhgzxTplqPjMfIMiZNxfBySa6Vw/edit#gid=0
(note: this table should not be taken as an example, it's translated from my bad notes)
Story ID	Archived OK? 	Problems
1 	 	Single failed Photobucket URL.
4 	 	Photobucket
12 	 	Photobucket
14 	 	404s
17 	 	Dead domain: forum-files2.fobby.net
19 	 	broken URL: pasted twice in a row
21 	, Contact Author	"410 Gone" from thefelt.webs.com; Dead domain: windowchronicles.com
22 	 	broken URL: pasted twice in a row
24 	 	Photobucket
25 	 	404s: imageshack, 22/22 rescued from Wayback
26 	 	404s: imageshack, -3(41)/44 rescued from Wayback
33 	 	Dead domain: yore.ma
35 	 	broken URL: single photobucket image ends in "89.pngp"
45 	 	Dead webhost customer: http://armada.lostsignalweb.com; Dead domain: imageplay.net; 404s: imageshack
46 	 	Dead domain: myfrogbag
48 	 	404s: imageshack, -10(105)/115 rescued from Wayback
53 	 	Dead domain: ardekantur.com
55 	 	404s: imageshack
59 	 	Photobucket
61 	 	404s
63 	 	404s
66 	 	Dead domain: myfrogbag
67 	 	404s: photobucket
76 	 	404s: photobucket
77 	 	404s: imageshack
78 	 	404s: imageshack
81 	 	Dead domain: TBD
87 	 	404s; Pulling HTML pages
89 	 	404s: imageshack
93 	 	Dead domain: TBD
104 	 	Dead domain: suspended webhost
106 	 	Dead domain: TBD
108 	 	404s: imagebin
109 	 	404s; Dead domain
111 	 	404s: imageshack
135 	 	404s: photobucket
158 	 	404s
160 	 	404s: imageshack
227 	 	uses blob: urls??
241 	 	Dropbox public folder
263 	 	HTML: imageshack homepage
270 	 	Dropbox public folder
277 	 	Dead domain:
285 	 	HTML
307 	 	404s
308 	 	404s
314 	 	Dead domain: blacktourney.com
319 	 	Dropbox public folder
323 	 	i.minus.com
325 	 	Dead domain: majhost.com
331 	 	Dropbox public folder
339 	 	A Single Photobucket BWE
341 	 	Dropbox public folder
350 	 	Dead domain: majhost.com
351 	 	Imageshack 404s
352 	 	SWFs hosted at files.myfrogbag.com
353 	 	SWFs hosted at files.myfrogbag.com
357 	 	Deleted imgur files
367 	 	Single photobucket BWE, retrieved from Wayback
728 	 	Dropbox public folder
=== 1000-1999 ===
=== 2000-2999 ===
=== 3000-3999 ===