nyuuzyou/wikis huggingface dataset:
Epic War

Epic War® is a huge database hosting user-created maps for the game Warcraft III. Online since 2004 with over 300,000 user submissions. The site is currently not in danger and the hosting is sufficiently funded by ads on the website.
== Site structure ==
Each submission (map) gets a unique ID, the metadata page shows a preview, description (extracted from the uploaded file). Only the download links are session-based (generated links, expire after 24h), so a wget mirror will get them fine, but WBM not so much. Maps are hosted on a sub-domain and are only reachable from the metadata pages.
Some maps have been deleted as the website prohibits pornographic content etc.
Example items (https://www.epicwar.com/maps/<id>/):
* 9980 - exists
* 10000 - deleted
* 110014 - removed (likely a duplicate)
* 270006 - exists, no description
* 113113 - exists, extracted description has buggy line breaks
* 277617 - invalid id
== Status ==
Approx. 200k maps were mirrored and put up in groups of 10k on Bittorrent in 2016; I'm in the process of fetching the missing pieces, the site admin was pleasant. Once done, I will update this article with a link. I don't see it warranted (as it stands) to put up 400GB+ of this on IA - VADemon (talk) 13:26, 7 February 2020 (UTC)
Eroshare

== Closure Notice ==
Dear Erosharers,
It’s been a pleasure serving the amateur community these past few years.
The amount of growth and demand we’ve experienced has been phenomenal to say the least. Sadly, we are no longer able to offer this service.
Eroshare.com will shut down on June 30th, 2017. Albums will remain viewable until that day. Please download and backup your files.
We wish everyone the best in their passions and pursuits!
Regards,
Randy and Paul
Etherpad

Etherpad is a web-based collaborative real-time editor, allowing authors to simultaneously edit a text document, and see all of the participants' edits in real-time, with the ability to display each author's text in their own colour.
It was not until the end of 2009 when Google wanted to remove its opponents, and thus bought over Etherpad for the making of its own project called Google Wave (now already defunct). Under community pressure, Google open sourced the codebase behind Etherpad, thus sparking many new clones of Etherpad.
== Archives ==
* Archive Team's Etherpad Time Capsule
== External links ==
* http://etherpad.org
== ArchiveBot ==
Since Etherpad uses JavaScript, just saving the main URL of documents on Etherpad instances will not work correctly. Instead you should save the HTML, text and Etherpad exports:
!a http://pad.example.org/p/example/export/html
!ao http://pad.example.org/p/example/export/txt
!ao http://pad.example.org/p/example/export/etherpad
== Instances ==
Paste hosting#Etherpad has a list of Etherpad instances.
Category:Google
Europe

== Countries ==
* Albania
* Andorra
* Armenia
* Austria
* Azerbaijan
* Belarus
* Belgium
* Bosnia and Herzegovina
* Bulgaria
* Croatia
* Cyprus
* Czech Republic
* Denmark
* Estonia
* Finland
* France
* Georgia
* Germany
* Greece
* Hungary
* Iceland
* Ireland
* Italy
* Kazakhstan
* Latvia
* Liechtenstein
* Lithuania
* Luxembourg
* Malta
* Moldova
* Monaco
* Montenegro
* Netherlands
* Macedonia
* Norway
* Poland
* Portugal
* Romania
* Russia
* San Marino
* Serbia
* Slovakia
* Slovenia
* Spain
* Sweden
* Switzerland
* Turkey
* Ukraine
* United Kingdom
* Vatican City
=== Other territories ===
* Abkhazia
* Artsakh
* Faroe Islands
* Gibraltar
* Isle of Man
* Kosovo
* Northern Cyprus
* South Ossetia
* Transnistria
Evernote

Evernote is a note-taking and task-management application. Evernote users who may be fearful of sudden company decisions making their personal notebooks inaccessible should export their notebooks.
== Vital signs ==
* In June 2016, Evernote introduced a new limitation to just 2 devices on the free plan (previously unlimited) and increased the price of their paid plans. 
* In January 2023, Evernote was acquired by Bending Spoons.
* In December 2023, Evernote reduced its free plan drastically (to a maximum of 50 editable notes and 1 active device https://evernote.com/compare-plans), taking effect the same day. This change and the lack of transition period have been considered unfavourable to long-term free users who have used Evernote for years.
== Export ==
The simplest way to export an Evernote notebook is from Evernote desktop which, if doing so, should be set as the only active device. Simply right click and export notebook. This will result in a .enex file which can be opened in a text editor such as notepad or notepad++ and imported into Joplin.
== Footnotes ==
Everybody Votes Channel

According to Wikipedia:
:: “The Everybody Votes Channel was a Wii Menu channel that allowed users to vote in simple opinion polls and compare and contrast opinions with those of friends, family, [country], and voters across the globe. The Everybody Votes Channel was available on February 13, 2007. Its release came as a surprise, as Nintendo made no announcement regarding it until after it was available for download on the Wii Shop Channel. Like the four other Wii channels, Nintendo ended support for the Everybody Votes Channel on June 28, 2013.”
There is currently no known archive of the voting results history.
Everyplay

Everyplay was a video hosting service used in many games such as Geometry Dash as a replay system. It shut down October 1st, 2018, due to "inactivity", though it was most likely due to the constantly-expressed toxicity of the platform.https://www.urbandictionary.com/define.php?term=everyplay It had 38 million videos, according to its shutdown notice.
== References ==
Everything2

EVERYTHING2.COM is a kind of proto-wiki, dating from 1999. Never as popular as wikipedia, it still has about 2 million pages, many of which show up nowhere else in the internet.
Archived by bbot. The torrent of the first two million nodes is here, (magnet:?xt=urn:btih:3da079e5932acdacfdf183ff6de1698b6f1a24b7&dn=Everything2+%280-2M%29&tr=http%3A%2F%2Fdenis.stalker.h3q.com%3A6969%2Fannounce) and a blog post on the tedious technical details is here.
Since E2 is an operational site, additional backups will be made as more content is added.
Ex.ua

ex.ua was the 11th largest website in Ukraine. It allowed you to create user accounts then store files of almost any size under that account.
The notice on the front page of the website appeared on . Public access was disabled on , but the API has not been turned of yet. The website shut down on . http://www.ex.ua/
On , archiving efforts have been aborted, due to the site amounting to little more than petabytes' worth of "megapiracy"
== Announcement == 
The notice in Russian is taken from the front page and can be read on the left. The English translation by Google Translate is found on the right.
== Archiving ==
The XSPF files are still available from the website. These files contain URLs that 301 redirect to the actual file for download.
The XSPF URLs are formatted like ex.ua/filelist/[0-9]+.xspf. URLs inside the XSPF file are formatted like ex.ua/get/[0-9]+, which give a 301 redirect to the file for download. The Wayback Machine was allowed access to the file past the 301 redirect on /get/ URL and has archived quite some files from the website.
If a file has been deleted, the /get/ URL will redirect to a /view/ URL.
The Wayback machine has archived an unknown percentage of the user accounts and file lists.
=== Discovery ===
On a search engine, search with 'site:ex.ua'. This will yield many results for files. The links of these results might not be accessible, but the files can still be downloaded using the above method.
Note that the results will get be removed because they are not working anymore.
=== Warrior project ===
A Warrior project is being written for this website. All XSPF files and /get/ URLs without the file they redirect to will be archived. Only on request the file the /get/ URL redirects to will be archived, due to the total size of the data. Only /get/ URLs suggested to us will be archived through the warrior project.
The XSPF and /get/ URLs will be brute-forced.
== Still-working ==
For some reason http://www.ex.ua/user/username URLs still work, and return a list of a user's friends. This needs to be dumped so a friend graph can be recreated.
It has also been found that http://www.ex.ua/user_say?login=username will retrieve a list of things a user talked about - namely, the files they shared. This is the only known way to associate a given folder with the user who created that folder, and is thus a high priority.
== Links ==
* https://en.wikipedia.org/wiki/EX.UA
* Shutdown information: https://torrentfreak.com/major-cyberlocker-ex-ua-to-shut-down-amid-police-crackdown-161118/
== References ==
Exfm

exfm was a social music discovery site, which was acquired by Rhapsody on August 4, 2014. After that, everything was moved to archive.ex.fm.
== Site structure ==
Everything can be discovered through the sitemaps. The URLs in the sitemaps all point to ex.fm, which just redirects to the acquisition notice. However, changing ex.fm to archive.ex.fm will allow the content to be properly accessed from the public archive.
Exploring by the Seat of your Pants

Exploring by the Seat of your Pants is a YouTube channel that hosts people from around the world, talking about places around the world. Teachers can "register" classes for it, allowing them to have prioritised questions in a video call. Recently they have started using Slido instead of YouTube chat for questions from unregistered people, since YouTube chat was getting flooded by eight-year-olds, but Slido is somehow worse (probably because you don't need an account, unlike YouTube). Recent shows (as of mid-2021) have been about Grosse-Île Irish Memorial, Toronto Zoo, the Arctic, and more. It seems to be ONLY hosted on YouTube. A hugely educational place stored on YouTube with no known backup? Sounds OK! /s
User:TheTechRobo tried to contact them about whether they back their stuff up or not, but he got no answer.
FOIAonline

FOIAonline was a centralised platform for Freedom Of Information Act (FOIA) requests to several federal agencies in the United States. It was decommissioned on .
== Site structure and quirks ==
* There is no index of public entries. Iterating over receival dates in the works but misses some requests. The tested affected requests do not even show up when specifically searched for.
* The search only returns 10000 results for a particular query.
* Each request has an associated 'type' with three possible values (Request, Appeal, and Referral). It must be present in the URL, although it's value doesn't matter beyond some headings.
* Actual data retrieval happens through an internal/undocumented API by JavaScript. The code is inefficient and retrieves information multiple times with slight differences in the URL (the type being capitalised as above or lowercased for no obvious reason). Cookies, a CSRF token, and referrer checks are in place.
* Request IDs seem to consist of the relevant agency, sometimes some extra identifiers (e.g. a year), and a sequential six-digit number. There are large gaps in the latter as returned by the receival date search, but at least the vast majority of those 'missing' requests appear to be inaccessible (403 on the internal API request).
* Both the search and each request's file lists (attachments and records) use POST requests, i.e. they can't work correctly in the Wayback Machine.
* API weirdness includes:
** Attachment endpoint has pagination but it isn't used.
** File sizes are returned in MiB as either a float or a string, depending on the endpoint.
** Attachments are only listed when a flag on the request info is set, but that flat doesn't actually mean there are attachments.
** The search for receival dates returns some duplicates – sometimes even on different dates!
** On requests with a large number of records, the API returns recordsTotal: 99999 until you get near the end of the pagination.
** The file list pages for some of the largest requests are very slow to retrieve, especially on the higher page numbers.
* The site employs an aggressive Web Application Firewall that blocks the downloads of some files whose names contain characters like a semicolon. This can be bypassed by replacing the filename; the file is identified by a UUID, and the filename appears to only be used for the Content-Disposition header. This bypass can't be applied manually in a browser's URL bar since the Referer header must be present and correct.
== References ==
FTP

The File Transfer Protocol, FTP, is a protocol for file transfer published as RFC 114 on 16 April 1971. In the older days of the internet the protocol was frequently used to upload and share files. Today the protocol is not used so much anymore. This made Archive Team decide to grab all the FTP servers.
The FTP grab started 30 November 2015.
== How can I help? ==
=== Running the script manually ===
If you use Linux and you're a bit familiar with it, you can try running the script directly.
The instructions can be found at https://github.com/ArchiveTeam/ftp-grab.
Some additional information
Don't forget to replace YOURNICKHERE with your nickname.
The number after --concurrent determines how many threads run at the same time. You can increase this number if your resources (RAM, CPU, bandwidth) are sufficient. However, if you constantly see messages about rate limiting, there is no need to increase the concurrency.
If you want to stop the script, please do it gracefully if possible. To do so, create an empty file named STOP in the folder of the script (terminal command: touch STOP). The script finishes the current item(s) and stops only after that. (If you kill the script immediately, the items get broken, and they will need to be reassigned to another user.) – Before starting the script again, don't forget to remove the STOP file.
If you see "Project code is out of date", kill the script, go to its folder (cd ftp-grab) and issue git pull https://github.com/ArchiveTeam/ftp-grab. After the updating has finished, re-launch the script.
=== Discovery items ===
The project needs to have items to be able to run. You can help discovering these items.
Scripts for creating items for the grab can be found at https://github.com/ArchiveTeam/ftp-queue. Instructions on how to run the grab can be found in the README. A list of FTPs that need to be scanned can be found at FTP/List.
Some additional information
User:Squidboy says:
It's worth noting that as of June 2019 ftp-queue has several issues that may make it hard to use.
=== Donating to the Internet Archive ===
Content downloaded by the ArchiveTeam will be uploaded to the Internet Archive, where it will be stored and be available – hopefully – forever. However, storing it costs thousands of dollars in the long run. So, if you can afford, please consider donating to the Internet Archive, so that this piece of history can be kept for us all. https://archive.org/donate/
== External Links ==
* Anonymous FTP Sites List
* @textfiles Talked it over with a few people. We decided to download all the FTP sites. All. of. Them. Smile for your photograph, FTP.
* Google uses its browser market dominance to speed up the demise of FTP
Category:Web applications
Facepunch Forums

Facepunch Forums were forums run by game development studio Facepunch. As of April 6, 2019, the forums claimed "433,765 registered users have made 32,593,834 posts in 757,000 threads". The forums' owners had previously signaled a potential shutdown in late 2018 in a . On 11 June 2019, the forums briefly went offline, redirecting to the replacement community knockout.chat instead. This redirect was removed again after an hour or two, but the forums shut down permanently on 12 June at about 19:05 UTC.
== Archival ==
* A virtually complete retrieval by User:JustAnotherArchivist using qwarc on 12 June 2019
* A number of ArchiveBot jobs in early 2015, mid 2016, early April 2019 and in June 2019
FamilySearch

FamilySearch is a free genealogy organization and website run by The Church of Jesus Christ of Latter-day Saints. It provides access to various genealogy records and allows users to view and contribute to a community built tree. Free access to various paid services is available at FamilySearch Family History Centers.https://www.familysearch.org/wiki/en/Family_History_Center_Portal
Partially due to it being a single community built tree, inaccurate, duplicate, and incorrectly formatted information is common.
Users have the ability to add URLs as external sources, however this is subject to link rot.
== References ==
Famous Internet videos

This page contains a list of famous Internet videos which should be archived.
*Me at the zoo, the first YouTube video ever
*Charlie bit my finger
*Potter Puppet Pals: The Mysterious Ticking Noise
*The Sneezing Baby Panda
*The Annoying Orange
*Star Wars Kid
*Leeroy Jenkins
*Fenton the Dog
*Cat mom hugs baby kitten
*Greatest freak out ever
*Numa Numa
*Dramatic Chipmunk
*Chocolate Rain
*David After Dentist
*Leave Britney Alone
*RickRoll'D
*Angry German Kid
*Husky Dog Talking - "I love you"
Category:Video hosting
FanFiction.Net

FanFiction.Net is the self-proclaimed "world's largest fanfiction archive and forum" and is one of if not the largest site hosting fanfiction. Has a sister site Fictionpress, much smaller, identical layout.
As of October 2018, there seems to be a lack of maintenance: security issues are left unpatched, the SSL certificates are expiring in early November, and the admin hasn't been active since late 2017. The site is therefore considered to be in danger of disappearing.
As of 2022 the Internet Archive's crawlers do not seem to have full access/or are blocked by Cloudflare, especially newer stories or non-default views of listings, at least with auto-save.
The site does not seem to care what is in the slug of submission URLs, and different processes seem to have generated URLs that use either underscores or dashes for spaces, so that e.g. fanfiction.net/s/6101020/1/How-To-Write-IZ-Fanfiction-in-6-Steps and fanfiction.net/s/6101020/1/How_To_Write_IZ_Fanfiction_in_6_Steps are "different" in the WBM.
== Archival ==
Most of the work is just staying on top of the new stories and user profiles.
The Archiveteam's 2012 scrape of the site can be found in the archiveteam_fanfiction collection (in 19 parts, totaling about 830 GB).
A nearly complete scrape of all stories on the site, in text-only format, was created independently of the Archiveteam project in September 2015 by bsmith093 on EFNet. It was later repacked, and is now available on IA: . (The original version was uploaded under the ID and accidentally duplicated by a different uploader under the ID ).
Fast.io

Fast.io is a CDN provider that acts as a distributor of data stored on Google Drive, GitHub, Dropbox, Box, OneDrive, or MediaFire.
On or around 2020-10-13, Fast.io announced that free accounts would be discontinued on 2020-12-31 in an email sent to its users.
On or around 2020-11-20, Fast.io announced that their entire service would shut down on 2021-01-15. The following notice was posted to their login page:
 
This is a difficult announcement to make. Beginning today we're winding down the Fast.io service and we will fully retire it on January 15th, 2021. We're proud that we've been able to help many of you host your content, but we need to focus our resources elsewhere.
Fast.io was a promising tool for simplifying the CDN deployment process. It has developed a loyal audience and showed real signs of value throughout its life. However, we have not been able to reach a level of growth or revenue that would lead to a viable business. Instead, we will be investing in a new direction.
After January 15th, 2021, Fast.io services will cease to operate. Your content will remain in your cloud storage however your sites will no longer be publicly accessible. After this date, you will also no longer be able to access your Fast.io account. Please ensure that you have retrieved any invoices or other data from your account before then.
We understand that this won't be an easy transition and we're working hard to make this process as easy as possible. We are committed to maintaining our services during this period and our team will focus on support to resolve any issues and assist with your migration to other platforms. Please contact us through intercom or email (help@fast.io) if you need assistance.
We are grateful you chose Fast.io as your hosting partner and thank you for taking this ride with us.
Only small bits and pieces were able to be grabbed by ArchiveBot before the service fully shut down on January 16th.
== References ==
Fediverse

The Fediverse is a network of interoperating servers used for web publishing. Due to the federation, instances are constantly joining the network or disappearing.
The main protocol used in the fediverse is ActivityPub. Many software platforms exist; the most widely used ones as of early 2019 are Mastodon, Pleroma, PeerTube, Friendica, GNU social, and WriteFreelyhttps://web.archive.org/web/20190312111429/https://fediverse.network/.
== ArchiveTeam and the Fediverse ==
In May 2019, AT (or at any rate, what was seen from the outside as AT-as-a-unitary-actor) archived a Mastodon server called , which caused blowback. Subsequently AT adopted a policy of not archiving Fediverse instances unless we get permission from the owners. This held until AT's de facto current leadership signaled that it is no longer a thing in 2023, but since then we haven't archived any Fediverse instances anyway due to its (i.e. Mastodon instances') heavy use of Javascript - and the expected post-buyout exodus from Twitter not really happening, making this less of a priority than we might've thought.
== JavaScript Workarounds ==
Most Fediverse instances use Mastodon, which requires JavaScript, which calls Mastodon APIs to retrieve data.
zygolophodon is a command-line client for the Mastodon APIs, and a patchset for it is aiming to add an option to output URLs for archiving, including JavaScript based pages, the API requests they make and RSS URLs as well as outlink URLs. Not production ready yet and only intended for targeted archiving.
The ThreadTree service is a thread "unroller" for Mastodon URLs that outputs plain HTML with no JavaScript requirement. Currently it has some issues that make it suboptimal for archiving, but it can still work OKish.
== References ==
Fetlife

Fetlife bills it's self as the #1 walled garden social network for people in the BDSM/kink/fetish community.
Their security sucks, and the site is easily crawlable and it's content somewhat archiveable.
== Grabbing someone elses images == 
Use fetscrape it's easy to set up, but is currently being worked on.
== Grabbing someone elses videos ==
Theres a site that does this and you don't need to pay to get access.
== Exporting your own Fetlife ==
MayMay has done a ton of great work with regard to FL, but there's others building on what maymay did
Maymays tool
Ffffound!

Ffffound! was an image bookmarking site that closed on May 15th, 2017.
==Tools for capturing the site==
NOTE: both ffffind.py and ffffound-export are written in Python 2, so make sure you have at least the Python 2 version of BeautifulSoup.
* ffffind.py: a Python script for getting all the images posted by a given user
* ffffound-export: a fork of ffffind.py which will create new html pages with which to view the images (we probably want to use this)
* BeautifulSoup: a Python library for extraction data from the web, required for ffffind.py or ffffound-export