nyuuzyou/wikis huggingface dataset:
Formats

A very good rule of thumb with data formats is to pick those that are no more complex than the data being represented, that are recoverable with simple tools and widely implemented.
In general, if you have written a text document and it's not viewable or editable in a low-level text editor (Notepad, Emacs, and so on), you should probably take the time to convert it into a plain-text format - keep the rich format also.
If you are backing up data in a format that's not widely understood, be sure to also keep backups of the software you use to open it and any registration keys. A file made with version 2.x of a piece of software may not open with the all new, singing and dancing version 5.x!
Tip: the Archive Team subdomain http://fileformats.archiveteam.org/ hosts a wiki dedicated to storing information about file formats.
== Text ==
Plain text, HTML and non-bloated XML formats are all good bets (DocBook, TEI etc.).
=== Markdown ===
[Markdown] is a human-readable, popular markup language. Even if editors somehow disappear, it is still very human-readable, and so you should still be able to get the gist of it even if there are no tools to open it.
=== PDF ===
The Portable Document Format standard created by Adobe has reached a point where it should be readable for posterity. The format is now open enough that it should be usable for backup for the foreseeable future; PDF/A is specifically designed for digital preservation. You can get a PDF reader here.
=== TeX ===
The TeX standard has been around since 1969. TeX documents are text based. It is widely used to prepare multi-thousand page documents for publication, as well as mathematical formulas. LaTeX is a free, open document preparation system based on this standard.
== Images ==
* PNG
* Lossless TIFF
* SVG
== Audio ==
Lossless:
* FLAC
Lossy:
* OGG
== Video ==
* Matroska
* OGV
* AVI
== Compression ==
* 7z
* gzip
* TAR
* ZIP
== Website crawls ==
WARC is required for Wayback Machine integration and is highly recommended. It retains important metadata (such as request/response headers) that would otherwise be lost.
== External links ==
* http://en.wikipedia.org/wiki/Category:Open_formats
* http://fileformats.archiveteam.org/ Let's Solve the File Format Problem!
Foro 3DJuegos

Foro 3DJuegos is a Spanish videogames forum that has been running since 2005.
In 2025-01-29 they announced the forum would be shut down at the end of March.
== Technical details ==
The forum appears to be custom software.
Some links are obfuscated with Javascript, which prevents a straightforward Archivebot job.
For example, within a forum category, some thread links are regular links:
 
 <a href="https://www.foro3djuegos.com/foros/tema/54322871/0/no-encuentro-este-juego-de-psp-resuelto/" class="url col_hover">No encuentro este juego de PSP [RESUELTO]</a>
 
while others appear like this:
 
 Sobre el foro Juegos en general [Leer]
 
Javascript code then replaces this with a proper link. The algorithm is:
* Get elements with class uAr06vA
* Find the classname that matches /[0-9A-F]+/
* Map characters 0A12B34C56D78E9F to 0123456789ABCDEF
* Decode the result as hexadecimal
For example 45CBCBC0C22D1F1F43C54A4EC0484319424F4E1F becomes https://example.com/.
Forrst

Forrst was a community where designers and developers share their work and leave feedback on other users' work. Forrst was bought by ZURB in January 2013.
Forrst was shut down on April 14th, 2014, but all posts were archived by Forrst. The site vanished sometime in 2017.
==External links==
The Forrst is Growing Into Something New (blogpost about a big site restructure)
ZURB Acquires Forrst!
Fortress of Solitude

Fortress of Solitude, often abbreviated as FOS, was ArchiveTeam's primary staging server, operated by Jason Scott (a.k.a. SketchCow). It is, in fact, a virtual machine on the network of the Internet Archive, and had a storage capacity of 16 terabytes as of 2015.
Content downloaded as part of ArchiveTeam projects (including ArchiveBot) is uploaded to so-called staging servers via rsync, and after being processed, data is transferred from there to the servers of the Internet Archive.
As of December 2022, it is used very little (if at all). Warrior projects don't use it anymore, and ArchiveBot was updated not to use it in early 2022IRC logs from 2022-12-19More IRC logs from 2022-12-19. Additionally, on 2022-12-15, rsync was deactivated.
Warrior projects now use several different staging servers, which change often based on project size, urgency, and the total throughput needed to the Internet Archive. The remaining Archive Team data on the server was cleaned up around .
== Links ==
*Upload history: Shows when and where the megawarc files (multi-gigabyte compilations of smaller WARC files) of ArchiveBot and ArchiveTeam projects have been uploaded.
*Pipeline monitor: Shows status of pipelines (processes for megawarcing and uploading) of different projects.
*System status: Some resource graphs of the FOS VM instance.
Category:ArchiveTeam Glossary
FortuneCity

thumb | Shutdown notice
== How to help ==
To run one or more FortuneCity downloaders, you'll need to be on Linux or a Linux-like OS.
Setting up:
git clone git://github.com/ArchiveTeam/fortunecity.git
cd fortunecity
./get-wget-warc.sh
Check the output: does it say wget is successfully compiled? Great!
Now you can run a download client. Choose a nickname and run:
./seesaw.sh YOURNICK
The script should start downloading and uploading. If it works, feel free to run a few more!
Please do not run more than 10 downloaders. It won't work. We need many individuals making small contributions.
If you want to stop, you can just kill the scripts. To stop gracefully,
touch STOP
and the script will stop after the current user.
There is no need to run upload-finished.sh. seesaw.sh will automatically upload your finished users to us.
== Common Problems ==
=== configure: error: --with-ssl was given, but GNUTLS is not available. ===
You don't have the proper headers available to be able to compile wget-warc. It should be fairly easy to fix though. If you're using a Debian or Ubuntu-based Linux distribution:
 apt-get install libgnutls-dev
If you're using a Fedora distribution:
 yum install gnutls-devel
If you're using FreeBSD :
 cd /usr/ports/security/gnutls/
 make all install clean
If you're using something else you'll just have to poke around in the documentation and figure it out :)
== Status ==
Archiving is well under way. Check the tracker for an up to date status report.
== Site structure ==
FortuneCities operated in multiple tlds: com co.uk es it se
(Once operated but died: de fr nl cn.fortunecity.com)
Main website:
 http://www.fortunecity.${tld}/
Username-based sites:
 http://members.fortunecity.${tld}/
 http://${username}.fortunecity.${tld}/${username}/
Area/street-based sites:
 http://www.fortunecity.${tld}/${area}/${street}/${number}/
 on .com also: http://${area}.fortunecity.com/${street}/${number}/
Range of numbers: unsure. At least includes 0 up to 2600. Note: the street names are case-sensitive.
=== Categories, areas, streets ===
Before 2001, FortuneCity used a category-area-street-based system.
The categories were the same for all tlds:
artsandhumanities
businessandcareers
computersandinternet
entertainment
homeandfamily
international
peopleandchat
recreationandsports
scifiandparanormal
travelandtransport
(This data is on https://github.com/ArchiveTeam/fortunecity/tree/master/explore)
The areas for each category can be found via Wayback (1999 or 2000): http://www.fortunecity.${tld}/explore/category/${category}.html
The streets for each area, too.
For com in 1999: http://${area}.fortunecity.com/
For others in 2000: http://www.fortunecity.${tld}/explore/area/${area}.html
The streets for the following .com areas couldn't be found via Wayback:
challenge cratervalley lavender littleitaly marina olympia skyscraper tatooine victorian
Category:Web hosting
Forums.deviliangame.com

Devilian is a MMORPG by Bluehole Ginno, published by Trion Worlds Inc.
The game was announced to be shutdown on March 5th, 2018.https://web.archive.org/web/20180122071942/https://www.trionworlds.com/devilian/en/2018/01/19/devilian-sunset-march-5th/https://www.mmobomb.com/news/trion-shutting-devilian-online-march-5/ The forums were last available on 18th March 2018 according to WayBack Machinehttps://web.archive.org/web/20180318095825/http://forums.deviliangame.com/. As of now, they redirect to trionworlds.com
Fotopedia

Fotopedia is user-contributed encyclopedia for photos
== Shutdown ==
Fotopedia is shutting down
July 31, 2014 by fotonauts
We are sorry to announce that Fotopedia is shutting down.
As of August 10, 2014, Fotopedia.com will close and our iOS applications will cease to function. Our community of passionate photographers, curators and storytellers has made this a wonderful journey, and we’d like to thank you for your hard work and your contributions. We truly believe in the concept of storytelling but don’t think there is a suitable business in it yet.http://blog.fotopedia.com/fotopedia-shutdown/
== Site Structure ==
* Getting original photos: http://images.cdn.fotopedia.com/M9q-AYsaCzw-pTuP7SlOTYg-max_2560.jpg → http://images.cdn.fotopedia.com/M9q-AYsaCzw-pTuP7SlOTYg-original.jpg
* Abusing the API: http://www.fotopedia.com/community#users/most_featured/epoch
* Photos from a user appears in articles and stories.
* fr.fotopedia.com, simple.fotopedia.com, etc corresponding to language from Wikipedia (Londres = London)
* Some things come from external sites like Flickr: /items/flickr-3110692965
* iOS exclusives: http://www.fotopedia.com/products
* Bot trap: /bottrap/trap?random=11e4a41e28fe113233837ae86775e98c
=== Types of URLs ===
* A user: /users/4d41h58ro27a6
* A user's photos: /users/4d41h58ro27a6/last_photos
* A user's stories: /reporter/users/4d41h58ro27a6
* A story: /reporter/stories/OxPlIc_Oce_
* A photo: /items/4d41h58ro27a6-mnYC-fLgByc
* An article: /wiki/List_of_musical_instruments
* A user's following list: /users/e4-OQXo_noA/all_following
* A user's collection of stories: /reporter/personal_magazine/bCW-oGGCRbc
* A user's albums: /users/everlasting/all_albums
* A user's album: /albums/hU7EIxucmf4
* Something?: /albums/567c25cd-7ff1-42c8-aff1-05afc9b89bab/entries/fc4543dc-52df-45ca-bc3d-eee560df1cd2/photos?page=2
* Projects: /projects/A76yOhjrDPE
== How to Help ==
Start up a Warrior. Alternatively, the scripts can be run manually as detailed in the source code repositories.
Phase 1 is content discovery. A list of usernames are combed for their IDs of photos, albums, stories etc. Phase 2 is the actual download of photos and pages.
== Archives ==
Archives are saved in the WARC format and are uploaded to archiveteam_fotopedia collection on the Internet Archive.
You can browse the archives using the Wayback Machine.
You can access the CC licensed files at 2014.08.fotopedia-cc-export-collection
== References ==
== External Links ==
* Fotopedia closes, but CC-licensed photos live on
Category:Image hosting
Fotozz.hu

fotozz.hu is a Hungarian photo sharing website and community.
== Site reconnaissance ==
The website was launched in 2003.
Raw numbers as of 2024-12-24:
* 167,856 high-quality photos
* 10,737 users
* 2,765 camera types
* 72 categories
which are fortunately all browsable.
In the last years, barely anyone has been uploading new photos (1-2 photos per day on average). That makes it a candidate for a complete archival. Even though the company Pixinfo Kft. running the site seems fine, the service has had some outages recently, also we've seen things happen.
== Archiving ==
Upcoming in 2025.
Category:Image hosting
Free Music Archive

The Free Music Archive is a free music website.
== Shutdown announcement ==
On 5 Nov 2018, FMA's admin cheyenne_h announced that the site would be shutting down later that month:
We regret to inform you that due to a funding shortage, the FMA will be closing down later this month.
...
The site may go down as early as November 9th, so now's the time to download your playlists, favorite songs, and do whatever personal archiving you need.
The shutdown was later pushed to 16 November, then to 1 December, and finally suspended entirely. On the 10th of December, it was announced that FMA had partnered with the camera gear rental platform KitSplit and would remain online indefinitely.
== Reboot ==
From the main page (freemusicarchive.org):
FMA NEXT is our project name for relaunching Free Music Archive. Focus: improve FMA's free to access music archive. Allow musicians to add more free music and premium content. Improve the look & feel. And last but not least: offer visitors free and premium content.
== Archives of the old site ==
As of 2018-11-08 log, SketchCow called the archival of FMA "all set":
* We are doing an Archivebot grab ()
* Free Music Archive mailed us [the Internet Archive] a hard drive
* Archive-It grabbed a copy
== See also ==
* Dogmazic.net
* DigCCmixter
* Jamendo
== External links ==
* http://freemusicarchive.org/
Free ProHosting

Free ProHosting is a free/premium webhosting company similar to GeoCities, but smaller in scale. In November 2010 they announced that they would shut down their hosting services at the end of that year. Before closing they offered 100 MB of hosting space and a 1.2 GB monthly traffic limit for any hosted accounts.
== Notes on Archiving ==
URLs are organized like this:
 <username>.<servername>.prohosting.com/
which is equivalent to
 <servername>.prohosting.com/<username>/
and to
 <servername>.prohosting.com/~<username>/
It looks like usernames are all lowercase.
=== List of known Servers ===
Please add new servers if you discover them.
* balder
* blake
* cloud
* dt
* earth
* fire
* hammer
* ice
* jove
* lightning
* marvelite
* odin
* opera
* rain
* sky
* snow
* storm
* thor
* thunder
* wave
* white
* wind
== External links ==
* http://free.prohosting.com
* Google results sits going offline in January 1st 2011 http://www.google.es/search?q=%22This+website+will+go+offline+on+January+1st.%22+site:prohosting.com
* Wikipedia links to sites in prohosting http://en.wikipedia.org/w/index.php?title=Special:LinkSearch&target=http%3A%2F%2F*.prohosting.com&limit=500&offset=0
Category:Archive Team
Category:Web hosting
Free Web Hosting Area

Free Web Hosting Area, or FreeWHA for short, is a free hosting website that has been active since 2005.
== Discovery ==
=== subdomains ===
* orgfree.com
* 6te.net
* ueuo.com
* eu5.org
* noads.biz
* coolpage.biz
* freeoda.com
* freevar.com
* freetzi.com
* xp3.biz
=== Not subdomain ===
This code is in the footer of all websites hosted by FreeWHA and can be used to find sites that do not use subdomain.
 <a href="https://www.freewebhostingarea.com" target="_blank" title="Free Web Hosting with PHP5 or PHP7">
 <img alt="Free Web Hosting" src="https://www.freewebhostingarea.com/images/poweredby.png" style="border-width: 0px;width: 180px; height: 45px; float: right;">
 </a>
Category:Web hosting
Freeblog.hu

Freeblog.hu was a Hungarian blog hosting service, one of the first and most popular ones, that operated between 2003 and 2013. In 2006, a sister site freevlog.hu also started (for videos).
Freevlog.hu ceased operations in October 2012, freeblog.hu followed in late 2013.
== Shutdown ==
According to Wayback Machine captures, the freevlog.hu sister site ceased to be available ("temporarily") from October 2012.http://web.archive.org/web/20121016061425/http://www.freevlog.hu/
A user wrote us in May 2013:
"freeblog.hu is a hungarian blog host. It was one of the first in the country and many great things have came out from there. They have not updated their system at least for 2 years, they had a data loss at the last summer and now it looks like their system is slowly dying. Export barely works, you can't post... etc. So here is a blog host with part of the country's internet history. It looks like they will be gone sooner or later."http://badcheese.com/~steve/atlogs/?chan=archiveteam&day=2013-05-25
According to an articlehttp://web.archive.org/web/20150915135907/http://faceportal.hu/a-freeblog-halala-igy-tamaszd-fel-blogodat/, from March 15, 2013 no new entries could be added on freeblog.hu, and there were also availability problems. Another article says that in September 2013, no pages except the main page could be loaded.http://ungparty.net/blog/2013/09/14/megszunt-freeblog-hu/
There was a message on the website for months (till the end) that they are waiting for a new component. Then, in late 2013, freeblog.hu cut the service without notifying their users. The last Wayback Machine capture of the main page was made on October 21, 2013.
The owner Magyar Vendor Kft., that used to run several other sites and also did other kind of business, got into a quite bad financial situation by the early 2010s, and although a rise seems to have started in 2012, there may be financial reasons behind the shutdown.
The fact that no balance sheet was made in year 2014 suggests that the company got liquidated.
== Archives ==
No complete archive of Freeblog was made. However, the Wayback Machine kept making snapshots, most (or some?) of the blogs may be found archived there.
== References ==
Freesound

Freesound is a website for free audio material exchange, founded through the International Computer Music Computer Conference in 2005.
A free account is required to download sounds on the website. 
Uploaders can choose licensing agreements for their sound, CC0, CC BY, and CC BY-NC - but emphasis is placed on collaborative potential of open sounds.
== API ==
User-specific generated keys are used to access Freesound's API.
The API tools search, retrieve metadata, retrieve comments and generate download links for sounds - this could possibly be used to scrape and archive the site.
According to Wikipedia ~230,000 sounds were present on the site as of February 2015. There were an estimated ~1TB of sounds in 2014.
A 2016 machine learning project available on Kaggle aggregated "230,000 sounds and the associated tags" from the Freesound database using the aforementioned API. 
The Kaggle dataset alone does not contain any sound samples, but the provided sounds.json file contains 230,000 Freesound links to download ~192kbps sounds in both ogg and mp3 format with associated tags.
Data such as title, description, uploader and comments are missing, but the file URLs may be a useful starting point for archival.
The website's own browser of sounds lists more than 420,000 items as of August 2019.
== Vital Signs ==
The site shows no immediate signs of going under.
It seems at one point the site relied on university-provided research hosting, and has had a troubled past with downtime. 
In June 2008 the site's domain switched from freesound.iua.upf.edu to freesound.org - since that point reliability has improved.
== Previous Discussion ==
It seems there was some fairly in-depth discussion about the process and politics of archiving Freesound in #archiveteam IRC May 2016. However, users dispanded to IRC channel '#freeallthesounds' which isn't archived - it is unclear whether progress was made.
== External links ==
* 
* 
Category:Web hosting
Froogle

Over the years, Google has discontinued many of its services and products. As a result, unknown amounts of user data have been deleted or are endangered. Archive Team has rescued some of the data on Google-owned sites, but not all. Therefore, Archive Team has decided to take a census of Google services, to see what has and hasn't been saved.
Join the discussion in #froogle.
== Sites ==
Site 	Status 	Saved? 	Number of Users 	Number of Items 	Size (bytes)
Blogger 	Online 	In progress 	 	 	
Chrome Web Store 	Online 	Yes (February 2021)	 	 	
FeedBurner 	Online 	No 	 	 	
FreeBase 	Read-only 	migrating 	 	 	25GB
goo.gl 	Online 	started 	 	 	
Google+ 	Offline 	Yes 	~2.2 billion, ~200 million with any public content	 	
Google Account (user profiles)	Online 	No 	 	 	
Google Answers 	Offline 	Yes 	 	 	1.5GB compressed
Google App Engine 	Online 	No 	 	 	
Google Baraza 	Offline 	Yes 	 	 	
Google Bookmarks 	Online 	No 	 	 	
Google Books 	Online 	No 	 	 	
Google Books Ngram 	Online 	In progress 	 	 	
Google Business Sitebuilder 	Offline 	Yes 	 	 	
Google Code 	Offline (Has public archive available)	In progress 	 	 	
Google Compute Engine 	Online 	No 	 	 	
Google Contacts 	Online 	No 	 	 	
Google Catalogs 	Offline 	No 	 	 	
Google Earth 	Online 	No 	 	 	
Google Finance 	Online 	No 	 	 	
Google Fonts 	Online 	No 	 	 	
Google Groups 	Online 	No 	 	 	
Google Helpouts 	Offline 	Yes 	 	 	
Google Images 	Online 	No 	 	 	
Google Keep 	Online 	No 	 	 	
Google Knol 	Offline 	Yes 	 	 	
Google Play 	Online 	No 	 	1.43 million apps	
Google Map Maker 	Online 	No 	 	 	
Google Maps 	Online 	No 	 	 	
Google Moderator 	Offline 	Yes 	 	 	
Google News 	Online 	No 	 	 	
Google News Archive 	Online 	No 	 	 	
Google Patents 	Online 	No 	 	 	
Google Scholar 	Online 	No 	 	 	
Google Search 	Online 	No 	 	 	
Google Shopping 	Online 	No 	 	 	
Google Translate 	Online 	No 	 	 	
Google Trends 	Online 	No 	 	 	
Google Voice 	Online 	No 	3.5 million ish 	 	
Google Web History 	Online 	No 	 	 	
Orkut 	[Read-only] 	No 	66 million 	>1 billion 	
Panoramio 	Closing 	No 	1 to 5 million 	72 million photos	
Picasa Web Albums 	Online 	No 	 	 	
YouTube 	Online 	No 	>1 billion 	 	500-600PB
Zagat 	Online 	No 	 	 	
== Google Apps ==
Many different Google sites have been "featurized" into Google Apps, including:
Site 	Status 	Saved?	Number of Users 	Number of Items	Size (bytes)
Gmail 	Online 	No 	500 million 	 	
Google Apps Vault 	Online 	No 	 	 	
Google Apps Marketplace	Online 	No 	 	 	
Google Calendar 	Online 	No 	 	 	
Google Classroom 	Online 	No 	 	 	
Google Docs 	Online 	No 	 	 	
Google Drawings 	Online 	No 	 	 	
Google Drive 	OnlineWeb hosting from Drive to be discontinued on August 31, 2016.	No 	>190 million users	 	
Google Forms 	Online 	No 	 	 	
Google Photos 	Online 	No 	 	 	
Google Sheets 	Online 	No 	 	 	
Google Sites 	Online 	No 	 	 	
Google Slides 	Online 	No 	 	 	
== References ==
== See also ==
* Woohoo
== External links ==
* @textfiles: But no, at this point, it's obvious archive team should audit every Google information store...
* List of Google products (Wikipedia)
* https://killedbygoogle.com/
Category:Google