Package: game-extraction-toolbox
Game Extraction ToolboxCLI Tools for investigating game files and extracting known packagesRequirementsPython 3.9 - 3.11This should be a final release, especially if on Windows / C++ build tools aren't installedPIPxInstallationQuick (if Python and Pipx are already installed)pipx install game-extraction-toolboxFullPython 3.9+Should be on Linux and MacOSX by default; for Windows, I recommendDigital Ocean's tutorialPIPXWindowspy -m pip install --user pipx
py -m pipx ensurepathLinuxpy -m pip install --user pipx
py -m pipx ensurepathMacOSXbrew install pipx
pipx ensurepathUsagegextoolbox --helpYou can get --help on any command or subcommand, for example:gextoolbox file --helpExtracting Known Game CollectionsThis tool has scripts for a substantial number of game rereleases.
To get a list of the known rereleases:gextoolbox tasks listTo get details, including configurable properties:gextoolbox tasks details --task TASKNAMETo run a task with default settings, extracting into the current directory:gextoolbox tasks extract --task TASKNAME --destdir ./To run a task with a custom input folder and an example property:gextoolbox tasks extract --task TASKNAME --srcdir /path/to/rerelease/ --destdir ./ --prop "include-partials=True"FAQWhy make this?I want to get ROMs legally, or as legally as possible. I've bought cart readers, but these 'official' ROMs are often different or for platforms that aren't viable for cartridge dumping. I want this to be feasible for others as well.Why are you reimplementing others? Why Python?Many solutions so far are written in Bash, Powershell, and Batch, none of which are not well suited to binary file manipulation. Some of these are documented in foreign languages. Many of these scripts haven't been updated in years. My ideal is that this CLI is useful for reasearching these packages and extracting them, and that common library improvements improve the extraction processes over time.Is this legal?I am not a lawyer, but any good lawyer would probably tell you that 'It depends'. That said, this is an effort to be as legal as possible; this code doesn't download things you don't own, and I intend to leave decryption (bypassing copyright protection devices) out, which would be illegal under the DMCA. This is also at a weird intersection of older law, such as Fair Use, which generally - in most jurisdictions - has some protections for format-shifting. That's largely the point of this - ROMs are flexible and can be played on $20 cheap handhelds, phones, Steam Decks, PCs, tablets, Chromebooks, web pages, etc.Why not just play the official release?I prefer not to be artificially locked to specific platforms - why play a 25 year old arcade title on my Steam PC that my phone can easily handle? Furthermore, some official releases are... not great. Subpar emulation on lower-end PCs, slow loading and menus, older MAME runtimes with known issues - we can do better.Why not just download the ROMs?It is illegal. One can discuss economic moral implications all day, but at the end of the day, it's theft. As a software developer myself, I don't want my code stolen - so I can't claim a moral high ground there. Equally importantly, showing classic game owners that the market will support rereleases increases the liklihood of rereleases, and I want people who can't/won't download ROMs to enjoy these titles as well.What artificial limitations does this tool have to prevent copyright claims?This tool will not hold decryption keys.This tool will not hold any 'whole' files, nor substantial parts of files. A limited 'patch' capability is allowed for less than 1% of a file, but this should be very limited.This MAME ROM is actually XXXXYY, not XXXXZZ - why do you have it that way?The goal of this tool is not to create a perfect ROM set for the latest MAME, because it can't. A few factors go into what version of MAME we target:What version of MAME did the rerelease use?Are there missing files, like encryption keys or dl-1425.bin, that force us to target a different version of MAME to make a ROM that is usable out of the box?When did MAME split specific collections?What are file CRCs?Did the rerelease modify files for legal/moral reasons?How can we make a playable ROM?For example, let's use Street Fighter 30th Anniversary Collection's copy of Street Fighter Alpha 2. It's technically both sfa2u and sfa2ur1, and neither! Here's why, using Arcade Italia'sSFA2UandSFA2UR1as a source:The primary difference between these two ROMs is maincpu, like sz2u.03.At first glance, this looks like sfa2ur1, because the sz2u.03 we create has a CRC of 84a09006. Case closed, right?Well, not so fast. If we make a sfa2ur1 or sfa2u zip file, neither are natively playable in MAME 0.246, because we don't have the sfa2u.key or dl-1425.bin.The standard solution here - since this project will not add copyrighted material not in the source - is to target an older version of MAME, 0.139 (aka Mame 2010 in RetroArch). Except... sfa2ur1 didn't exist until MAME 0.141!We then look at sfa2u before the split. Lo and behold, sz2u.03 has the CRC of 84a90006. OK, so it's sfa2u, right?Kinda. The gfx CRCs don't match EITHER ROM. This is most likely because Capcom modified the graphics to remove copyrighted or offensive content. However, there's no MAME clone with the gfx CRCs we have.Since our goal is to make a playable ROM, not a perfect one, we can test the ROM in MAME 0.139. It works, so we'll call it sfa2u, but make a note in the documentation for that command.These are noted in the documentation for each script. If you think a ROM is misidentified, please:Search for a closed issue - these will likely be brought up multiple times!If none exists, use a source likeArcade Italiato verify. Make sure you look at the original file we create, and watch for the following things:The MAME version we targetAny missing filesWhen a ROM was splitCRCs for every fileIf you are solidly convinced that there's a misnaming, open a ticket and we'll take a look.CollectionsPlayableCollectionStatusVerificationNotesArcade Collection Anniversary Classics88%YAll games except scramble!Atari Vault90%YSome arcade ROMs from this collection are incomplete.Blizzard Arcade Collection100%YBreakers Collection100%YBubsy Two-Fur100%YCapcom Arcade Stadium 1 (via Depot)95%NRequires Steam depot downloading, a couple shaky ROMs...Capcom Beat 'Em Up Bundle95%Y6/7 playable on some version of MAME, but wof/wofj missing audiocpu dataCapcom Fighting Collection90%YCPS2 is semi-standard. No Enc keys present. CP3 game is a curveball!Collection of SaGa/Final Fantasy Legend100%YDisney Afternoon Collection100%YDisney Classics Aladdin & Lion King w/DLC100%YIncludes Jungle Book DLCDouble Dragon Trilogy100%YIREM Arcade Classics100%YMega Man Legacy Collection 1100%YMega Man Battle Network Legacy Collection 1100%YMega Man Battle Network Legacy Collection 2100%YMega Man X Legacy Collection 175%YX4 doesn't appear to be ROM basedNamco Arcade Game Series: Dig Dug100%YNamco Arcade Game Series: Galaga100%YNamco Arcade Game Series: Ms. Pac-Man100%YNamco Arcade Game Series: Pacman100%YNeoGeo Classics by SNK (Humble Store)95%YBaseball Stars 2 is more complexPac Man Museum Plus40%YSome progress, but there are a lot of non-extractable titles.Sega Genesis and Mega Drive Collection90%YSome compressed variants not yet extractedSega Smash Pack 1100%YSega Smash Pack 1100%YAll games except Sega Swirl!Sonic Adventure DX (Hidden Game Gear games)100%YThis is only the Game Gear games - SADX itself can not be made into a ROM/ISO!SNK 40th Anniversary Collection100%YAll games supported by an emulator are extracted!Street Fighter 30th Anniversary Collection90%YNow includes all playable international versions.Wonder Boy: The Dragon's Trap100%YZombies Ate My Neighbors and Ghoul Patrol100%YCompletely UnplayableCollectionStatusNotesCapcom Arcade Stadium 10%We can only unwrap KPKA at this time!Capcom Arcade Stadium 20%We can only unwrap KPKA at this time!Mortal Kombat Arcade Kollection1%Audio ROMs are stripped from this titlePsikyo Shooter Collector's Bundle1%SE ROMs appear to be missingROM Status GuideStatusNotesNo ROMThis title in this collection does not appear to be ROM based; it's a port, remake, remaster, rebuild, etc.PartialSome extraction has been done for this title, but it is still unusable. Typically only good for research or combining with other ROMs.PlayableThis ROM is not perfect, but is functional in at least some emulators.GoodThis game is a verifiable, functional ROM.What makes a ROM 'Playable' instead of 'Good'?The ROM has missing files that can be filled with placeholders or left out.Files are present that work as normal, but have bad CRCs. This may be due to copyright changes, etc., but ultimately does not match a known good source.Publishers may offer modified ROMs - such as increased difficulty, theming, or new game modes - which are often not publicly tracked.It may only work in older versions of emulators due to odd dumps, missing files, etc.