Package: osu-data
osu! Data on Dockerpip install osu-data; osu-data -m mania -v top_1000 -ym YYYY_MMDocker must be installed and running on your machine.Retrieves database data fromhttps://data.ppy.sh/and hosts it on a local MySQL
server.
Optionally, store all ranked/loved.osufiles in a service with the-ftag.Get StartedIMPORTANT: MySQL data persists across runs.
Recreate the MySQL Service if you changed the data used.Install via pippip install osu-dataMinimally, specify:-m,--mode:
The game mode to build the database with.osu,taiko,catchormania-v,--version:
The database version.top_1000,top_10000orrandom_10000Optionally, specify:-ym,--year_month:
The year and month of the database in the formatYYYY_MMDefault uses
the latest. We recommend to specify this to avoid issues where the data
isn't ready on the start of each month.-p,--port:
The port to expose MySQL on. Default is3308-f,--files:
Whether to download.osufiles.-np,--nginx-port:
The port to expose the nginx service on. Default is8080.
Not used if-fis not specified.--...:
See below table, these are optional flags to include or exclude more
data.By specifying the flag, will INVERT the default value.OptionDefault Value--beatmap-difficulty-attribsFalse--beatmap-difficultyFalse--scoresTrue--beatmap-failtimesFalse--user-beatmap-playcountFalse--beatmapsTrue--beatmapsetsTrue--user-statsTrue--sample-usersTrue--countsTrue--difficulty-attribsTrue--beatmap-performance-blacklistTrueThese options are chosen to be the most useful for analysis, and performance.E.g.osu-data\-mosu-vtop_1000-ym2023_08-p3308-f\--beatmap-difficultyDownload the top 1000 osu! standard beatmapsfrom August 2023expose MySQL on port 3308download.osufilesinclude beatmap difficulty dataConnect on:localhost:<MYSQL_PORT>localhost:<NGINX_PORT>(if-fis specified)Common IssuesDocker daemon is not running. Make sure that Docker is installed and
running. If you're using Docker Desktop, make sure it's actually started.MySQL Data isn't incorrect. A few reasonsImport was abruptly stopped. This can cause some.sqlfiles to be
missing / incomplete. Delete the whole compose project and try again.Didn't specify the optional flags to include files. By default, some.sqlfiles are not loaded. Take a look atosu-data -hand specify the
optional flags to include them.Data is outdated. By default, on every re-run ofosu-data, the data
is
preserved. To update the data, you must delete the whole compose project
and try again.wget: server returned error: HTTP/1.1 404 Not Found. This happens when
you try to pull aYYYY_MMthat doesn't exist, and happens often when the
data isn't yet ready on the start of each month.
Check onhttps://data.ppy.sh/to see whichYYYY_MMare available.rm: can't remove '../osu.mysql.init/*': This is safe to ignore.MySQL Credentials. By default, the MySQL doesn't have a password, so just
userootas the username and leave the password blank.Nofilesservice. This is default,filesservice is optional and
must be activated with the-ftag.osu-data -hfor more info.mysql.cnfThe database is tuned to be fast in importing speed, thus shouldn't be used for
production. Notably, we setinnodb_doublewrite = 0which can compromise
data integrity in the event of a crash. If you want to use this for production,
we recommend to set this up from this Git repo, and tweakmysql.cnf.Important MattersDo not distribute the built images as per peppy's request.
Instead, you can just share the code to build your image, which should yield
the same result.This database is meant to be for analysis, it's not tuned for production.
Tweakmysql.cnfafter importing
for more MySQL customizations.Finally, be mindful on conclusions you make from the data.