Package: yawgbot
yawgbotYet Another WG helper bot. WG stands forWohngemeinschaft, a German word that refers to a living arrangement in which
several tenants share an apartment. It is aimed to automate the extremely complex and time-consuming task of finding a
room or an apartment to rent. Initially it was only working forwg-gesucht.de, now it's a
plugin-based system ready to be extended for every platform to find accommodation.From a technical standpointyawgbotis a simple tool that runs different plugins. A plugin either scrapes websites usingrequestsandbeautifulsoup4or uses APIs (both public and undocumented) both to search for listings and to contact landlords. It uses a SQLite database to save contacted ads and to perform fewer requests to platforms. It also comes with a simple web UI to track your progress and gather your findings across different platforms.The database is stored:on GNU/Linux at~/.local/share/yawgbot/yawgbot.sqliteon Windows at%USERPROFILE%\AppData\Local\rcastellotti\yawgbot\yawgbot.sqliteon macOS at~/Library/Application Support/yawgbot/yawgbot.sqliterunning manuallyTo run the bot manually:install the package withpip3 install yawgbot(avirtual environmentis suggested)configure.config.myl, storedon GNU/Linux at~/.config/yawgbot/config.ymlon Windows at%USERPROFILE%\AppData\Local\rcastellotti\yawgbot\config.ymlon macOS at~/Library/Preferences/yawgbot/config.ymlcreate a new file namedbot.pywith the following content and run it:fromyawgbotimportBotimportlogginglogging.basicConfig(level=logging.INFO)bot=Bot(platforms=["wg-gesucht"])bot.run()running periodically with celeryRunning in background at time intervals is achieved usingCeleryto schedule tasks. It is configured to useSQLiteas bothbackend and broker, to know
more read the docs. The filerun_yawgbot.py, for example, runs the bot each 5 minutes, to use it:runcelery -A run_yawgbot.celery worker --loglevel=INFOto run the workerruncelery -A run_yawgbot.celery beatto schedule the botweb UITo start the web UI simply runyawgbot-webpluginscreating a pluginExtendingyawgbotis a pretty straightforward process. A plugin is a simple module extending thePluginBaseclass, an example iswg-gesucht.py,
supported plugins can be used just by instantiatingBotwith theplatformsargument. Custom plugins can be registered using theBot.register_plugin()method.To start creating a template you can use this boilerplate code:fromyawgbot.pluginBaseimportPluginBaseclassYawgbotPlugin(PluginBase):def__init__(self):passdefget_ads(self,url):passdefparse_ad(self,ad):passdefcontact_ad(self,slug):passdefrun(self):passIf you develop a plugin, consider creating a PR, I will be more than happy to work with you to make it an official plugin and ship it withyawgbot.Each plugin should load configuration values using the staticBot.config()method.To better understand plugins you can read the code insrc/yawgbot/plugins.wg-gesuchtconfigwg-gesucht:
  username: ""
  password: ""
  message_template: ""
  base_url: ""When configuring for use go onhttps://wg-gesucht.deand copy the url you are using to look for accommodation, then replace the last number with{}in order to be able to search across multiple pages, for example, if looking for apartments in Munich: base url ishttps://www.wg-gesucht.de/1-zimmer-wohnungen-in-Munchen.90.1.1.1.html, replace the last1as follows:https://www.wg-gesucht.de/1-zimmer-wohnungen-in-Munchen.90.1.1.{}.html