Package: whoogle-search-kdev
Get Google search results, but without any ads, javascript, AMP links, cookies, or IP address tracking. Easily deployable in one click as a Docker app, and customizable with a single config file. Quick and simple to implement as a primary search engine replacement on both desktop and mobile.ContentsFeaturesDependenciesInstall/DeployHeroku Quick DeployRepl.itFly.iopipxpipManualDockerArch/AURHelm/KubernetesEnvironment Variables and ConfigurationUsageExtra StepsSet Primary Search EnginePrevent Downtime (Heroku Only)Manual HTTPS EnforcementUsing with Firefox ContainersReverse ProxyingNginxContributingFAQPublic InstancesScreenshotsMirrors (read-only)GitLabGogsFeaturesNo ads or sponsored contentNo JavaScript*No cookies**No tracking/linking of your personal IP address***No AMP linksNo URL tracking tags (i.e. utm=%s)No referrer headerTor and HTTP/SOCKS proxy supportAutocomplete/search suggestionsPOST request search and suggestion queries (when possible)View images at full res without site redirect (currently mobile only)Light/Dark/System theme modes (with support forcustom CSS theming)Randomly generated User AgentEasy to install/deployDDG-style bang (i.e.!<tag> <query>) searchesOptional location-based searching (i.e. results near <city>)Optional NoJS mode to view search results in a separate window with JavaScript blocked*No third party JavaScript. Whoogle can be used with JavaScript disabled, but if enabled, uses JavaScript for things like presenting search suggestions.**No third party cookies. Whoogle uses server side cookies (sessions) to store non-sensitive configuration settings such as theme, language, etc. Just like with JavaScript, cookies can be disabled and not affect Whoogle's search functionality.***If deployed to a remote server, or configured to send requests through a VPN, Tor, proxy, etc.DependenciesIf using Heroku Quick Deploy,you can skip this section.Docker (Windows,macOS,Ubuntu,other Linux distros)Only needed if you intend on deploying the app as a Docker imagePython3libcurl4-openssl-devandlibssl-devmacOS:brew install openssl curl-opensslUbuntu:sudo apt-get install -y libcurl4-openssl-dev libssl-devArch:pacman -S curl opensslInstallThere are a few different ways to begin using the app, depending on your preferences:A)Heroku Quick DeployProvides:Free deployment of appFree HTTPS url (https://<your app name>.herokuapp.com)Downtime after periods of inactivity (solution)Notes:Requires a (free) Heroku accountSometimes has issues with auto-redirecting tohttps. Make sure to navigate to thehttpsversion of your app before adding as a default search engine.B)Repl.itNote: Requires a (free) Replit accountProvides:Free deployment of appFree HTTPS url (https://<app name>.<username>.repl.co)Supports custom domainsDowntime after periods of inactivity (solution 1,solution 2)C)Fly.ioYou will need aPAIDFly.ioaccount to deploy Whoogle.Install the CLI:https://fly.io/docs/hands-on/installing/Deploy the appflyctlauthlogin
flyctllaunch--imagekoval01/whoogle-search-kdev:latestYour app is now available athttps://<app-name>.fly.dev.D)pipxPersistent install:pipx install git+https://github.com/koval01/whoogle-search-kdev.gitSandboxed temporary instance:pipx run --spec git+https://github.com/koval01/whoogle-search-kdev.git whoogle-search-kdevE) pippip install whoogle-search-kdev$whoogle-search-kdev--help
usage:whoogle-search-kdev[-h][--port<portnumber>][--host<ipaddress>][--debug][--https-only][--userpass<username:password>][--proxyauth<username:password>][--proxytype<socks4|socks5|http>][--proxyloc<location:port>]WhoogleSearchconsolerunner

optionalarguments:-h,--helpShowthishelpmessageandexit--port<portnumber>Specifiesaporttorunon(default5000)--host<ipaddress>Specifiesthehostaddresstouse(default127.0.0.1)--debugActivatesdebugmodefortheserver(defaultFalse)--https-onlyEnforcesHTTPSredirectsforallrequests--userpass<username:password>Setsausername/passwordbasicauthcombo(defaultNone)--proxyauth<username:password>Setsausername/passwordforaHTTP/SOCKSproxy(defaultNone)--proxytype<socks4|socks5|http>Setsaproxytypeforallconnections(defaultNone)--proxyloc<location:port>Setsaproxylocationforallconnections(defaultNone)See theavailable environment variablesfor additional configuration.F) ManualNote:Content-Security-Policyheaders can be sent by Whoogle if you setWHOOGLE_CSP.Clone the repo and run the following commands to start the app in a local-only environment:gitclonehttps://github.com/koval01/whoogle-search-kdev.gitcdwhoogle-search-kdev
python3-mvenvvenvsourcevenv/bin/activate
pipinstall-rrequirements.txt
./runSee theavailable environment variablesfor additional configuration.systemd ConfigurationAfter building the virtual environment, you can add something like the following to/lib/systemd/system/whoogle.serviceto set up a Whoogle Search systemd service:[Unit]Description=Whoogle[Service]# Basic auth configuration, uncomment to enable#Environment=WHOOGLE_USER=<username>#Environment=WHOOGLE_PASS=<password># Proxy configuration, uncomment to enable#Environment=WHOOGLE_PROXY_USER=<proxy username>#Environment=WHOOGLE_PROXY_PASS=<proxy password>#Environment=WHOOGLE_PROXY_TYPE=<proxy type (http|https|proxy4|proxy5)#Environment=WHOOGLE_PROXY_LOC=<proxy host/ip># Site alternative configurations, uncomment to enable# Note: If not set, the feature will still be available# with default values.#Environment=WHOOGLE_ALT_TW=farside.link/nitter#Environment=WHOOGLE_ALT_YT=farside.link/invidious#Environment=WHOOGLE_ALT_IG=farside.link/bibliogram/u#Environment=WHOOGLE_ALT_RD=farside.link/libreddit#Environment=WHOOGLE_ALT_MD=farside.link/scribe#Environment=WHOOGLE_ALT_TL=farside.link/lingva#Environment=WHOOGLE_ALT_IMG=farside.link/rimgo#Environment=WHOOGLE_ALT_WIKI=farside.link/wikiless#Environment=WHOOGLE_ALT_IMDB=farside.link/libremdb#Environment=WHOOGLE_ALT_QUORA=farside.link/quetre# Load values from dotenv only#Environment=WHOOGLE_DOTENV=1Type=simpleUser=<username># If installed as a package, add:ExecStart=<python_install_dir>/python3 <whoogle_install_dir>/whoogle-search-kdev --host 127.0.0.1 --port 5000# For example:# ExecStart=/usr/bin/python3 /home/my_username/.local/bin/whoogle-search-kdev --host 127.0.0.1 --port 5000# Otherwise if running the app from source, add:ExecStart=<whoogle_repo_dir>/run# For example:# ExecStart=/var/www/whoogle-search-kdev/runExecReload=/bin/kill -HUP $MAINPIDRestart=alwaysRestartSec=3SyslogIdentifier=whoogle[Install]WantedBy=multi-user.targetThen,sudo systemctl daemon-reload
sudo systemctl enable whoogle
sudo systemctl start whoogleTor ConfigurationoptionalIf routing your request through Tor you will need to make the following adjustments.
Due to the nature of interacting with Google through Tor we will need to be able to send signals to Tor and therefore authenticate with it.There are two authentication methods, password and cookie. You will need to make changes to your torrc:CookieUncomment or add the following lines in your torrc:ControlPort 9051CookieAuthentication 1DataDirectoryGroupReadable 1CookieAuthFileGroupReadable 1Make the tor auth cookie readable:This is assuming that you are using a dedicated user to run whoogle. If you are using a different user replacewhooglewith that user.chmod tor:whoogle /var/lib/torchmod tor:whoogle /var/lib/tor/control_auth_cookieRestart the tor service:systemctl restart torSet the Tor environment variable to 1,WHOOGLE_CONFIG_TOR. Refer to theEnvironment Variablessection for more details.This may be added in the systemd unit file or env fileWHOOGLE_CONFIG_TOR=1PasswordRun this command:tor --hash-password {Your Password Here}; put your password in place of{Your Password Here}.Keep the output of this command, you will be placing it in your torrc.Keep the password input of this command, you will be using it later.Uncomment or add the following lines in your torrc:ControlPort 9051HashedControlPassword {Place output here}; put the output of the previous command in place of{Place output here}.Now take the password from the first step and place it in the control.conf file within the whoogle working directory, ie.misc/tor/control.confIf you want to place your password file in a different location set this location with theWHOOGLE_TOR_CONFenvironment variable. Refer to theEnvironment Variablessection for more details.Heavily restrict access to control.conf to only be readable by the user running whoogle:chmod 400 control.confFinally set the Tor environment variable and use password variable to 1,WHOOGLE_CONFIG_TORandWHOOGLE_TOR_USE_PASS. Refer to theEnvironment Variablessection for more details.These may be added to the systemd unit file or env file:WHOOGLE_CONFIG_TOR=1WHOOGLE_TOR_USE_PASS=1G) Manual (Docker)Ensure the Docker daemon is running, and is accessible by your user accountTo add user permissions, you can executesudo usermod -aG docker yourusernameRunningdocker psshould return something besides an error. If you encounter an error saying the daemon isn't running, trysudo systemctl start docker(Linux) or ensure the docker tool is running (Windows/macOS).Clone and deploy the docker app using a method below:Docker CLIThrough Docker Hub:dockerpullkoval01/whoogle-search-kdev
dockerrun--publish5000:5000--detach--namewhoogle-search-kdevkoval01/whoogle-search-kdev:latestor with docker-compose:gitclonehttps://github.com/koval01/whoogle-search-kdev.gitcdwhoogle-search-kdev
docker-composeupor by building yourself:gitclonehttps://github.com/koval01/whoogle-search-kdev.gitcdwhoogle-search-kdev
dockerbuild--tagwhoogle-search-kdev:1.0.
dockerrun--publish5000:5000--detach--namewhoogle-search-kdevwhoogle-search-kdev:1.0Optionally, you can also enable some of the following environment variables to further customize your instance:dockerrun--publish5000:5000--detach--namewhoogle-search-kdev\-eWHOOGLE_USER=username\-eWHOOGLE_PASS=password\-eWHOOGLE_PROXY_USER=username\-eWHOOGLE_PROXY_PASS=password\-eWHOOGLE_PROXY_TYPE=socks5\-eWHOOGLE_PROXY_LOC=ip\whoogle-search-kdev:1.0And kill with:docker rm --force whoogle-search-kdevUsingHeroku CLIherokulogin
herokucontainer:login
gitclonehttps://github.com/koval01/whoogle-search-kdev.gitcdwhoogle-search-kdev
herokucreate
herokucontainer:pushweb
herokucontainer:releaseweb
herokuopenThis series of commands can take a while, but once you run it once, you shouldn't have to run it again. The final command,heroku openwill launch a tab in your web browser, where you can test out Whoogle and evenset it as your primary search engine.
You may also edit environment variables from your app’s Settings tab in the Heroku Dashboard.Arch Linux & Arch-based DistributionsThere is anAUR package available, as well as a pre-built and daily updated package available atChaotic-AUR.Helm chart for KubernetesTo use the Kubernetes Helm Chart:Ensure you haveHelm>=3.0.0installedClone this repositoryUpdatecharts/whoogle/values.yamlas desiredRunhelm install whoogle ./charts/whoogleUsing your own server, or alternative container deploymentThere are other methods for deploying docker containers that are well outlined inthis article, but there are too many to describe set up for each here. Generally it should be about the same amount of effort as the Heroku deployment.Depending on your preferences, you can also deploy the app yourself on your own infrastructure. This route would require a few extra steps:A server (I personally recommendDigital OceanorLinode, their cheapest tiers will work fine)Your own URL (I suppose this is optional, but recommended)SSL certificates (free throughLet's Encrypt)A bit more experience or willingness to work through issuesEnvironment VariablesThere are a few optional environment variables available for customizing a Whoogle instance. These can be set manually, or copied intowhoogle.envand enabled for your preferred deployment method:Local runs: SetWHOOGLE_DOTENV=1before runningWithdocker-compose: Uncomment theenv_fileoptionWithdocker build/run: Add--env-file ./whoogle.envto your commandVariableDescriptionWHOOGLE_URL_PREFIXThe URL prefix to use for the whoogle instance (i.e. "/whoogle")WHOOGLE_DOTENVLoad environment variables inwhoogle.envWHOOGLE_USERThe username for basic auth. WHOOGLE_PASS must also be set if used.WHOOGLE_PASSThe password for basic auth. WHOOGLE_USER must also be set if used.WHOOGLE_PROXY_USERThe username of the proxy server.WHOOGLE_PROXY_PASSThe password of the proxy server.WHOOGLE_PROXY_TYPEThe type of the proxy server. Can be "socks5", "socks4", or "http".WHOOGLE_PROXY_LOCThe location of the proxy server (host or ip).EXPOSE_PORTThe port where Whoogle will be exposed.HTTPS_ONLYEnforce HTTPS. (Seehere)WHOOGLE_ALT_TWThe twitter.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_YTThe youtube.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_IGThe instagram.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_RDThe reddit.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_TLThe Google Translate alternative to use. This is used for all "translate ____" searches.  Set to "" to disable.WHOOGLE_ALT_MDThe medium.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_IMGThe imgur.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_WIKIThe wikipedia.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_IMDBThe imdb.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_ALT_QUORAThe quora.com alternative to use when site alternatives are enabled in the config. Set to "" to disable.WHOOGLE_AUTOCOMPLETEControls visibility of autocomplete/search suggestions. Default on -- use '0' to disable.WHOOGLE_MINIMALRemove everything except basic result cards from all search queries.WHOOGLE_CSPSets a default set of 'Content-Security-Policy' headersWHOOGLE_RESULTS_PER_PAGESet the number of results per pageWHOOGLE_TOR_SERVICEEnable/disable the Tor service on startup. Default on -- use '0' to disable.WHOOGLE_TOR_USE_PASSUse password authentication for tor control port.WHOOGLE_TOR_CONFThe absolute path to the config file containing the password for the tor control port. Default: ./misc/tor/control.conf WHOOGLE_TOR_PASS must be 1 for this to work.Config Environment VariablesThese environment variables allow setting default config values, but can be overwritten manually by using the home page config menu. These allow a shortcut for destroying/rebuilding an instance to the same config state every time.VariableDescriptionWHOOGLE_CONFIG_DISABLEHide config from UI and disallow changes to config by clientWHOOGLE_CONFIG_COUNTRYFilter results by hosting countryWHOOGLE_CONFIG_LANGUAGESet interface languageWHOOGLE_CONFIG_SEARCH_LANGUAGESet search result languageWHOOGLE_CONFIG_BLOCKBlock websites from search results (use comma-separated list)WHOOGLE_CONFIG_THEMESet theme mode (light, dark, or system)WHOOGLE_OPEN_AI_TOKENToken for accessing the OpenAI APIWHOOGLE_CONFIG_SAFEEnable safe searchesWHOOGLE_CONFIG_ALTSUse social media site alternatives (nitter, invidious, etc)WHOOGLE_CONFIG_NEARRestrict results to only those near a particular cityWHOOGLE_CONFIG_TORUse Tor routing (if available)WHOOGLE_CONFIG_NEW_TABAlways open results in new tabWHOOGLE_CONFIG_VIEW_IMAGEEnable View Image optionWHOOGLE_CONFIG_GET_ONLYSearch using GET requests onlyWHOOGLE_CONFIG_URLThe root url of the instance (https://<your url>/)WHOOGLE_CONFIG_STYLEThe custom CSS to use for styling (should be single line)WHOOGLE_CONFIG_PREFERENCES_ENCRYPTEDEncrypt preferences token, requires preferences keyWHOOGLE_CONFIG_PREFERENCES_KEYKey to encrypt preferences in URL (REQUIRED to show url)UsageSame as most search engines, with the exception of filtering by time range.To filter by a range of time, append ":past " to the end of your search, where  can behour,day,month, oryear. Example:coronavirus updates :past hourExtra StepsSet Whoogle as your primary search engineNote: If you're using a reverse proxy to run Whoogle Search, make sure the "Root URL" config option on the home page is set to your URL before going through these steps.Browser settings:Firefox (Desktop)Version 89+Navigate to your app's url, right click the address bar, and select "Add Search Engine".Previous versionsNavigate to your app's url, and click the 3 dot menu in the address bar. At the bottom, there should be an option to "Add Search Engine".Once you've added the new search engine, open your Firefox Preferences menu, click "Search" in the left menu, and use the available dropdown to select "Whoogle" from the list.Note: If your Whoogle instance uses Firefox Containers, you'll need togo through the steps hereto get it working properly.Firefox (iOS)In the mobile app Settings page, tap "Search" within the "General" section. There should be an option titled "Add Search Engine" to select. It should prompt you to enter a title and search query url - use the following elements to fill out the form:Title: "Whoogle"URL:http[s]://\<your whoogle url\>/search?q=%sFirefox (Android)Version <79.0.0Navigate to your app's urlLong-press on the search text fieldClick the "Add Search Engine" menu itemSelect a name and click okClick the 3 dot menu in the top rightNavigate to the settings menu and select the "Search" sub-menuSelect Whoogle and press "Set as default"Version >=79.0.0Click the 3 dot menu in the top rightNavigate to the settings menu and select the "Search" sub-menuClick "Add search engine"Select the 'Other' radio buttonName: "Whoogle"Search string to use:https://\<your whoogle url\>/search?q=%sAlfred(Mac OS X)Go toAlfred Preferences>Features>Web Searchand clickAdd Custom Search. Then configure these settingsSearch URL: `https://<your whoogle url>/search?q={query}Title:Whoogle for '{query}'(or whatever you want)Keyword:whoogleGo toDefault Resultsand click theSetup fallback resultsbutton. Click+and add Whoogle, then  drag it to the top.Chrome/Chromium-based BrowsersAutomaticVisit the home page of your Whoogle Search instance -- this will automatically add the search engine if therequirementsare met (GET request, no OnSubmit script, no path). If not, you can add it manually.ManualUnder search engines > manage search engines > add, manually enter your Whoogle instance details with a<whoogle url>/search?q=%sformatted search URL.Prevent Downtime (Heroku only)Part of the deal with Heroku's free tier is that you're allocated 550 hours/month (meaning it can't stay active 24/7), and the app is temporarily shut down after 30 minutes of inactivity. Once it becomes inactive, any Whoogle searches will still work, but it'll take an extra 10-15 seconds for the app to come back online before displaying the result, which can be frustrating if you're in a hurry.A good solution for this is to set up a simple cronjob on any device at your home that is consistently powered on and connected to the internet (in my case, a PiHole worked perfectly). All the device needs to do is fetch app content on a consistent basis to keep the app alive in whatever ~17 hour window you want it on (17 hrs * 31 days = 527, meaning you'd still have 23 leftover hours each month if you searched outside of your target window).For instance, adding*/20 7-23 * * * curl https://<your heroku app name>.herokuapp.com > /home/<username>/whoogle-refreshwill fetch the home page of the app every 20 minutes between 7am and midnight, allowing for downtime from midnight to 7am. And again, this wouldn't be a hard limit - you'd still have plenty of remaining hours of uptime each month in case you were searching after this window has closed.Since the instance is destroyed and rebuilt after inactivity, config settings will be reset once the app enters downtime. If you have configuration settings active that you'd like to keep between periods of downtime (like dark mode for example), you could instead add*/20 7-23 * * * curl -d "dark=1" -X POST https://<your heroku app name>.herokuapp.com/config > /home/<username>/whoogle-refreshto keep these settings more or less permanent, and still keep the app from entering downtime when you're using it.HTTPS EnforcementOnly needed if your setup requires Flask to redirect to HTTPS on its own -- generally this is something that doesn't need to be handled by Whoogle Search.Note: You should have your own domain name andan https certificatein order for this to work properly.Heroku: Ensure that theRoot URLconfiguration on the home page begins withhttps://and nothttp://Docker build: Add--build-arg use_https=1to your run commandDocker image: Set the environment variable HTTPS_ONLY=1Pip/Pipx: Add the--https-onlyflag to the end of thewhoogle-search-kdevcommandDefaultrunscript: Modify the script locally to include the--https-onlyflag at the end of the python run commandUsing with Firefox ContainersUnfortunately, Firefox Containers do not currently pass throughPOSTrequests (the default) to the engine, and Firefox caches the opensearch template on initial page load. To get around this, you can take the following steps to get it working as expected:Remove any existing Whoogle search engines from Firefox settingsEnableGET Requests Onlyin Whoogle configClear Firefox cacheRestart FirefoxNavigate to Whoogle instance andre-add the engineReverse ProxyingNginxHere is a sample Nginx config for Whoogle:server {
	server_name your_domain_name.com;
	access_log /dev/null;
	error_log /dev/null;

	location / {
	    proxy_set_header X-Real-IP $remote_addr;
	    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	    proxy_set_header X-Forwarded-Proto $scheme;
	    proxy_set_header Host $host;
	    proxy_set_header X-NginX-Proxy true;
	    proxy_pass http://localhost:5000;
	}
}You can then add SSL support using LetsEncrypt by following a guide such asthis one.ContributingUnder the hood, Whoogle is a basic Flask app with the following structure:app/routes.py: Primary app entrypoint, contains all API routesrequest.py: Handles all outbound requests, including proxied/Tor connectivityfilter.py: Functions and utilities used for filtering out content from upstream Google search resultsutils/bangs.py: All logic related to handling DDG-style "bang" queriesresults.py: Utility functions for interpreting/modifying individual search resultssearch.py: Creates and handles new search queriessession.py: Miscellaneous methods related to user sessionstemplates/index.html: The home page templatedisplay.html: The search results templateheader.html: A general "top of the page" query header for desktop and mobilesearch.html: An iframe-able search pagelogo.html: A template consisting mostly of the Whoogle logo as an SVG (separated to help keepindex.htmla bit cleaner)opensearch.xml: A template used for supportingOpenSearch.imageresults.html: An "experimental" template used for supporting the "Full Size" image feature on desktop.static/<css|js>CSS/Javascript files, should be self-explanatorystatic/settingsKey-value JSON files for establishing valid configuration valuesIf you're new to the project, the easiest way to get started would be to try fixingan open bug report. If there aren't any open, or if the open ones are too stale, try taking on afeature request. Generally speaking, if you can write something that has any potential of breaking down in the future, you should write a test for it.The project follows thePEP 8 Style Guide, but is liable to change. Static typing should always be used when possible. Function documentation is greatly appreciated, and typically follows the below format:defcontains(x:list,y:int)->bool:"""Check a list (x) for the presence of an element (y)Args:x: The list to inspecty: The int to look forReturns:bool: True if the list contains the item, otherwise False"""returnyinxTranslatingWhoogle currently supports translations usingtranslations.json. Language values in this file need to match the "value" of the according language inlanguages.json(i.e. "lang_en" for English, "lang_es" for Spanish, etc). After you add a new set of translations totranslations.json, open a PR with your changes and they will be merged in as soon as possible.FAQWhat's the difference between this andSearx?Whoogle is intended to only ever be deployed to private instances by individuals of any background, with as little effort as possible. Prior knowledge of/experience with the command line or deploying applications is not necessary to deploy Whoogle, which isn't the case with Searx. As a result, Whoogle is missing some features of Searx in order to be as easy to deploy as possible.Whoogle also only uses Google search results, not Bing/Quant/etc, and uses the existing Google search UI to make the transition away from Google search as unnoticeable as possible.I'm a huge fan of Searx though and encourage anyone to use that instead if they want access to other search engines/a different UI/more configuration.Why does the image results page look different?A lot of the app currently piggybacks on Google's existing support for fetching results pages with Javascript disabled. To their credit, they've done an excellent job with styling pages, but it seems that the image results page - particularly on mobile - is a little rough. Moving forward, with enough interest, I'd like to transition to fetching the results and parsing them into a unique Whoogle-fied interface that I can style myself.ScreenshotsDesktopMobile