Package: django-amazon-price-monitor
Table of Contentsdjango-amazon-price-monitorBasic structureModelsFrontend componentsAngular Frontend APIAmazon API componentLicenseSetupPrerequisitesIncluded angular librariesBasic setupSettingsMust have settingsCeleryRest-FrameworkSite URLAWS and Product Advertising API credentialsAmazon associatesImages protocol and domainOptional settingsProduct synchronizationNotificationsCachingCelery settingsDevelopment setup with DockerStructureImage: baseImage: webImage: celeryImage: dataUsageOverride settingsStart/Stop/BuildTemplatesfooterManagement Commandsprice_monitor_batch_create_productsprice_monitor_recreate_productprice_monitor_searchLoggersprice_monitorprice_monitor.product_advertising_apiprice_monitor.utilsTestsInternalsModel graphProduct advertising api synchronizationChange Log0.70.6.10.60.50.40.3b2Pre-Releasesdjango-amazon-price-monitorMonitors prices of Amazon products via Product Advertising API.Basic structureThis is a reusable Django app that can be plugged into your project. It
consists basically of this parts:ModelsFrontend componentsAngular Frontend APIAmazon API componentModelsProductrepresentation of an Amazon productPricerepresentation of a price of an Amazon product at a specific timeSubscriptionsubscribe to a product at a specific price with an email
notificationFrontend componentsThe frontend displays all subscribed products with additional
information and some graphs for price history.The features are the following:list productsshow product detailsshow product price graphsadd subscriptionsadjust subscription price valuedelete subscriptionsAngular Frontend APISimply the API consumed by AngularJS, based on Django REST Framework.Amazon API componentFetches product information from Amazon Product Advertising API through
several tasks powered by Celery and weaves the data into the models.LicenseThis software is licensed with the MIT license. So feel free to do with
it whatever you like.SetupPrerequisitesPython3.33.43.5Django1.81.8 or 1.91.9For additional used packages seesetup.py.Included angular librariesangular-django-rest-resource (commit:
81d752b363668d674201c09d7a2ce6f418a44f13)Basic setupAdd the following apps toINSTALLED_APPS:INSTALLED_APPS = (
    ...
    'price_monitor',
    'price_monitor.product_advertising_api',
    'rest_framework',
)Then migrate:python manage.py migrateAdjust the settings appropiately,see next chapter.Include the url configuration.Setup celery - you’ll need the beat and a worker.SettingsThe values of the following displayed settings are their default
values. If the value is ‘…’ then there is no default value.Must have settingsThe following settings are absolutely necessary to the price monitor
running, please set them:CeleryYou need to have a broker and a result backend set.BROKER_URL = ...
CELERY_RESULT_BACKEND = ...

# some additional settings
CELERY_ACCEPT_CONTENT = ['pickle', 'json']
CELERY_CHORD_PROPAGATES = TrueRest-FrameworkWe use Rest-Framework for Angular frontend:REST_FRAMEWORK = {
    'PAGINATE_BY': 50,
    'PAGINATE_BY_PARAM': 'page_size',
    'MAX_PAGINATE_BY': 100,
}Site URLSpecify the base URL under which your site will be available. Defaults to:http://localhost:8000Necessary for creating links to the site within the notification emails.# base url to the site
PRICE_MONITOR_BASE_URL = 'https://....'AWS and Product Advertising API credentials# your Amazon Web Services access key id
PRICE_MONITOR_AWS_ACCESS_KEY_ID = '...'

# your Amazon Web Services secret access key
PRICE_MONITOR_AWS_SECRET_ACCESS_KEY = '...'

# the region endpoint you want to use.
# Typically the country you'll run the price monitor in.
# possible values: CA, CN, DE, ES, FR, IT, JP, UK, US
PRICE_MONITOR_AMAZON_PRODUCT_API_REGION = '...'

# the assoc tag of the Amazon Product Advertising API
PRICE_MONITOR_AMAZON_PRODUCT_API_ASSOC_TAG = '...'Amazon associatesAs the links to Amazon will be affiliate links with your Amazon associate tag (see above), you have to set your name for the disclaimer
(seehttps://partnernet.amazon.de/gp/associates/agreement).# name of you/your site
PRICE_MONITOR_AMAZON_ASSOCIATE_NAME = 'name/sitename'
# Amazon site being used, choose from on of the following
    'Amazon.co.uk'
    'Local.Amazon.co.uk'
    'Amazon.de'
    'de.BuyVIP.com'
    'Amazon.fr'
    'Amazon.it'
    'it.BuyVIP.com'
    'Amazon.es'
    'es.BuyVIP.com'
PRICE_MONITOR_AMAZON_ASSOCIATE_SITE = '<ONE FROM ABOVE>'Images protocol and domain# if to use the HTTPS URLs for Amazon images.
# if you're running the monitor on SSL, set this to True
# INFO:
#  Product images are served directly from Amazon.
#  This is a restriction when using the Amazon Product Advertising API
PRICE_MONITOR_IMAGES_USE_SSL = True

# domain to use for image serving.
# typically analog to the api region following the URL pattern
#  https://images-<REGION>.ssl-images-amazon.com
PRICE_MONITOR_AMAZON_SSL_IMAGE_DOMAIN = 'https://images-eu.ssl-images-amazon.com'Optional settingsThe following settings can be adjusted but come with reasonable default
values.Product synchronization# time after which products shall be refreshed
# Amazon only allows caching up to 24 hours, so the maximum value is 1440!
PRICE_MONITOR_AMAZON_PRODUCT_REFRESH_THRESHOLD_MINUTES = 720  # 12 hoursNotificationsTo be able to send out the notification emails, set up a proper email
backend (seeDjango
documentation).# time after which to notify the user again about a price limit hit (in minutes)
PRICE_MONITOR_SUBSCRIPTION_RENOTIFICATION_MINUTES = 10080  # 7 days

# sender address of the notification email
PRICE_MONITOR_EMAIL_SENDER = 'noreply@localhost'

# currency name to use on notifications
PRICE_MONITOR_DEFAULT_CURRENCY = 'EUR'

# subject and body of the notification emails
gettext = lambda x: x
PRICE_MONITOR_I18N_EMAIL_NOTIFICATION_SUBJECT = gettext(
    'Price limit for %(product)s reached'
)
PRICE_MONITOR_I18N_EMAIL_NOTIFICATION_BODY = gettext(
    'The price limit of %(price_limit)0.2f %(currency)s has been reached for the '
    'article "%(product_title)s" - the current price is %(price)0.2f %(currency)s.'
    '\n\nPlease support our platform by using this '
    'link for buying: %(link)s\n\n\nRegards,\nThe Team'
)

# name of the site in notifications
PRICE_MONITOR_SITENAME = 'Price Monitor'Caching# key of cache (according to project config) to use for graphs
# None disables caching.
PRICE_MONITOR_GRAPH_CACHE_NAME = None

# prefix for cache key used for graphs
PRICE_MONITOR_GRAPH_CACHE_KEY_PREFIX = 'graph_'Celery settingsTo be able to run the required Celery tasks, Celery itself has to be set
up. Please see theCelery
Documentationabout how to setup the whole thing. You’ll need a broker and a result
backend configured.Development setup with DockerThe package comes with an easy to use Docker setup - you just needdockeranddocker-compose.
The setup is nearly similar to the one oftreasury( a project bydarignac), you
can read thedocumentationthere to get a better insight.StructureThere are 5 containers:dbPostgres databaseredisCelery brokerweba django project containing thedjango-amazon-price-monitorpackagecelerythe celery for the django projectdatacontainer for mounted volumesThewebandcelerycontainers are using a docker image being set up underdocker/web.Image: baseBasic image with all necessary system packages and pre-installedlxmlandpsycopg2.
The image can be found onDocker Hub.Image: webIt comes with a Django project with login/logout view, that can be found underdocker/web/project.
The image derives frompricemonitor/basefrom above.The directory structure within the container is the following (base dir:/srv/):root:/srv tree
├── logs                [log files]
├── media               [media files]
├── project             [the django project]
├── static              [static files]
└── pricemonitor        [the pricemonitor package]Starts via the start scriptdocker/web/web_run.shthat does migrations and the starts therunserver.Image: celeryBasically the same asweb, but starts the Celery worker with beat.If you want to develop anything involving tasks, see theUsagesection below.Image: dataThedatacontainer mounts several paths:Folder in containerFolder on hostInformation/var/lib/postgresql/data<PROJECTROOT>/docker/postgresPostgres data directoryKeeps the DB data even if container is removed/srv/logs<PROJECTROOT>/docker/logsDjango logs (see project settings)/srv/media<PROJECTROOT>/docker/mediaDjango media files/srv/project<PROJECTROOT>/docker/web/projectthe Django projectis copied on Dockerfile to get it up and runningthen mounted over (the copy is overwritten)/srv/pricemonitor<PROJECTROOT>thedjango-amazon-price-monitorlibis copied on Dockerfile to get it up and runningthen mounted over (the copy is overwritten)UsageOverride settingsTo override some settings as well as to set up therequired AWS settingsyou can create adocker-compose.override.ymland fill with the specific values
(also seedocker-compose documentation).Please see or adjust thedocker\web\project\settings.pyfor all settings that are read from the environment. They can be overwritten.A sampledocker-compose.override.ymlfile could look like this:version: '2'
services:
  celery:
        command: /bin/true
        environment:
          PRICE_MONITOR_AWS_ACCESS_KEY_ID: XXX
          PRICE_MONITOR_AWS_SECRET_ACCESS_KEY: XXX
          PRICE_MONITOR_AMAZON_PRODUCT_API_REGION: DE
          PRICE_MONITOR_AMAZON_PRODUCT_API_ASSOC_TAG: XXX
          PRICE_MONITOR_AMAZON_PRODUCT_REFRESH_THRESHOLD_MINUTES: 5
          PRICE_MONITOR_SUBSCRIPTION_RENOTIFICATION_MINUTES: 60It will avoid the automatic startup of celery (command: /bin/true) and set the required settings for AWS (in fact they are only needed in the celery
container). You can then manually start the container and execute celery which is quite useful if you develop anything that includes changes in the tasks and
thus requires the celery to be restarted (execute from thedockerfolder!):alex@tyrion:~/projects/github/django-amazon-price-monitor/docker$ docker-compose run celery bash
Starting docker_data_1


# check environment variables

root@9d64bbd23e98:/srv/project# env
HOSTNAME=9d64bbd23e98
EMAIL_BACKEND=django.core.mail.backends.filebased.EmailBackend
POSTGRES_DB=pm_db
TERM=xterm
PYTHONUNBUFFERED=1
PRICE_MONITOR_SUBSCRIPTION_RENOTIFICATION_MINUTES=60
POSTGRES_PASSWORD=6i2vmzq5C6BuSf5k33A6tmMSHwKKv0Pu
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
SECRET_KEY=Vceev7yWMtEQzHaTZX52
PWD=/srv/project
BROKER_URL=redis://redis/1
C_FORCE_ROOT='True'
PRICE_MONITOR_AWS_SECRET_ACCESS_KEY=XXX
POSTGRES_USER=pm_user
SHLVL=1
HOME=/root
PRICE_MONITOR_AMAZON_PRODUCT_REFRESH_THRESHOLD_MINUTES=5
PRICE_MONITOR_AMAZON_PRODUCT_API_REGION=DE
PRICE_MONITOR_AMAZON_PRODUCT_API_ASSOC_TAG=XXX
DEBUG='True'
PRICE_MONITOR_AWS_ACCESS_KEY_ID=XXX
_=/usr/bin/env


# start celery (worker and beat) (can also execute /srv/celery_run.sh)

root@9d64bbd23e98:/srv/project# celery --beat -A glue worker

 -------------- celery@9d64bbd23e98 v3.1.23 (Cipater)
---- **** -----
--- * ***  * -- Linux-3.16.0-4-amd64-x86_64-with-debian-8.0
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app:         glue:0x7fc6b5269e10
- ** ---------- .> transport:   redis://redis:6379/1
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 8 (prefork)
-- ******* ----
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery

[2016-03-20 10:02:26,776: WARNING/MainProcess] celery@9d64bbd23e98 ready.Start/Stop/BuildUse the make file to execute the most common tasks. It will executedocker-composewith the project namepmresulting in the container’s name patternpm_*.docker-build-base: - builds the base docker image
docker-build-web:  - builds the web docker image
docker-up:         - uses docker-compose to bring the containers up
docker-stop:       - uses docker-compose to stop the containers
docker-ps:         - runs docker-compose psA fixture with a Django useradminand the passwordpasswordis loaded automatically.TemplatesAs the fronted is done by Angular, there is only a single template with very limited possibilities to adjust,price_monitor/angular_index_view.html. You
can extends the template and adjust the following blocks.footerIs rendered on the very bottom of the page. You have to use Bootstrap compatible markup, e.g.:{% block footer %}
        <div class="row">
                <div class="col-md-12">Additonal footer</div>
        </div>
{% endblock %}Management Commandsprice_monitor_batch_create_productsA management command to batch create a number of products by providing
their ASIN:python manage.py price_monitor_batch_create_products <ASIN1> <ASIN2> <ASIN3>price_monitor_recreate_productRecreates a product with the given asin. If product already exists, it
is deleted.Only use in development!python manage.py price_monitor_recreate_product <ASIN>price_monitor_searchSearches for products at Amazon (not within the database) with the given
ASINs and prints out their details.python manage.py price_monitor_search <ASIN1> <ASIN2> ...Loggersprice_monitorThe app uses the logger “price_monitor” to log all error and info
messages that are not included within a dedicated other logger. Please
see theDjango logging
documentationfor how to setup loggers.price_monitor.product_advertising_apiLogger for everything related to the ProductAdvertisingAPI wrapper class
that accesses the Amazon Product Advertising API through bottlenose.price_monitor.utilsLogger for the utils module.TestsCoverageInternalsModel graphProduct advertising api synchronizationTask workflowImage of Product advertising api synchronization workflowChange Log0.7Features:footer can now be extended through template blockfooterproduct addition in frontend improved#79(PR#104)removedurlpatternsto please Django 1.10 deprecationadded docker setup for development (PR#101)list products with audience rating 18+ in notification mail if region is Germany and product is also 18+#92(PR#93)Bugfixes:now catching parsing errors of returned XML from Amazon API#96fixed date range of displayed prices in price graph#90fixed display of old prices of price graph#97updated to latestpython-dateutilversion, somehow refs#950.6.1Bugfixes:StartupTask fails with exception#94Tests fail if today is the last day of November#950.6Features:djangorestframework 3.2 compatibility#86(PR#88)Bugfixes:FindProductsToSynchronizeTask is rescheduled twice or more#89(PR#91)Unable to parse 2015-02 to a datetime#57lots of codestyleminor bugfixes0.5Features:Add link to PM frontend in notification email#76Django 1.9 support (seepull request #80)Bugfixes:FindProductsToSynchronizeTask is not always rescheduled#61Font files not included in package#75Identify as Amazon associate#77Pull requests:Ensured that FindProductsToSynchronizeTask will be scheduled#78(dArignac)Django 1.9 support#80(dArignac)0.4Features:Deprecate old frontend#73Make angular the default frontend#70Bugfixes:Products with the same price over graph timespae have an empty graph#67Notification of music albums#33Add artist for audio products#71Pull requests:Remove old frontend#74(dArignac)Fix for empty graphs is packaged now #67#72(mmrose)0.3b2Features:Prepare for automatic releases#68Increase performance of Amazon calls#41Django 1.8 compatibility#32Data reduction and clean up#27Limit graphs#26Show highest and lowest price ever#25Implement a full-usable frontend`#8 <https://github.com/ponyriders/django-amazon-price-monitor/issues/8>`__Add more tests#2Bugfixes:Graphs empty for some products#65Don’t show other peoples price limits#63Graphs do not render correct values#60‘NoneType’ object has no attribute ‘url’#59Rename SynchronizeSingleProductTask#56Sync on product creation not working#55Clear old products and prices#47Deleting a product subscription does not remove it from list view#42Endless synchronization queue#38Mark unavailable products#14Closed issues:Unpin beautifulsoup4==4.3.2#50Pull requests:fixed access of unavilable image urls #59#66(dArignac)63 subscriptions of other users#64(mmrose)Mark unavailable products#62(mmrose)Sync on product creation not working#58(dArignac)Products are now requeried after deletion in list view #42#54(mmrose)Show highest and lowest price (#25)#53(mmrose)Now the new FKs are also set during sync #25#52(mmrose)Adding datamigration for new min, max and current price FKs #25#51(mmrose)Performance improvements on product API view#49(mmrose)Remove unused data`#48 <https://github.com/ponyriders/django-amazon-price-monitor/pull/48>`__ (dArignac)Amazon query performance increase#46(dArignac)Django 1.8 compatibility#45(dArignac)Bugfix: Endless queue#40(dArignac)waffle.io Badge#37(waffle-iron)Pre-Releasesunfortunately everything before was not packaged and released nor tracked.