birknilson/python-paypal-sdk — pull request #5 by parhamfh (2014-05-20 14:24:17+00:00)
Title: WIP: Add mapping between PayPal and Python char.encoding names 

I noticed that we have a lot of `CRITICAL` errors in the PayPal IPN log which simply seem to be due to the fact that PayPal and Python have different naming conventions as far as some character encodings go (mostly east-Asian it seems) so I thought I'd put this one out there and map all the supported PayPal encodings over to the equivalent name of the encoding in Python.

Some are not supported by Python, but they seem to be very old and almost obsolete so there is a very low likelihood of receiving IPNs with those encodings. Despite that, not quite sure about what to do.. today we just ignore crashing critical IPNs I think.. don't know the full extent of the consequences of that approach to be able to tell if that is a good approach to opt for in the case of not-supported encodings either :smile:

PayPal supported encodings (See table 2): https://developer.paypal.com/docs/classic/paypal-payments-standard/integration-guide/formbasics/
Python supported charsets: https://docs.python.org/2/library/codecs.html#standard-encodings

It also might be worth discussing why those decode failure errors are marked as `CRITICIAL`, is it worth the level, if so, why haven't we done anything sooner? There are a couple of these each day it seems.

```
parham bonsai.ops.production /var/log/tictail/paypal.ipn $ sudo grep -ci 'CRITICAL: IPN: Cannot decode given notification' 2014-05-1*
2014-05-10.log.gz:0
2014-05-11.log.gz:0
2014-05-12.log:3
2014-05-13.log:3
2014-05-14.log:5
2014-05-15.log:4
2014-05-16.log:2
2014-05-17.log:1
2014-05-18.log:4
2014-05-19.log:3
```

Here is an example of a log entry from todays IPN log:

```
May 20 09:36:32 tuborg paypal.ipn[6681]: CRITICAL: IPN: Cannot decode given notification: mc_gross=504.98&invoice=tictail%3A1576486%3A1400578575&protection_eligibility=Eligible&address_status=unconfirmed&payer_id=HDTUH36LHVKX8&tax=0.00&address_street=Building+manager+office+%2C+11+Victoria+park+PDE+%2CZetland&payment_date=02%3A36%3A18+May+20%2C+2014+PDT&payment_status=Completed&charset=windows-874&address_zip=2017&first_name=JIABAI&mc_fee=22.52&address_country_code=AU&address_name=Jiabai+Yu++533¬ify_version=3.8&custom=64718%3A1576486&payer_status=verified&address_country=Australia&address_city=Sydney&quantity=1&verify_sign=AftUST9b90cVq1URFnOhKfNg12l7AlWNcw54R-Rcm4sUwU0zy0fZhO5q&payer_email=yujiabai%40gmail.com&memo=37+both&txn_id=1GA32800XJ204250W&payment_type=instant&last_name=YU&address_state=Nsw&receiver_email=wrongsole%40gmail.com&payment_fee=22.52&receiver_id=353ZCTN24P8RS&txn_type=express_checkout&item_name=&mc_currency=USD&item_number=&residence_country=AU&handling_amount=0.00&transaction_subject=&payment_gross=504.98&shipping=45.00&ipn_track_id=93de83e561144
````
(using `windows-874` which Python simply calls `cp874`, which is a Thai charset)