internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: liberapay/liberapay.com

Issue:
Email addresses aren't cleaned up
One user apparently input their email address with a trailing space, and since we don't strip those the address was rejected by MangoPay's system when the user filled the identity form. Sentry: https://sentry.changaco.oy.lc/share/issue/322e3936/


Discussion:
Note: the user did receive the verification email, so this doesn't affect email delivery, only MangoPay account creation.


Resolving patch:
diff --git a/liberapay/models/participant.py b/liberapay/models/participant.py
--- a/liberapay/models/participant.py
+++ b/liberapay/models/participant.py
@@ -564,6 +564,11 @@ def add_email(self, email, cursor=None):
             Returns the number of emails sent.
         """
 
+        # normalize the address: strip it, and lowercase the domain name
+        email = email.strip()
+        i = email.rfind('@')
+        email = email[:i] + email[i:].lower()
+
         if not EMAIL_RE.match(email):
             raise BadEmailAddress(email)
 


Test patch:
diff --git a/tests/py/test_email.py b/tests/py/test_email.py
--- a/tests/py/test_email.py
+++ b/tests/py/test_email.py
@@ -182,6 +182,10 @@ def test_cannot_add_too_many_emails(self):
         with self.assertRaises(TooManyEmailAddresses):
             self.alice.add_email('alice@example.coop')
 
+    def test_email_addresses_are_normalized(self):
+        self.alice.add_email('\t Alice." and Bob"@ExAmPlE.InFo \n')
+        assert self.alice.get_any_email() == 'Alice." and Bob"@example.info'
+
     def test_emails_page_shows_emails(self):
         self.verify_and_change_email('alice@example.com', 'alice@example.net')
         body = self.client.GET("/alice/emails/", auth_as=self.alice).text
Repository: liberapay/liberapay.com

Issue:
Logging in has become more difficult
In #421 I optimized the sign-in template for new users, putting the sign-up form on top. This results in the sign-in page (<https://liberapay.com/sign-in>) impeding users that are simply trying to log in.

The solution is simple: use two separate templates.


Discussion:
Related article: [Why ‘Sign Up’ and ‘Sign In’ Button Labels Confuse Users](http://uxmovement.com/buttons/why-sign-up-and-sign-in-button-labels-confuse-users/) (from https://github.com/gratipay/gratipay.com/pull/4269#issuecomment-269891953).

Resolving patch:
diff --git a/liberapay/utils/__init__.py b/liberapay/utils/__init__.py
--- a/liberapay/utils/__init__.py
+++ b/liberapay/utils/__init__.py
@@ -68,6 +68,7 @@ def get_participant(state, restrict=True, redirect_stub=True, allow_member=False
     if status == 'closed':
         if user.is_admin:
             return participant
+        state['closed_account'] = participant
         response.html_template = 'templates/account-closed.html'
         raise response.error(410)
     elif status == 'stub':


Test patch:
diff --git a/tests/py/test_sign_in.py b/tests/py/test_sign_in.py
--- a/tests/py/test_sign_in.py
+++ b/tests/py/test_sign_in.py
@@ -167,7 +167,7 @@ def test_email_login(self):
     def test_email_login_bad_email(self):
         data = {'log-in.id': 'unknown@example.org'}
         r = self.client.POST('/sign-in', data, raise_immediately=False)
-        assert r.code == 403
+        assert r.code != 302
         assert SESSION not in r.headers.cookie
         Participant.dequeue_emails()
         assert not self.get_emails()
Repository: liberapay/liberapay.com

Issue:
Cannot name community "C++"
I expect that to be a valid name. Bug or feature?


Discussion:
"+" is currently not allowed in community names. I guess it wouldn't be too problematic to allow it.

Resolving patch:
diff --git a/liberapay/models/community.py b/liberapay/models/community.py
--- a/liberapay/models/community.py
+++ b/liberapay/models/community.py
@@ -7,7 +7,7 @@
 
 
 name_maxlength = 40
-name_allowed_chars_pattern = r'\w\.-'
+name_allowed_chars_pattern = r'\w\.\+-'
 name_pattern = r'^[%s]{1,%s}$' % (name_allowed_chars_pattern, name_maxlength)
 name_re = re.compile(name_pattern, re.U)
 normalize_re = re.compile(r'[^%s]+' % name_allowed_chars_pattern, re.U)
diff --git a/liberapay/wireup.py b/liberapay/wireup.py
--- a/liberapay/wireup.py
+++ b/liberapay/wireup.py
@@ -373,12 +373,12 @@ def load_i18n(canonical_host, canonical_scheme, project_root, tell_sentry):
             tell_sentry(e, {})
 
     # Prepare a unique and sorted list for use in the language switcher
-    percent = lambda l: sum((percent(s) if isinstance(s, tuple) else 1) for s in l if s) / len(l)
+    percent = lambda l, total: sum((percent(s, len(s)) if isinstance(s, tuple) else 1) for s in l if s) / total
     for l in locales.values():
         if l.language == 'en':
             l.completion = 1
             continue
-        l.completion = percent([m.string for m in l.catalog if m.id])
+        l.completion = percent([m.string for m in l.catalog if m.id and not m.fuzzy], len(l.catalog))
     loc_url = canonical_scheme+'://%s.'+canonical_host
     lang_list = sorted(
         (


Test patch:
diff --git a/tests/py/test_communities.py b/tests/py/test_communities.py
--- a/tests/py/test_communities.py
+++ b/tests/py/test_communities.py
@@ -14,11 +14,11 @@ def setUp(self):
 
         # Alice joins a community.
         self.alice = self.make_participant('alice', balance=100)
-        c = self.alice.create_community('something')
+        c = self.alice.create_community('C++')
         self.alice.update_community_status('memberships', True, c.id)
 
     def test_community_member_shows_up_on_community_listing(self):
-        html = self.client.GET('/for/something/').text
+        html = self.client.GET('/for/C++/').text
         assert html.count('alice') == 2  # entry in New Participants
Repository: liberapay/liberapay.com

Issue:
Dots aren't allowed in usernames
Should they be? Someone just asked about this in IRC.


Discussion:
Just a small clarification, my question was about dots in teams/organizations names, not user names. But maybe the restriction is global to any type of account?
All account types are in the same namespace and share the same rules. Only community names have a different set of allowed characters.

Resolving patch:
diff --git a/liberapay/constants.py b/liberapay/constants.py
--- a/liberapay/constants.py
+++ b/liberapay/constants.py
@@ -40,7 +40,7 @@ def check_bits(bits):
 ASCII_ALLOWED_IN_USERNAME = set("0123456789"
                                 "abcdefghijklmnopqrstuvwxyz"
                                 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
-                                "-_")
+                                "-_.")
 
 AVATAR_QUERY = '?s=160&default=retro'
 AVATAR_SOURCES = 'libravatar bitbucket facebook github google mastodon twitter'.split()
diff --git a/liberapay/exceptions.py b/liberapay/exceptions.py
--- a/liberapay/exceptions.py
+++ b/liberapay/exceptions.py
@@ -89,6 +89,10 @@ class UsernameAlreadyTaken(ProblemChangingUsername):
     def msg(self, _):
         return _("The username '{0}' is already taken.", *self.args)
 
+class UsernameBeginsWithRestrictedCharacter(ProblemChangingUsername):
+    def msg(self, _):
+        return _("The username '{0}' begins with a restricted character.", *self.args)
+
 
 class ProblemChangingEmail(LazyResponse400): pass
 
diff --git a/liberapay/models/participant.py b/liberapay/models/participant.py
--- a/liberapay/models/participant.py
+++ b/liberapay/models/participant.py
@@ -39,6 +39,7 @@
     TooManyEmailAddresses,
     UserDoesntAcceptTips,
     UsernameAlreadyTaken,
+    UsernameBeginsWithRestrictedCharacter,
     UsernameContainsInvalidCharacters,
     UsernameIsEmpty,
     UsernameIsRestricted,
@@ -1275,6 +1276,9 @@ def change_username(self, suggested, cursor=None):
         if set(suggested) - ASCII_ALLOWED_IN_USERNAME:
             raise UsernameContainsInvalidCharacters(suggested)
 
+        if suggested[0] == '.':
+            raise UsernameBeginsWithRestrictedCharacter(suggested)
+
         lowercased = suggested.lower()
 
         if lowercased in website.restricted_usernames:


Test patch:
Repository: liberapay/liberapay.com

Issue:
Null bytes aren't handled properly
Original report: https://hackerone.com/reports/262852 | Gratipay's fix: https://github.com/gratipay/gratipay.com/pull/4582

I can't reproduce the XSS in Liberapay, however we do have the same problem of null bytes not being handled properly in DB queries. That's <https://github.com/psycopg/psycopg2/issues/420>, which was fixed a year ago. Thus what we need to do is upgrade psycopg2.

cc @EdOverflow


Discussion:

Resolving patch:
diff --git a/liberapay/wireup.py b/liberapay/wireup.py
--- a/liberapay/wireup.py
+++ b/liberapay/wireup.py
@@ -298,6 +298,14 @@ def tell_sentry(exception, state, allow_reraise=True):
                     # Don't reraise this in tests
                     allow_reraise = False
 
+        if isinstance(exception, ValueError):
+            if 'cannot contain NUL (0x00) characters' in str(exception):
+                # https://github.com/liberapay/liberapay.com/issues/675
+                response = state.get('response') or pando.Response()
+                response.code = 400
+                response.body = str(exception)
+                return {'exception': None}
+
         if not sentry:
             # No Sentry, log to stderr instead
             traceback.print_exc()


Test patch:
diff --git a/tests/py/test_state_chain.py b/tests/py/test_state_chain.py
--- a/tests/py/test_state_chain.py
+++ b/tests/py/test_state_chain.py
@@ -218,3 +218,7 @@ def test_non_dict_body(self):
     def test_no_trailing_slash_redirects(self):
         r = self.client.GET('/foo', raise_immediately=False)
         assert r.code == 404, r.text
+
+    def test_null_byte_results_in_400(self):
+        r = self.client.GET('/foo%00', raise_immediately=False)
+        assert r.code == 400, r.text
Repository: liberapay/liberapay.com

Issue:
addresses/address
When going to https://liberapay.com/ACCOUNT/emails/ , I'm shown with :

![liberapay-adresses-courriel](https://user-images.githubusercontent.com/8705846/30387658-8101c6e6-98ad-11e7-89d8-ee01b8bd3822.png)

It's strange to have a plural form when having only one address which I guess is the case for most of the users. First time I saw that, I thought "Is it compulsory to have multiple addresses ?".

I do acknowledge that a better solution than https://github.com/liberapay/liberapay.com/pull/693 could be find. Unfortunately, I don't know how to do so.

Maybe a check could be done to see if there is one or more address/es and then display the correct string. I'm quite confident I already saw something like that in software translation po files (something call "plural form" IIRC). Not sure if that can be done here, depending on the translation system used by liberapay.



Discussion:

Resolving patch:
diff --git a/liberapay/utils/i18n.py b/liberapay/utils/i18n.py
--- a/liberapay/utils/i18n.py
+++ b/liberapay/utils/i18n.py
@@ -4,6 +4,7 @@
 from collections import namedtuple, OrderedDict
 from datetime import date, datetime, timedelta
 from decimal import Decimal, InvalidOperation
+from hashlib import md5
 from io import BytesIO
 import re
 from unicodedata import combining, normalize
@@ -316,6 +317,24 @@ def getdoc(name):
     context['getdoc'] = getdoc
 
 
+def extract_custom(extractor, *args, **kw):
+    for match in extractor(*args, **kw):
+        msg = match[2]
+        if isinstance(msg, tuple) and msg[0] == '':
+            unused = "<unused singular (hash=%s)>" % md5(msg[1]).hexdigest()
+            msg = (unused, msg[1], msg[2])
+            match = (match[0], match[1], msg, match[3])
+        yield match
+
+
+def extract_jinja2_custom(*args, **kw):
+    return extract_custom(jinja2.ext.babel_extract, *args, **kw)
+
+
+def extract_python_custom(*args, **kw):
+    return extract_custom(extract_python, *args, **kw)
+
+
 def extract_spt(fileobj, *args, **kw):
     pages = list(split_and_escape(fileobj.read().decode('utf8')))
     npages = len(pages)
@@ -326,9 +345,9 @@ def extract_spt(fileobj, *args, **kw):
         python_page = i < 3 and i < npages and not page.header
         json_page = renderer in ('json_dump', 'jsonp_dump')
         if python_page or json_page:
-            extractor = extract_python
+            extractor = extract_python_custom
         else:
-            extractor = jinja2.ext.babel_extract
+            extractor = extract_jinja2_custom
         if extractor:
             for match in extractor(f, *args, **kw):
                 yield match


Test patch:
Repository: liberapay/liberapay.com

Issue:
The list of top individuals is incomplete
While looking at https://liberapay.com/explore/individuals I realized that ploum isn't listed. It's because he doesn't have a profile statement. The thinking was that without a statement there isn't much to see on a profile page, so there's little point in linking to it. However it also makes the list incomplete.
The list of top individuals is incomplete
While looking at https://liberapay.com/explore/individuals I realized that ploum isn't listed. It's because he doesn't have a profile statement. The thinking was that without a statement there isn't much to see on a profile page, so there's little point in linking to it. However it also makes the list incomplete.


Discussion:

Resolving patch:
diff --git a/liberapay/constants.py b/liberapay/constants.py
--- a/liberapay/constants.py
+++ b/liberapay/constants.py
@@ -232,6 +232,8 @@ def make_standard_tip(label, weekly):
     make_standard_tip(_("Maximum"), DONATION_WEEKLY_MAX),
 )
 
+SUMMARY_MAX_SIZE = 100
+
 USERNAME_MAX_SIZE = 32
 
 del _


Test patch:
Repository: liberapay/liberapay.com

Issue:
Email addresses with non-ascii characters are rejected
https://twitter.com/transposees/status/923842688948101121


Discussion:

Resolving patch:
diff --git a/liberapay/models/participant.py b/liberapay/models/participant.py
--- a/liberapay/models/participant.py
+++ b/liberapay/models/participant.py
@@ -704,10 +704,10 @@ def add_email(self, email, cursor=None):
             Returns the number of emails sent.
         """
 
-        # normalize the address: strip it, and lowercase the domain name
+        # normalize the address: strip it, then lowercase and encode the domain name
         email = email.strip()
-        i = email.rfind('@')
-        email = email[:i] + email[i:].lower()
+        i = email.rfind('@') + 1
+        email = email[:i] + email[i:].lower().encode('idna').decode()
 
         if not EMAIL_RE.match(email):
             raise BadEmailAddress(email)


Test patch:
diff --git a/tests/py/test_email.py b/tests/py/test_email.py
--- a/tests/py/test_email.py
+++ b/tests/py/test_email.py
@@ -1,3 +1,5 @@
+# -*- coding: utf-8 -*-
+
 from __future__ import absolute_import, division, print_function, unicode_literals
 
 from liberapay.exceptions import (
@@ -46,6 +48,15 @@ def test_participant_can_add_email(self):
         response = self.hit_email_spt('add-email', 'alice@example.com')
         assert response.text == '{}'
 
+    def test_participant_can_add_email_with_unicode_domain_name(self):
+        punycode_email = 'alice@' + 'accentué.com'.encode('idna').decode()
+        self.hit_email_spt('add-email', 'alice@accentué.com')
+        assert self.alice.get_any_email() == punycode_email
+
+    def test_participant_cannot_add_email_with_unicode_local_part(self):
+        r = self.hit_email_spt('add-email', 'tête@exemple.fr', should_fail=True)
+        assert r.code == 400
+
     def test_participant_cant_add_bad_email(self):
         bad = (
             'a\nb@example.net',
@@ -138,6 +149,14 @@ def test_verify_email(self):
         actual = Participant.from_username('alice').email
         assert expected == actual
 
+    def test_verify_email_with_unicode_domain(self):
+        punycode_email = 'alice@' + 'accentué.fr'.encode('idna').decode()
+        self.alice.add_email(punycode_email)
+        nonce = self.alice.get_email(punycode_email).nonce
+        self.hit_verify(punycode_email, nonce)
+        actual = Participant.from_username('alice').email
+        assert punycode_email == actual
+
     def test_email_verification_is_backwards_compatible(self):
         """Test email verification still works with unencoded email in verification link.
         """
Repository: liberapay/liberapay.com

Issue:
Pricing change for direct debits
From Mangopay's end-of-month newsletter:

> We have updated our pricing to be even more competitive, and let's be honest, who doesn't love better pricing! For cash-in our SEPA Direct Debit is now €0.50 (€0.80 before), and the BACS Direct Debit is now priced at £0.50 (£0.80 before). No worries, no action needed from your side. We have applied this pricing to all our clients since the 1st of November.

So we've been over-charging donors for a month and we'll likely have a significant surplus in November.



Discussion:
The amount at which direct debit becomes the cheapest method of payment will thus be lowered from 160 to 100 euros.

Resolving patch:
diff --git a/liberapay/constants.py b/liberapay/constants.py
--- a/liberapay/constants.py
+++ b/liberapay/constants.py
@@ -128,8 +128,8 @@ def with_vat(self):
     'USD': Fees(Decimal('0.025'), Money('0.30', 'USD')),  # 2.5% + $0.30
 }
 FEE_PAYIN_DIRECT_DEBIT = {
-    'EUR': Fees(0, Money('0.80', 'EUR')),  # €0.80
-    'GBP': Fees(0, Money('0.80', 'GBP')),  # £0.80
+    'EUR': Fees(0, Money('0.50', 'EUR')),  # €0.50
+    'GBP': Fees(0, Money('0.50', 'GBP')),  # £0.50
 }
 FEE_PAYOUT = {
     'EUR': {
@@ -208,12 +208,12 @@ def with_vat(self):
     'USD': Money('95.00', 'USD'),  # fee ≈ 3.27%
 }
 PAYIN_CARD_MAX = {k: Money('2500.00', k) for k in ('EUR', 'USD')}
-PAYIN_DIRECT_DEBIT_MIN_EUR_GBP = Decimal('25.00')  # fee ≈ 3.6%
+PAYIN_DIRECT_DEBIT_MIN_EUR_GBP = Decimal('15.00')  # fee ≈ 3.78%
 PAYIN_DIRECT_DEBIT_MIN = {
     'EUR': Money(PAYIN_DIRECT_DEBIT_MIN_EUR_GBP, 'EUR'),
     'GBP': Money(PAYIN_DIRECT_DEBIT_MIN_EUR_GBP, 'GBP'),
 }
-PAYIN_DIRECT_DEBIT_TARGET_EUR_GBP = Decimal('99.00')  # fee ≈ 0.94%
+PAYIN_DIRECT_DEBIT_TARGET_EUR_GBP = Decimal('99.00')  # fee ≈ 0.59%
 PAYIN_DIRECT_DEBIT_TARGET = {
     'EUR': Money(PAYIN_DIRECT_DEBIT_TARGET_EUR_GBP, 'EUR'),
     'GBP': Money(PAYIN_DIRECT_DEBIT_TARGET_EUR_GBP, 'GBP'),


Test patch:
Repository: liberapay/liberapay.com

Issue:
Can't cancel a bank wire payin
Users who initiate a bank wire payin can't cancel it if they change their mind or realize that their bank doesn't allow them to transfer euros abroad.

It's not possible to cancel a bank wire payin on MangoPay, which is why we don't already have this feature, but we should be able to implement a fake cancellation on our side:
- ~~[ ] add a `canceled` status to `exchanges` (there will be some code to modify, e.g. `git grep "'failed'"`)~~
- [x] add a "Cancel" button to the bankwire page which changes the exchange status to `failed` with "Canceled by the user" in the `note` column

~~Later, maybe:~~
- ~~[ ] identify bank wires that should have been canceled and change their status~~
- ~~[ ] send the list of all canceled bank wires to MangoPay so they can manually mark them as failed on their side~~

*Edit: no longer necessary now that bank wire payins expire automatically after a month. However we'll need something else instead:*

- [ ] ~~ignore mangopay's expiration notifications for "canceled" bank wires~~ *not necessary if we use the `failed` status instead of creating a `canceled` one*

Mangopay account switch can mess up the wallets table
Which in turn can crash payday: <https://github.com/liberapay/salon/issues/181#issuecomment-347861123>.


Discussion:

Resolving patch:
diff --git a/liberapay/billing/transactions.py b/liberapay/billing/transactions.py
--- a/liberapay/billing/transactions.py
+++ b/liberapay/billing/transactions.py
@@ -251,6 +251,10 @@ def payin_bank_wire(db, participant, debit_amount):
     return payin, e
 
 
+def cancel_bank_wire_payin(db, exchange, payin, participant):
+    record_exchange_result(db, exchange.id, payin.Id, 'failed', "canceled", participant)
+
+
 def record_unexpected_payin(db, payin):
     """Record an unexpected bank wire payin.
     """


Test patch:
Repository: liberapay/liberapay.com

Issue:
Wording is not clear enough when adding money
https://twitter.com/josh_cheek/status/921716393766211584


Discussion:
I am interested in this, but I have been looking around trying to find where the error messages are stored and they seem to be in a database somewhere which I cannot find.  Could you lead me in the right direction?
Actually this issue isn't about the error message, it's about the wording of the page before that, which apparently doesn't make it clear that the payment is attempted immediately and in full, as opposed to a weekly charge.

Resolving patch:
diff --git a/liberapay/constants.py b/liberapay/constants.py
--- a/liberapay/constants.py
+++ b/liberapay/constants.py
@@ -219,6 +219,17 @@ def with_vat(self):
 }
 PAYIN_DIRECT_DEBIT_MAX = {k: Money('2500.00', k) for k in ('EUR', 'USD')}
 
+PAYMENT_METHODS = {
+    'mango-ba': _("Direct Debit"),
+    'mango-bw': _("Bank Wire"),
+    'mango-cc': _("Credit Card"),
+}
+PAYMENT_SLUGS = {
+    'mango-ba': 'direct-debit',
+    'mango-bw': 'bankwire',
+    'mango-cc': 'card',
+}
+
 PERIOD_CONVERSION_RATES = {
     'weekly': Decimal(1),
     'monthly': Decimal(12) / Decimal(52),
diff --git a/liberapay/models/participant.py b/liberapay/models/participant.py
--- a/liberapay/models/participant.py
+++ b/liberapay/models/participant.py
@@ -23,7 +23,8 @@
     ASCII_ALLOWED_IN_USERNAME, AVATAR_QUERY,
     DONATION_LIMITS, EMAIL_RE,
     EMAIL_VERIFICATION_TIMEOUT, EVENTS,
-    PASSWORD_MAX_SIZE, PASSWORD_MIN_SIZE, PERIOD_CONVERSION_RATES, PRIVILEGES,
+    PASSWORD_MAX_SIZE, PASSWORD_MIN_SIZE, PAYMENT_SLUGS,
+    PERIOD_CONVERSION_RATES, PRIVILEGES,
     SESSION, SESSION_REFRESH, SESSION_TIMEOUT, USERNAME_MAX_SIZE, ZERO
 )
 from liberapay.exceptions import (
@@ -1237,6 +1238,13 @@ def url(self, path='', query=''):
             query = '?' + urlencode(query)
         return '{scheme}://{host}/{username}/{path}{query}'.format(**locals())
 
+    def get_payin_url(self, network, e_id):
+        path = 'wallet/payin/%s' % PAYMENT_SLUGS[network]
+        if network == 'mango-ba':
+            return self.url(path + '/%s' % e_id)
+        else:
+            return self.url(path, dict(exchange_id=e_id))
+
     def get_teams(self):
         """Return a list of teams this user is a member of.
         """


Test patch:
Repository: liberapay/liberapay.com

Issue:
Incomplete translation warning
It's not good that we display partially translated pages to users, but what's worse is that we don't display a warning about the translation being incomplete.


Discussion:

Resolving patch:
diff --git a/liberapay/utils/i18n.py b/liberapay/utils/i18n.py
--- a/liberapay/utils/i18n.py
+++ b/liberapay/utils/i18n.py
@@ -261,6 +261,8 @@ def i_format(loc, s, *a, **kw):
                 c[k] = format_datetime(o, locale=loc)
             elif isinstance(o, date):
                 c[k] = format_date(o, locale=loc)
+            elif isinstance(o, Locale):
+                c[k] = loc.languages.get(o.language) or o.language.upper()
             if wrapper:
                 c[k] = wrapper % (c[k],)
     return s.format(*a, **kw)


Test patch: