Repository: LMFDB/lmfdb

Issue:
Server errors on many Maass form pages
Many Maass form pages are throwing internal server errors that show up in the flasklogs of the www.lmfdb.org server.  This is apparently due to some sort of data inconsistency -- the specified ids are object ids of records in MaassWaveForms/Coefficients (e.g. see http://www.lmfdb.org/api/MaassWaveForms/Coefficients/4f55571188aece241f00000d for the first example listed below), but the ObjectId listed in the coeff_id attribute of that record is not found.

There are at least 49 affected pages:

http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f55571188aece241f00000d
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5692ca88aece2afe000002
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5693de88aece2afe00000e
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56952f88aece2afe00001e
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5697fc88aece2afe00003e
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56984e88aece2afe000044
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56990188aece2afe00004d
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56990388aece2afe00004e
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56990888aece2afe00004f
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56990b88aece2afe000050
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56991588aece2afe000051
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56991f88aece2afe000053
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56992188aece2afe000054
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56998f88aece2afe00005e
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f56999c88aece2afe000060
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5699a288aece2afe000061
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5699a588aece2afe000062
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5699ad88aece2afe000063
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a1588aece2afe00006d
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a1c88aece2afe00006e
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a2288aece2afe00006f
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a2688aece2afe000070
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a2a88aece2afe000071
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a2e88aece2afe000072
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a3088aece2afe000073
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a3588aece2afe000074
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a3b88aece2afe000076
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a8488aece2afe000081
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a8788aece2afe000082
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a8c88aece2afe000083
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a8e88aece2afe000084
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a9088aece2afe000085
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a9288aece2afe000086
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569a9c88aece2afe000088
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569aa188aece2afe000089
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569aa388aece2afe00008a
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569aa488aece2afe00008b
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f569aaa88aece2afe00008c
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5929f388aece2afe0000d3
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d541f88aece4440000025
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d544288aece4440000026
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d546188aece4440000027
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d549888aece4440000028
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d54bc88aece4440000029
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d5a7488aece444000002a
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d5ab188aece444000002b
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d5aea88aece444000002c
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d5b5688aece444000002d
http://www.lmfdb.org/ModularForm/GL2/Q/Maass/4f5d5ba088aece444000002e



Discussion:
The error I get from the first of these, when run locally, is from the line dim = info['MF'].dim where info['MF'] has no field called 'dim'.  The fields it has are

['C', 'R', '**class**', '**delattr**', '**dict**', '**doc**', '**format**', '**getattribute**', '**hash**', '**init**', '**module**', '**new**', '**reduce**', '**reduce_ex**', '**repr**', '**setattr**', '**sizeof**', '**str**', '**subclasshook**', '**weakref**', '_db', '_maassid', 'atkinlehner', 'character', 'coeffs', 'download_text', 'even_odd', 'fricke', 'has_plot', 'level', 'next_maassform_id', 'precision', 'prev_maassform_id', 'set_table', 'symmetry', 'table', 'the_character', 'the_weight', 'weight']

In line 109 of mwf_classes.py the value of ff returned from db.get_Maass_forms(id=self._maassid) is [] and the constructor returns, without setting dim.  Perhaps dim should be set to 0 before returning?

I'm not sure setting dim to 0 is the right solution, this looks to me like an inconsistency in the data and it is probably better to either remove the records that are causing the problem or to have the code simply say that the form was not found if it can't find everything that it expects to be there.

Agreed.  The fault in the code is that the constructor sometimes _silently_ returns an incomplete instance of the class, but the function which calls the constructor is not checking for that.
To help catch this in future, then, we could (1)  output an error log whenever this premature exit to the constructor is triggered; (2) catch the occasions where a WebMaasForm object after being constructed has no 'dim' member and display a good error message for the user.

Returning incomplete instances of classes is a "feature" of all the code in lmfdb/modular_forms/.,.. which often causes problems (we have many similar issues with classical modular forms).  The constructor code is often so general that it doesn't actually know what should or should not be there in any particular case (this is a case where making the code too general makes it less useful).

Displaying a meaningful error message to the user (e.g. "unable to construct this Maass form blah, please report this problem") is certainly better than an internal server error (but it still leaves open the issue of fixing the underlying problem).

I should also mention a possibly related issue #1517, which is about warning messages being logged by the Maass forms code (but for what looks like a different reason).

@JohnCremona I will make a change along the lines above as an interim measure.  I also have a lot of import cleanup work in progress on the modular forms code, but I think I can do this separately without creating a merge conflict (but best if you don't change the mwf code right now).

OK, I won't. 

Once #1807 is pushed to prod this can be closed.

@JohnCremona Rather than closing it we should probably just rename it -- the 49 pages listed in the description still do not work, the only thing that has changed is that the user now gets a more informative error message.  My concern is that these pages are being accessed clicks on the plots or via next/prev links on pages that do work.

OK good point.


Resolving patch:
diff --git a/lmfdb/modular_forms/maass_forms/maass_waveforms/backend/mwf_classes.py b/lmfdb/modular_forms/maass_forms/maass_waveforms/backend/mwf_classes.py
--- a/lmfdb/modular_forms/maass_forms/maass_waveforms/backend/mwf_classes.py
+++ b/lmfdb/modular_forms/maass_forms/maass_waveforms/backend/mwf_classes.py
@@ -110,7 +110,7 @@ def __init__(self, db, maassid, **kwds):
         ff = db.get_Maass_forms(id=self._maassid)
         # print "ff=",ff
         if len(ff) == 0:
-            return
+            raise KeyError("massid %s not found in database"%maassid)
         f = ff[0]
 
         # print "f here=",f
diff --git a/lmfdb/modular_forms/maass_forms/maass_waveforms/views/mwf_main.py b/lmfdb/modular_forms/maass_forms/maass_waveforms/views/mwf_main.py
--- a/lmfdb/modular_forms/maass_forms/maass_waveforms/views/mwf_main.py
+++ b/lmfdb/modular_forms/maass_forms/maass_waveforms/views/mwf_main.py
@@ -145,7 +145,10 @@ def render_one_maass_waveform(maass_id, **kwds):
         info.get('download', '') == 'all'):
         DB = connect_db()
         maass_id = info['maass_id']
-        f = WebMaassForm(DB, maass_id)
+        try:
+            f = WebMaassForm(DB, maass_id)
+        except KeyError:
+            flask.abort(404)
         filename = str(f._maassid) + '.txt'
         if info.get('download', '') == 'coefficients':
             res = f.coeffs
@@ -177,6 +180,8 @@ def plot_maassform(maass_id):
         return flask.abort(404)
     DB = connect_db()
     data = DB.get_maassform_plot_by_id(maass_id)
+    if not data:
+        return flask.abort(404)
     data = data['plot']
     response = make_response(loads(data))
     response.headers['Content-type'] = 'image/png'
@@ -191,7 +196,11 @@ def render_one_maass_waveform_wp(info):
     DB = connect_db()
     maass_id = info['maass_id']
     mwf_logger.debug("id1={0}".format(maass_id))
-    info['MF'] = MF = WebMaassForm(DB, maass_id)
+    try:
+        MF = WebMaassForm(DB, maass_id)
+    except KeyError:
+        return flask.abort(404)
+    info['MF'] = MF
     info['title'] = "Maass form"
     info['bread'] = [('Modular forms', url_for('mf.modular_form_main_page')),
                      ('Maass waveforms', url_for('.render_maass_waveforms'))]


Test patch: