Repository: oppia/oppia

Issue:
BadValueError: Entity has uninitialized properties: role
I noticed this with a version of develop that's about 7 days old. This doesn't seem to happen on the local dev server, but it does repro in our test prod server. Are we not properly migrating something to a new data model? I didn't investigate beyond looking at the trace.

Trace:
```
Traceback (most recent call last): (/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/controllers/base.py:479)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/controllers/library.py", line 203, in get
    splash_page_featured_exploration_ids))
  File "/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/domain/summary_services.py", line 327, in get_displayable_exp_summary_dicts_matching_ids
    return get_displayable_exp_summary_dicts(filtered_exploration_summaries)
  File "/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/domain/summary_services.py", line 389, in get_displayable_exp_summary_dicts
    exploration_summary.contributors_summary)
  File "/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/domain/summary_services.py", line 81, in get_human_readable_contributors_summary
    contributor_ids)
  File "/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/domain/user_services.py", line 815, in get_human_readable_user_ids
    users_settings = get_users_settings(user_ids)
  File "/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/domain/user_services.py", line 351, in get_users_settings
    user_settings_models = user_models.UserSettingsModel.get_multi(user_ids)
  File "/base/data/home/apps/2-5-0-purge-answers.402025613165200876/core/storage/base_model/gae_models.py", line 107, in get_multi
    entities = ndb.get_multi(entity_keys)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3931, in get_multi
    for future in get_multi_async(keys, **ctx_options)]
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 430, in _help_tasklet_along
    value = gen.send(val)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 765, in get
    pbs = entity._to_pb(set_key=False).SerializePartialToString()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3158, in _to_pb
    self._check_initialized()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3014, in _check_initialized
    'Entity has uninitialized properties: %s' % ', '.join(baddies))
BadValueError: Entity has uninitialized properties: role
```


Discussion:
Oh oh. @1995YogeshSharma -- did we not specify a default value for `role`?

/cc @wxyxinyu in case this needs a hotfix.
OK -- doesn't need a hotfix (Xinyu, is this right? Did this change go in the release?) but there is an important omission -- we do need a default arg for UserSettingsModel in core.storage.gae_models.user otherwise old models will not load. @1995YogeshSharma could you please fix this?
Confirming this doesn't need a hotfix, since the changes weren't part of the most recent release. We do still need to make the default value change, though, and this is a must-fix before the next release.

@1995YogeshSharma -- apologies for missing this during review earlier!
@seanlip  @BenHenning .....
 
Correct me if I am wrong.
The problem is not related to "not having default value for role in gae_models" since in such cases gae gives a value of None to the property, the actual problem is "having role as required=True without having a default value".

But, I tried to replicate the error by using following process:- 
1.) Checkout a commit from develop before the introduction of any role thing.
2.) Create a sample set of users.
3.) Go to config tab and add them to admin_usernames, moderator_usernames etc.
4.) Checkout a branch with everything related to role (including milestone 1.1, 1.2, 1.3). Since, 1.2 and 1.3 are not merged, this branch exist on my machine only.
5.) Run the migration job.
6.) Verify results.
I got expected results from this without such error.

So, I can think of two things we can do (but can't verify as I don't get error):-
1.) Remove the required=True for now and add it after migration job has run.
2.) Give a default role for now and remove it after migration job has run.
I think the issue came from using the site without running the migration
job. Have you tried that repro?

On Sat, Jun 17, 2017 at 10:49 AM Yogesh Sharma <notifications@github.com>
wrote:

> @seanlip <https://github.com/seanlip> @BenHenning
> <https://github.com/benhenning> .....
>
> Correct me if I am wrong.
> The problem is not related to "not having default value for role in
> gae_models" since in such cases gae gives a value of None to the property,
> the actual problem is "having role as required=True without having a
> default value".
>
> But, I tried to replicate the error by using following process:-
> 1.) Checkout a commit from develop before the introduction of any role
> thing.
> 2.) Create a sample set of users.
> 3.) Go to config tab and add them to admin_usernames, moderator_usernames
> etc.
> 4.) Checkout a branch with everything related to role (including milestone
> 1.1, 1.2, 1.3). Since, 1.2 and 1.3 are not merged, this branch exist on my
> machine only.
> 5.) Run the migration job.
> 6.) Verify results.
> I got expected results from this without such error.
>
> So, I can think of two things we can do (but can't verify as I don't get
> error):-
> 1.) Remove the required=True for now and add it after migration job has
> run.
> 2.) Give a default role for now and remove it after migration job has run.
>
> —
> You are receiving this because you were mentioned.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/oppia/oppia/issues/3553#issuecomment-309229911>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AMYdvmo2R28uPwZjclkHP_a7-chP7knHks5sFBGSgaJpZM4N9FrN>
> .
>

@BenHenning ... ok, I got the error. Thanks.
Not the same one but still. Adding the default value does solve the error.

@seanlip .... should I go ahead and add feconf.ROLE_EXPLORATION_EDITOR as default value? I think it can be removed after migration?

If yes, should I do it in separate PR or one of the existing PRs ?
@1995YogeshSharma yes, I think everything you say sounds good. Separate PR makes sense too. Maybe add a TODO to say "remove default value after migration" or similar, and add it to your milestone task list.

Thanks!

Resolving patch:
diff --git a/core/storage/user/gae_models.py b/core/storage/user/gae_models.py
--- a/core/storage/user/gae_models.py
+++ b/core/storage/user/gae_models.py
@@ -33,8 +33,12 @@ class UserSettingsModel(base_models.BaseModel):
     """
     # Email address of the user.
     email = ndb.StringProperty(required=True, indexed=True)
-    # User role. Required for authorization.
-    role = ndb.StringProperty(required=True, indexed=True)
+    # User role. Required for authorization. User gets a default role of
+    # exploration editor.
+    # TODO (1995YogeshSharma): Remove the default value once the one-off
+    # migration (to give role to all users) is run.
+    role = ndb.StringProperty(
+        required=True, indexed=True, default=feconf.ROLE_ID_EXPLORATION_EDITOR)
     # Identifiable username to display in the UI. May be None.
     username = ndb.StringProperty(indexed=True)
     # Normalized username to use for duplicate-username queries. May be None.


Test patch: