Repository: oppia/oppia

Issue:
Perform manual end to end testing of text input classifier
Currently, we have working Oppia-ml using which we can do end to end testing of our classifier. The issue is to do such testing (instructions given below) for text classifier.

Steps to do end to end testing:
* Start Oppia-ml locally.
* Start Oppia locally.
* Enable ML related constants in Oppia (feconf.ENABLE_ML_CLASSIFIERS, feconf.SHOW_TRAINABLE_UNRESOLVED_ANSWERS, app.js.ENABLE_ML_CLASSIFIERS)
* Create an exploration that uses text input interaction and create some sample training data. Alternatively, you can also load an existing (if it exists otherwise you have to create a new one) exploration that already which uses text input interaction and has training data preloaded.
* Once this new exploration is saved, check the logs of Oppia-ml and Oppia and make sure that Oppia-ml fetches the training data from Oppia and returns the trained model back to Oppia which is stored in the database by Oppia. You can also use Oppia's GAE admin dashboard and check database entities manually to make sure that new training job is created.
* Once the training data is stored successfully, test run the exploration and make sure that prediction API is running and returns expected predictions. For this purpose, I suggest you create an exploration that does not have any hard rules. So just getting a response after submitting an answer is sufficient for verification of prediction API and trained model.

This issue may or may not require a PR. If text input classifier runs the end to end manual test successfully then no PR will be needed but if there are any issues then PR (for Oppia or Oppia-ml) is required for fixing those issues in the code base.



Discussion:
Hi, @GanitGenius would you like to work on this issue?
@prasanna08 i did some investigation, following the above steps:

 - I loaded 'classifier_demo_exploration.yaml'

 - I didn't got the prediction for Text Input Classifier. i tried putting some of training_data but it didn't work.

 - I check the Datastore Viewer and there was no Entity like 'ClassifierTrainingJobModel' which suggests Training jobs were not stored. Is there any other way to store them??
 
 - On editing It gave some console errors and i found some typos like 'trainingDataService', 'responsesService' and 'angularNameService' were used instead their first letter should be capitalized.

 - I started editing the exploration and found training the classifier a bit messy/misleading. For example below, the feedback shows only for default case, is not editable. Also clicking "Yes" doesn't adds training data to any answer_group. Also i do not understand what 'Is this right? (No, Yes)' is for??

![screenshot from 2018-02-11 04-52-20](https://user-images.githubusercontent.com/22469265/36067684-078dd8b6-0ee8-11e8-84f1-e8f02580533d.png)
 
 - I think editor.UntrainedAnswersHandler is still in progress as [this](https://github.com/oppia/oppia/blob/develop/core/controllers/editor.py#L561) is used without updating [submitted_answers](https://github.com/oppia/oppia/blob/develop/core/controllers/editor.py#L550). 
This is called every time exploration is published, is this the controller for taking in training_data??

So, I think gsoc's ml project addresses point 5 i.e designing the training interface properly and is the UI part of that project. Is the design for this finalised?

Resolving patch:
diff --git a/extensions/interactions/TextInput/TextInput.py b/extensions/interactions/TextInput/TextInput.py
--- a/extensions/interactions/TextInput/TextInput.py
+++ b/extensions/interactions/TextInput/TextInput.py
@@ -24,7 +24,7 @@ class TextInput(base.BaseInteraction):
     description = 'Allows learners to enter arbitrary text strings.'
     display_mode = base.DISPLAY_MODE_INLINE
     is_trainable = True
-    _dependency_ids = []
+    _dependency_ids = ['text_input_prediction']
     answer_type = 'NormalizedString'
     instructions = None
     narrow_instructions = None


Test patch: