Package: ruth-nlu
Ruth Natural Language UnderstandingWelcome to RUTH (Really Understanding The Humans) NLU documentation. RUTH is a open sourced Natural Language Understanding (NLU) framework developed bypuretalk.ai. It is a Python module that allows you to parse natural language sentences and extract information from them.RUTH is cli based tool that can be used to train and test models.InstallationQuick installation$ pip install ruth-pythonInstallation from source$ git clone https://github.com/prakashr7d/Research-implementation-NLU-engine.git
$ cd Research-implementation-NLU-engine
$ python setup.py installUsing makefile (for linux & mac users)Makefileis a file that contains a set of directives used by make build automation tool to generate executables and other non-source files of a program from the program's source files.$ git clone https://github.com/prakashr7d/Research-implementation-NLU-engine.git
$ cd Research-implementation-NLU-enginefor ubuntu,$ make bootstrapfor mac,$ make bootstrap-macthen finally to install package run the following bash command$ make installPytorch installation with GPU support$ pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116DocumentationGetting StartedThe main objective of this lib performs to extract information by parsing the sentence written in natural language. To getting started with RUTH let's follow the below steps.
Run the following command to build an initial project with data and a default pipeline file.$ mkdir project_name
$ ruth initOutputProject will be initialized with below structure.
├──data
│└──example.yml
└──pipeline.ymlProject will be created with example data and pipelineCLIRUTH has a CLI interface to train and test the model, to get started with the CLI interface, run the following command$ ruth --helpfor example, to train the model, run the following commandusage: $ ruth [-h] [-v] {train,test} ...TrainingTo train the model, run the following command$ ruth train -p path/to/pipeline.yaml 
  -d path/to/dataset.jsonParameters-p, --pipeline  Pipeline file 
-d, --data dataset pathSaving Trained modelsOnce the training is finished the model will be saved in a directory namedmodelsin the current working directory.Dataset formatRUTH uses a yaml file to store the training data, the yaml file should have the following syntaxexampleversion: "0.1"
nlu:
- intent: ham
  examples: |
    - WHO ARE YOU SEEING?
    - Great! I hope you like your man well endowed. I am  &lt;#&gt;  inches
    - Didn't you get hep b immunisation in nigeria.
    - Fair enough, anything going on?
    - Yeah hopefully, if tyler can't do it I could maybe ask around a bit
- intent: spam
  examples: |
    - Did you hear about the new Divorce Barbie? It comes with all of Ken's stuff!
    - I plane to give on this month end.
    - Wah lucky man Then can save money Hee
    - Finished class where are you.
    - K..k:)where are you?how did you performed?Pipeline---
RUTH is a pipeline based NLU engine, it has 3 basic main components
- Tokenizer
- Featurizer
- Intent ClassifierInpipeline-data.ymlfile is used to define the pipeline and its components
Example of pipeline-basic.yml file for Support Vector Machine (SVM) based intent classifier and CountVectorizer based featurizer.task:pipeline:-name:'WhiteSpaceTokenizer'-name:'CountVectorFeaturizer'-name:'SVMClassifier'task:pipeline:-name:'HFTokenizer'model_name:'bert-base-uncased'-name:'HFClassifier'model_name:'bert-base-uncased'ParsingTo parse the text, run the following command$ ruth parse -m path/to/model_dir 
  -t "I want to book a flight from delhi to mumbai"Parameters-m, --model_path  model file (optional)
-t, --text  text message (required)If model path is not provided, Parse function will use the latest model in the model directory as a default model.TestingTo test the model performance, run the following command$ ruth evaluate -p path/to/pipeline-basic.yml 
  -d path/to/datasetParameters-p, --pipeline  pipeline file 
-d, --data  dataset file
-o, --output_folder  to save result as PNG file (optional)
-m, --model_path (optionol)If model path is not provided,Evaluate functionwill use the latest model in the model directory as a default model. If output folder is not provided, the result will be saved inresultsfolder in the current working directory.DeploymentRUTH uses FastAPI to deploy the model as a REST API, to deploy the model, run the following command$ ruth deploy -m path/to/model_dirParameters-m, --model_path  model file (required)
-p, --port port number (optional)
-h, --host host name (optional)OutputINFO:Startedserverprocess[1]INFO:Waitingforapplicationstartup.
INFO:Applicationstartupcomplete.
INFO:Uvicornrunningonhttp://localhost:5500(PressCTRL+Ctoquit)APIOnce the model is deployed, you can use the following API to parse the textPOST /parse
{
    "text": "I want to book a flight from delhi to mumbai"
}Output{
    "text": "hello ruth!",
    "intent_ranking": [
        {
            "name": "greet",
            "accuracy": 0.9843385815620422
        },
        {
            "name": "how_are_you",
            "accuracy": 0.0017248070798814297
        },
        {
            "name": "voice_mail",
            "accuracy": 0.0008955258526839316
        },
    ],
    "intent": {
        "name": "greet",
        "accuracy": 0.9843385815620422
    }
}Connect us withDevoloped by Puretalk@2022