Package: testgailbotapi
GailBotAboutResearchers studying human interaction, such as conversation analysts, psychologists, and linguists all rely on detailed transcriptions of language use. Ideally, these should include so-called paralinguistic features of talk, such as overlaps, prosody, and intonation, as they convey important information. However, transcribing these features by hand requires substantial amounts of time by trained transcribers. There are currently no Speech to Text (STT) systems that are able to annotate these features. To reduce the resources needed to create transcripts that include paralinguistic features, we developed a program called GailBot. GailBot combines STT services with plugins to automatically generate first drafts of conversation analytic transcripts. It also enables researchers to add new plugins to transcribe additional features, or to improve the plugins it currently uses. We argue that despite its limitations, GailBot represents a substantial improvement over existing dialogue transcription software.Find the full paper published by Dialogue and Discoursehere.StatusGailBot version: 0.1a11 (Pre-release)
Release type: APIInstallationGailBot can be installed using pip by the following command:pip install --upgrade pip
    pip install pyaudio
    pip install ffmpeg-python

    pip install gailbot
    pip install git+https://github.com/linto-ai/whisper-timestamped
    pip install git+https://github.com/m-bain/whisperx.gitUsage - GailBot APIThis release features a convenient API to use GailBot and create custom plugin suites. To use the API and its features, user should import the GailBot API class like the following:from gailbot import GailBotOnce you have imported the GailBot API class, you may initialize an instance of GailBot by doing the following:gb = GailBot(ws_root="your_workspace_path")Here, we have initialized an instance of GailBot using a path to a workspace directory of your choosing. The GailBot instance is called "gb".Methods for interaction with engine, profile settings and input source files are now available as methods of your GailBot instance.Now, we will try to use the GailBot to transcribe some input audio files. To do so, we will need to set up a new profile, add input source files, register and apply plugin suite, and finally transcribe. See the example below:settings_dictionary = {
        "core": {},
        "plugins": {
            "plugins_to_apply": ["demoPlugin"]
        },
        "engines": {
            "engine_type": "watson",
            "watson_engine": {
                "watson_api_key": WATSON_API_KEY,
                "watson_language_customization_id": WATSON_LANG_CUSTOM_ID,
                "watson_base_language_model": WATSON_BASE_LANG_MODEL,
                "watson_region": WATSON_REGION,

            }
        }
    }

    gb.create_new_setting("demo_profile", settings_dictionary)
    gb.add_source(
        source_path="your_source_file_path"
        output_dir="your_output_directory_path"
    )
    gb.register_plugin_suite("path_to_test_plugin_suite")
    gb.apply_setting_to_source(
        "your_source_file_path", 
        "demo_profile"
    )
    gb.transcribe()In the above example, we first create a dictionary with key-value pairs that are required to create a GailBotSettings object. Note that "plugins_to_apply" is a list of plugin names that will be applied for that specific settings profile. Since GailBot currently supports IBM Watson STT, users must first create anIBM Bluemix account. Next, a watson api key and region must be created withIBMand specified in the settings profile.With the settings dictionary specified, we create a new profile called "demo_profile" with the values defined in the settings dictionary.Next, we add input source files by specifying the paths to their directories.Then, register a plugin suite into your GailBot instance.
Finally, apply the profile setting you've set up ("demo_profile") to your source input
and begin transcribing.Supported Plugin SuitesA core GailBot feature is its ability to apply plugin suites during the transcription process. While different use cases may require custom plugins, the Human Interaction Lab maintains and distributes a pre-developed custom suite -- HiLabSuite.HiLabSuiteThis is the main plugin suite that is maintained by the Human Interaction Lab. It uses a multi-layered approach to generate a list structure storing transcription results, supports multiple data views (word level, utterance level etc.), and produces output in various formats.The following demonstrates how HiLabSuite may be used with GailBot:HILABSUITE_PLUGINS = [
        "hilab",
        "OutputFileManager",
        "SyllableRatePlugin",
        "GapPlugin",
        "PausePlugin",
        "OverlapPlugin",
        "CSVPlugin",
        "TextPlugin",
        "XmlPlugin",
        "ChatPlugin"
    ]

    settings_dict = {
        "core": {},
        "plugins": {
            "plugins_to_apply": HILABSUITE_PLUGINS
        },
        "engines": {
            "engine_type": "watson",
            "watson_engine": {
                "watson_api_key": WATSON_API_KEY,
                 "watson_language_customization_id": "",
                "watson_base_language_model": WATSON_BASE_LANG_MODEL,
                "watson_region": WATSON_REGION,

            }
        }
    }

    gb = GailBot(ws_root="your_workspace_path")
    plugin_suite_paths

    gb.create_new_setting("demo_profile", settings_dict)

    gb.register_plugin_suite("path_to_HiLabSuite")

    gb.add_source(
        source_path="your_source_file_path"
        output_dir="your_output_directory_path"
    )

    gb.apply_setting_to_source(
        "your_source_file_path", 
        "demo_profile"
    )

    gb.transcribe()In the above code, we initialize GailBot, create a new settings profile that applies plugins for the HILabPlugin suite, add a source to transcribe, and produce results by applying the plugin suite.Note that in the get_settings_dict() method, users will have to enter their custom WATSON_API_KEY, WATSON_REGION, and WATSON_BASE_LANG_MODEL. These are generated from theIBM Watsonservice.Custom PluginsA core GailBot feature is its ability to allow researchers to develop and add custom plugins that may be applied during the transcription process, in addition to the provided built-in HiLabSuite.ContributeUsers are encouraged to direct installation and usage questions, provide feedback, details regarding bugs, and development ideas byemail.AcknowledgementsSpecial thanks to members of theHuman Interaction Labat Tufts University and interns that have worked on this project.CiteUsers are encouraged to cite GailBot using the following BibTex:@article{umair2022gailbot,
  title={GailBot: An automatic transcription system for Conversation Analysis},
  author={Umair, Muhammad and Mertens, Julia Beret and Albert, Saul and de Ruiter, Jan P},
  journal={Dialogue \& Discourse},
  volume={13},
  number={1},
  pages={63--95},
  year={2022}
}Liability NoticeGailbot is a tool to be used to generate specialized transcripts. However, it
is not responsible for output quality. Generated transcripts are meant to
be first drafts that can be manually improved. They are not meant to replace
manual transcription.GailBot may use external Speech-to-Text systems or third-party services. The
development team is not responsible for any transactions between users and these
services. Additionally, the development team does not guarantee the accuracy or
correctness of any plugin. Plugins have been developed in good faith and we hope
that they are accurate. However, users should always verify results.By using GailBot, users agree to cite Gailbot and the Tufts Human Interaction Lab
in any publications or results as a direct or indirect result of using Gailbot.