Package: oremi-sds
Oremi Sound Detector ServerOremi Sound Detector Server is a WebSocket server designed to detect sound
events, including wake words and a predefined list of songs, for the Oremi
Personal Assistant.The server listens on port5023for incoming connections from clients, which
can continuously stream audio data. Once connected, clients can send audio
data inbytes, and the server will process it in real-time, detecting sounds
and sending JSON messages back to the client when a sound is recognized.Oremi SDS detects the Oremi wake word including sounds: Shout, Bellows,
Children shouting, Laughter, Baby laughter, Crying, sobbing, Baby cry, infant
cry, Whistling, Wheeze, Snoring, Cough, Sneeze, Burping, and Hiccup.Table of ContentsInstallUsageProtocolInitializationSound DetectionConnection Closure CodesExample ImplementationContributeVersioningLicenseInstallpipinstallOremi-SDSUsageusage:oremi-sds[-h][--hostHOST][-pPORT]-mMODEL[-tTHRESHOLD][-nNUM_THREADS][-cCONFIG][--verbose][-v]OremiSoundDetectorServer

options:-h,--helpshowthishelpmessageandexit-mMODEL,--modelMODELPathtotheTensorFlowLitemodelfilename(required).-tTHRESHOLD,--thresholdTHRESHOLDDetectionthresholdforfilteringpredictions(default:0.1).-nNUM_THREADS,--num-threadsNUM_THREADSNumberofthreadsforTensorFlowLiteinterpreter(default:-1,auto-select).-cCONFIG,--configCONFIGPathtotheconfigurationfile(default:config.json).--hostHOSTHostaddresstoconnectto(default:127.0.0.1).-pPORT,--portPORTPortnumbertoconnectto(default:5023).--verboseEnableverboselogging.-v,--versionShowtheversionoftheapplication.You can download the model Tensorflow Lite model here:https://storage.googleapis.com/download.tensorflow.org/models/tflite/task_library/audio_classification/rpi/lite-model_yamnet_classification_tflite_1.tfliteProtocolOremi Sound Detector Server operates using a WebSocket-based protocol for
real-time sound detection. The protocol involves an initialization step where
the client provides essential details such as the number of audio channels,
sample rate, block size, language, and features to enable. Once the session is
initialized, the client can continuously stream audio data to the server. The
server processes the audio in real-time and sends JSON messages back to the
client when it detects specific sounds, such as wake words or predefined
songs.The section outlines the message structures, initialization process, sound
detection mechanism, and possible connection closure codes. Developers can use
this protocol documentation as a reference to interact with the server and
build their applications accordingly.Initialization1. ClientWhen a client connects to the server, it must send an initialJSON initiation messagewithin 5 seconds or else the connection will be closed with code1002and
reasonInit Timeout.{"type":"init","language":"fr","samplerate":16000,"num_channel":1,"blocksize":4000,"features":["wakeword-detector","sound-detector"]}2. ServerThe server responds with:{"type":"init","server":"Oremi Sound Detector Server/1.0.0"}Note:If the client doesn't send the initialization message within 5
seconds, the server will close the connection with code1002and reasonInit Timeout.Sound Detection1. ClientOnce the session is initialized, the client can continuously send audio
stream in bytes.2. ServerThe server processes the audio stream in real-time and sends aJSON sound messagewhen it detects a sound:Example for wakeword{"type":"sound","sound":"wakeword","score":1.0,"datetime":"2023-08-02T20:33:22.805154"}Example for cough{"type":"sound","sound":"cough","score":0.4140625,"datetime":"2023-08-02T20:41:05.058204"}Connection Closure CodesPossible connection closure codes:1000: Indicates a normal closure, meaning that the purpose for which the connection was established has been fulfilled.1002: Init Timeout1003: Invalid Message4000: Unexpected ErrorExample ImplementationFor an example of how to implement a client for the "Oremi Sound Detector
Server," you can refer to theclient.py filein the GitLab repository.
The example demonstrates how to connect to the
server, send audio data, and handle the JSON messages received from the
server.ContributePlease followCONTRIBUTING.md.VersioningGiven a version numberMAJOR.MINOR.PATCH, increment the:MAJORversion when you make incompatible API changes,MINORversion when you add functionality in a backwards-compatible manner,
andPATCHversion when you make backwards-compatible bug fixes.Additional labels for pre-release and build metadata are available as extensions
to theMAJOR.MINOR.PATCHformat.SeeSemVer.orgfor more details.LicenseLicensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License.
You may obtain a copy of the License atLICENSE.