Package: tgear-sdk
Tactigon Gear SDKWARNING!!!This project is now discontinued, please refer to thenew Tactigon GearThis package enables the wearable device Tactigon Skin to connect to your python project using Bluetooth Low Energy.SDK version: 3.0.4ContentTactigon Gear SDKWARNING!!!Content1. Architecture2. Prerequisites3. User registration4. Tactigon Skin4.1. Status led5. Tactigon Gear SDK General example6. Tactigon Gear CLI instructions6.1. Tactigon Gear CLI config files6.2. Tactigon Gear CLI training session6.3. Tactigon Gear CLI Training Data Upload6.4. Tactigon Gear CLI Model Building Instructions6.5. Tactigon Gear CLI Model Download Instructions6.6. Tactigon Gear CLI Gesture Recognition Setup and TestAppendix1.ArchitectureTactigon Gear environment has the following architecture:Server is located on the cloud and it is manteined by Next Industries s.r.l.
Server has a web interface where you can handle your profile and your data (models and gestures)Provided Tactigon Gear SDK is the implementation of Tactigon Gear environment client sideTactigon Gear SDK is used for collecting new raw data, send the data to server,
ask server to train a model using the raw data, and download the model from server.
Finally use the model for testing real-time gesture recognition.2.PrerequisitesIn order to use the Tactigon Gear SDK the following prerequisites needs to be observed:Python version: following versions has been used and tested. It is STRONGLY recommended to use these ones depending on platform.Win10: 3.8.7Linux: 3.8.5Mac osx: 3.8.8Raspberry: 3.7.3It is recommended to create a dedicated python virtual environment and install the packages into the virtual environment:python -m venv venvpip install tgear-sdkDepending on your installation (Linux, Raspberry, Mac users) you may need to usepython3andpip3instead ofpythonandpiprespectively3.User registrationIn order to perform new training and download them you need to register on following web side:https://www.thetactigon.com/ai/web/.Click on "Register"Enter a valid email address and click "Send"After a few minutes you will receive a confirmation email with aRegistration linkClick on theRegistration linkTo complete the registration:Set your password (both the e-mail address and the password can be changed in a second time)Choose a “Display Name”Accept specified terms and conditionsClick “confirm” buttonOnce registration is done you can go to Profile section and click onJson Filebutton to download file user_data.json:Login inGo to the profile page by clicking on the top right profile button.In this page click on “Json File” button to find json file with user_id and auth_keyClick on “Save file” button to download in your pcThe use of this file is described later in this doc.For security reasons, passwords are stored on our server in encrypted form.
If you forgot your password it is not possible to recover it; you need to set a new one, starting from this page.
A password reset link will be sent to the e-mail address you provide, which must match the one previously set in your user profile.4.Tactigon Skin4.1.Status ledTactigon Skin led indicates current fucntional state:when not paired to any device it rotates red/green/blue ledwhen paired (connected to PC/raspberry/smartphone) but not transmitting any data, blue led blinkswhen paired and transmitting data, grean led blinks5.Tactigon Gear SDK General exampleWhen using TGear SDK the following steps are needed:# Import libraryimporttimefromosimportpathfromdatetimeimportdatetimefromtgear_sdkimportTGear_Enginefromtgear_sdk.modelsimportHAL,RealTimeConfig,Voice,TGear_Pipes_Name,AudioSource,TSpeechObject,TSpeechCommand,TSpeech,HotWord,TGear_Connection_Statusdefmain():# initialize config objectshal=Hal(BLE_RIGHT_ADDRESS="MAC:ADDRESS:RIGHT:DEVICE",BLE_RIGHT_NAME="TACTI",BLE_RIGHT_ENABLE=True,BLE_RIGHT_ADDRESS="",BLE_RIGHT_NAME="",BLE_RIGHT_ENABLE=False)real_time=RealTimeConfig(MODEL_NAME_RIGHT="MODEL_R")voice=Voice(model="path/to/model",scorer="path/to/scorer")# initialize tgear_enginetgear=TGear_Engine(hal,real_time,voice)# configure tgear_engine to use right tskin and trasmit gesture, angle, button and voice datatgear.config("RIGHT",gesture_pipe_en=True,angle_pipe_en=True,button_pipe_en=True,voice_pipe_en=True)# get end of pipe to collect sent datagesture=tgear.get_pipe("RIGHT",TGear_Pipes_Name.GEST)angle=tgear.get_pipe("RIGHT",TGear_Pipes_Name.ANGLE)button=tgear.get_pipe("RIGHT",TGear_Pipes_Name.BUTTON)voice_pipe=tgear.get_pipe("RIGHT",TGear_Pipes_Name.VOICE)TICK_PERIOD_MS=100tick=datetime.now()g=Nonea=Noneb=None# start sendingtgear.start()whileTrue:_,right_conn=tgear.connection_status()ifright_conn!=TGear_Connection_Status.CONNECTED:print("Wait for connection")time.sleep(0.5)continueifgestureandgesture.poll():g=gesture.recv()ifangleandangle.poll():a=angle.recv()ifbuttonandbutton.poll():b=button.recv()delta=datetime.now()-tickifdelta.total_seconds()<TICK_PERIOD_MS/1000:continuetick=datetime.now()print("gesture:",g,"| angle:",a,"| button:",b)ifb==[1]:print("stop")breakelifb==[4]:print("Voice")tgear.select_voice("RIGHT")tspeech_cmd=TSpeechCommand.listen(TSpeechObject([TSpeech([HotWord("draw")],children=TSpeechObject([TSpeech([HotWord("circle")]),TSpeech([HotWord("cone")]),])),]))voice_pipe.send(tspeech_cmd)result=Noneifvoice_pipe.poll(20):result=voice_pipe.recv()else:voice_pipe.send(TSpeechCommand.stop())print("Voice results:",result)tgear.select_sensors("RIGHT")whilegesture.poll():g=gesture.recv()whileangle.poll():a=angle.recv()whilebutton.poll():b=button.recv()g=Nonea=Noneb=Nonetgear.stop()if__name__=="__main__":main()6.Tactigon Gear CLI instructionsTo run the Tactigon Gear CLI open a command window (cmd/powerShell in Windows, terminal windows for all the others O.S.), move to installation folder and run the python scriptcli.pywith the following command:python cli.pyThe following menu will appear:Gear SDK Client, select an option:
Enter 0 to test server connection
Enter 1 to data collection
Enter 2 to send data to server
Enter 3 to train a new model
Enter 4 to download a model from server
Enter 5 to simple gesture recognition
Type anything to EXITRegardless of choosen menu item, system is ready to use when showing message reportind succesfully connection to Tactigon Skin.
Succesfully connection is also confirmed by Tactigon Skin led (see next paragraph)Before start using the SDK user needs to change the config files according to his needs.Linux/raspberry users: if working in SSH shell you may need to specify a DISPLAY with something like this:export DISPLAY=:0.06.1.Tactigon Gear CLI config filesBefore using the SDK some general information must be set in the two following config files:user_data.json : Locally store login information. Need to match credentials on the server side{"user_name":"joe","auth_key":"_abcde123456__"}hal.json : This config file store all necessary hardware related information{"BLE_RIGHT_ADDRESS":"BE:A5:7F:47:54:65",#TactigonRightBLEAddress(SeeFindTactigonBLEAddresssection)"BLE_LEFT_ADDRESS":"BE:A5:7F:47:54:65",#TactigonLeftBLEAddress(SeeFindTactigonBLEAddresssection)"BLE_RIGHT_ENABLE":"True",#TactigonRightenableflag"BLE_LEFT_ENABLE":"True",#TactigonLeftenableflag"SERIAL_COM_PORT":"COM3",#Noneedtochange-debug"NUM_SAMPLE":10,#Noneedtochange-debug"INTERFACE":"Bluetooth"#Noneedtochange-debug}client.json : This config file store http client related information{"SERVER_URL":"https://tgear.eu.pythonanywhere.com",#TactigonGearServerIPAddress"TRAINING_SESSIONS":["26_02_2021_1","26_02_2021_2"],#listofsessiontoupload(seeTrainingsection)"MODEL_SESSIONS":["26_02_2021_1","26_02_2021_2"],#listofsessionformodelbuilding(seeModelBuildingsection)"MODEL_GESTURES":["circle","square"],#listofgesturesrecognizedbythemodel(seeModelBuildingsection)"MODEL_NAME":"MODEL_RIGHT",#nameassignedtotherequestedmodel(seeModelBuildingsection)"MODEL_DATA_PATH":"data/models/",#Noneedtochange"USER_DATA_PATH":"config_files/user_data.json",#Noneedtochange"RES":{#Noneedtochange-debug"user":" ",#Noneedtochange-debug"password":" ",#Noneedtochange-debug"status":"unsuccessful"#Noneedtochange-debug}6.2.Tactigon Gear CLI training sessionBefore running a training session the /config_files/data_collection.json data must be edited:data_collection.json : This config file store all necessary information for data collection{"RAW_DATA_PATH":"./data/raw/",#pathwhereyouwanttostorerawdata-noneedtochange"GESTURE_NAME":["square_cw","circle_acw"],#listofgesturesyouwanttorecord"SESSION_INFO":"19_02_2021_A",#recordingsessioninformation"HAND":"RIGHT"#whichhandisgoingtobeused(right/left)}Turn ON the Tactigon used for the training sessionRun themain.pypython scriptSelect option 1Enter 1 for data collectionWait for te Tactigon to be connected and activate gesture recognition by pressing button 4 on the TactigonFollow console instructions to train all the gestures entered in the data_collection.json config fileAt the end of the training select option 8Enter 9 for EXITto exit the client6.3.Tactigon Gear CLI Training Data UploadOnce user has run one or more training session (even with different gesture sets), training data can be uploaded to the server and used later on to generate a Model. To do that follow the instructions below:Edit the "TRAINING_SESSIONS" field int theclient.jsonconfig file including all the sessions that user wants to upload to the server. Session are indicated by the "SESSION_INFO" field in thedata_collection.jsonconfig file used during training.Run themain.pypython scriptCheck server connection select option 0Enter 0 to test server connectionSelect option 2Enter 2 to send data to serverEnter 9 for EXITto exit the client6.4.Tactigon Gear CLI Model Building InstructionsEdit the "MODEL_SESSIONS" field int theclient.jsonconfig file including all the sessions that user wants to use for building the model. Session are indicated by the "SESSION_INFO" field in thedata_collection.jsonconfig file used during training.Edit the "MODEL_GESTURES" field int theclient.jsonconfig file including all the gestures that the model has to recognize.Edit the "MODEL_NAME" field int theclient.jsonconfig file to assign a name to the generated model.Run themain.pypython scriptCheck server connection select option 0Enter 0 to test server connectionSelect option 3Enter 3 to train a new modelA new model is created on server side and stored to DataBaseEnter  for EXITto exit the client6.5.Tactigon Gear CLI Model Download InstructionsEdit the "MODEL_NAME" field int theclient.jsonconfig file to select the model to download from server.Select option 4Enter 4 to download model from server6.6.Tactigon Gear CLI Gesture Recognition Setup and TestBefore using the gesture recognition feature the /config_files/real_time.json data must be edited:real_time.json : This config file store the models name used by the gesture recognition engine{"MODEL_NAME_RIGHT":"MODEL_RIGHT",#nameofthemodelusedforrighthandgesturerecognition"MODEL_NAME_LEFT":"MODEL_LEFT",#nameofthemodelusedforlefthandgesturerecognition"MODEL_PATH":"data/models/"#Noneedtochange}Run thecli.pypython scriptSelect option 5Enter 5 for simple gesture recognitionRecognized gestures will be printed on the terminalEnter 9 for EXITto exit the clientAppendixThe Tactigon Gear SDK architecture: