[Samsung Gear S3] tincansoftware:
App Development Update

Hi all!

For those on tenterhooks, here's a brief update on where we are with the app.

We won't be releasing a beta or pre-release version at this stage; as the app relies on Accessibility Services we feel it would be safest for all involved to distribute it only through the Play Store. I advise only installing accessibility services through the Play store, unless you've compiled it yourself from the source. Malicious accessibility service implementations are the basis of Android malware - we're developing a custom accessibility service specifically to secure the app against misuse.

I've been using a PoC/Alpha version of the app for the past few weeks. This PoC has a list of preset queries to choose from on the Gear, which when selected are sent to the Android companion and entered into GA via a 3rd party accessibility service. My PoC receives no response from GA - I basically just use it to fire home control queries to my Google Home/Assistant.

After confirming that I could send and receive from GA to the companion via the Accessibility API, I began developing a public version including:

Writing a native Android Accessibility Service.

Writing json schemas to use when getting the results from GA, and when sending results to the Gear.

Parsing GA response into jsonObject.

Designing Gear UI views & elements.

Displaying GA results in Gear UI.

Implementing STT input & TTS output.

I anticipate we should be able to submit a finished version of the apps to Samsung & Google stores by late-June to early-July, depending on how each of my developers go with their tasks. To speed up the release we're currently aiming at the following major iterations of feature releases:

Basic: Accessibiity Service, Adequate Gear UI for I/O, STT input, GA querying, GA chat response via TTS & Gear display.

Adds: Alexa support, Preferences

Adds: Full GA result, LinkChip

Adds: Option to perform STT & TTS via Google

Adds: Rich Response, SuggestionChips

Beyond these current tasks, we also plan to implement the following over time:

Refine formatting of GA results on the Gear. Add Rich Notification support.

Provide the assistant input as direct voice (often returns better voice interactions if triggered by voice command).

A complete replacement of the Accessibiity Service with GA API calls in app.

Add the option to perform STT and TTS on phone, watch or web.

Development of custom Assistant wrapper to enhance GA's abilities (eg, Open Navigation on the watch when invoked through the watch interface).

IFTTT channel to enable communication between watch and phone without BT connection.

GA for Gear, stand-alone (watch only app).

If there's anything else you think would be useful to add feel free to mention it here and we'll see what we can do about implementing it. The first release shouldn't be far off, however if y'all are in a hurry to start speaking to GA I can rework my PoC setup and submit this rather limited version in the next day or two? I believe it may be limited to Android 8.0+ or rooted devices, but I'm not sure - I had it running on 7.1.1 as well, but we don't have that phone anymore.

---------- Post added at 09:11 PM ---------- Previous post was at 08:57 PM ----------

Svoice isn't too bad in most respects; it's lightweight, integrated into the watch, and easy to implement. The downsides are inferior speech recognition compared to many other STT engines (although this varies between users), and limited language support. We're still using Svoice hooks for voice input in the app - Samsung are big fans of device security, which often makes access to seemingly simple methods almost impossible :( .

At the moment Svoice is the easiest option to implement for a decent result which offers phrase detection. We plan on switching from application controls to the Tizen Speech API in the next few months which will facilitate phrase aware audio input, and thus can be used to perform STT using an engine on the phone instead of Svoice.