Repository: nvaccess/nvda

Issue:
Zoom Does Not work As Expected with NVDA
### Steps to reproduce:

1. Go to this link and install the zoom client: https://zoom.us/download
1. Install the program.
1. When you launch the program, tab to either "start a meeting with video", or "start a meeting without video". Press enter on either of these buttons.
1. Press alt+a to mute or un-mute your audio. Press alt+v to turn your video on or off. 
1. Press alt+h to open the chat panel.You are immediately put into the edit field to type text. Type a few words. use your arrow keys to move around the text, mostly the left and right arrow.  

### Expected behavior:

For step 4 above, you should hear the fact that audio is either muted or un-muted, or that video is started or stopped. For step 5 above, you should be able to navigate the text with arrow keys after typing it. It should also be noted that Zoom gives notifications of everything that goes on in the room. For example, when recording is stopped or started, what someone types in the chat panel, when screen sharing is started or stopped, and so on.

### Actual behavior:

When I do the above steps, I get no notification of anything having occurred. I receive nothing telling me that an action has been taken. I cannot see text that I type into any edit field within Zoom by using arrow keys. I can, however, hear the text when I tab away from and then come back to the edit field.  

System configuration:
Dual-core 2.16 ghz Intel Celeron, 4 gb ram, 500 gb hard drive. Video card is on-board Intel video.
NVDA version:
NVDA next-14601,e62a168e 
> NVDA Installed or portable:
Installed


 Other information:
> Example: Running in a VM

Windows version:
> Windows 6.3 build 9600

Name and version of other software in use when reproducing the issue:

The absolutely latest version of Zoom
### Other questions:

Does the issue still occur after restarting your PC?
Yes
Have you tried any other versions of NVDA?
> No 
Please list them and the result
Roommate uses version 2017.3, and he has same issues that I described above.
Zoom Does Not work As Expected with NVDA
### Steps to reproduce:

1. Go to this link and install the zoom client: https://zoom.us/download
1. Install the program.
1. When you launch the program, tab to either "start a meeting with video", or "start a meeting without video". Press enter on either of these buttons.
1. Press alt+a to mute or un-mute your audio. Press alt+v to turn your video on or off. 
1. Press alt+h to open the chat panel.You are immediately put into the edit field to type text. Type a few words. use your arrow keys to move around the text, mostly the left and right arrow.  

### Expected behavior:

For step 4 above, you should hear the fact that audio is either muted or un-muted, or that video is started or stopped. For step 5 above, you should be able to navigate the text with arrow keys after typing it. It should also be noted that Zoom gives notifications of everything that goes on in the room. For example, when recording is stopped or started, what someone types in the chat panel, when screen sharing is started or stopped, and so on.

### Actual behavior:

When I do the above steps, I get no notification of anything having occurred. I receive nothing telling me that an action has been taken. I cannot see text that I type into any edit field within Zoom by using arrow keys. I can, however, hear the text when I tab away from and then come back to the edit field.  

System configuration:
Dual-core 2.16 ghz Intel Celeron, 4 gb ram, 500 gb hard drive. Video card is on-board Intel video.
NVDA version:
NVDA next-14601,e62a168e 
> NVDA Installed or portable:
Installed


 Other information:
> Example: Running in a VM

Windows version:
> Windows 6.3 build 9600

Name and version of other software in use when reproducing the issue:

The absolutely latest version of Zoom
### Other questions:

Does the issue still occur after restarting your PC?
Yes
Have you tried any other versions of NVDA?
> No 
Please list them and the result
Roommate uses version 2017.3, and he has same issues that I described above.


Discussion:
I can confirm this using NVDA master-14625,b6c4d81f. The chat field has dev info of:
name: u'Input chat text Type message here\u2026'
role: ROLE_EDITABLETEXT
states: STATE_FOCUSED
isFocusable: False
hasFocus: True
Python object: <NVDAObjects.IAccessible.IAccessible object at 0x0551A950>
Python class mro: (<class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: None
location: (1045, 662, 287, 58)
value: None
appModule: <'appModuleHandler' (appName u'zoom', process ID 26136) at address 519ef90>
appModule.productName: u'Zoom Meetings'
appModule.productVersion: u'4,0,36525,0619'
TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'>
windowHandle: 2622586
windowClassName: u'ZPConfChatWndClass'
windowControlID: 0
windowStyle: 1442840576
windowThreadID: 39736
windowText: u'Zoom Group Chat'
displayText: u'Type message here\u2026'
IAccessibleObject: <POINTER(IAccessible) ptr=0x82b5e10 at 55e8f80>
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=2622586, objectID=156045680, childID=0
IAccessible accName: u'Input chat text Type message here\u2026'
IAccessible accRole: ROLE_SYSTEM_TEXT
IAccessible accState: STATE_SYSTEM_FOCUSED, STATE_SYSTEM_VALID (4)
IAccessible accDescription: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
IAccessible accValue: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
Looks like this is something that could be improved, we will gladly accept a pull request to fix this.
I got myself a new computer, and it has Windows 10. Narrator reads notifications in Zoom. Not sure about edit fields. 
@govsta1981  @feerrenrut  @ehollig  I am the Engineer from ZOOM, sometimes we need proactively announce some text in the meeting, something like "your audio is muted" "xxx started screen share", here are the method we used to realize this feature: NotifyWinEvent(EVENT_SYSTEM_ALERT, m_hWnd, (LPARAM)pAccControl, CHILDID_SELF);		pAccControl is an control that implemented IAccessible interface and his role type is ROLE_SYSTEM_ALERT. it works on both JAWS and Narrator, but can't work on NVDA, we will really appreciate if you guys can also support it.
Hi, NVDA does support UIA-based system alert event. I suggest using either a live region changed event or a name changed event so someone who plans to write an app module or a support class for this can implement this easily. Thanks.

 

From: richardweiym [mailto:notifications@github.com] 
Sent: Friday, February 23, 2018 12:26 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Subject: Re: [nvaccess/nvda] Zoom Does Not work As Expected with NVDA (#7754)

 

@govsta1981 <https://github.com/govsta1981>  @feerrenrut <https://github.com/feerrenrut>  @ehollig <https://github.com/ehollig>  I am the Engineer from ZOOM, sometimes we need proactively announce some text in the meeting, something like "your audio is muted" "xxx started screen share", here are the method we used to realize this feature: NotifyWinEvent(EVENT_SYSTEM_ALERT, m_hWnd, (LPARAM)pAccControl, CHILDID_SELF); pAccControl is an control that implemented IAccessible interface and his role type is ROLE_SYSTEM_ALERT. it works on both JAWS and Narrator, but can't work on NVDA, we will really appreciate if you guys can also support it.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <https://github.com/nvaccess/nvda/issues/7754#issuecomment-367942314> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkB-ryNVBPe1Rwvyckn1vcs6hcMq5ks5tXnYrgaJpZM4QcBrX> .


@josephsl , it is not UIA-based, it is just legacy IAccessible solution...
Hi, I see. As I said, because NVDA only supports UIA-based system alert, the best case for now is using generic IAccessible name change or live region event. If this is a standalone app, maybe it might be possible to let NVDA respond to an alert event. If it’s a web-based solution, perhaps a live region could work. Thanks.

 

From: richardweiym [mailto:notifications@github.com] 
Sent: Friday, February 23, 2018 12:34 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Zoom Does Not work As Expected with NVDA (#7754)

 

@josephsl <https://github.com/josephsl>  , it is not UIA-based, it is just legacy IAccessible solution...

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <https://github.com/nvaccess/nvda/issues/7754#issuecomment-367943879> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkDPXWzZ6--xavNK7aQMKi4oa2AwIks5tXngDgaJpZM4QcBrX> .


I think this needs a little more investigation before we can say we 
won't do it.
Thanks for the report. We will look into it soon.


@josephsl  Thanks for you advice, it is standalone Windows Desktop Application. As we tested,  "name change event" can't lead Screen Reader to announce the new text, so we uses this solution..

@michaelDCurran  Thanks
@richardweiym Another thing I noticed is that some checkboxes don't announce their state change. Are you guys notifying WinEvent?
@derekriemer  thanks for your remind, we will fix it
I am about to investigate EVENT_SYSTEM_ALERT.
I'll let you know if I can get NVDA to notify these.
Okay, I know how to make this work. After a few more possible flood mitigations are worked out, I'll open a pull request, and NVDA will work better with zoom's alerts.
@derekriemer  thank you so much, looking forward to your fix release.
I can confirm this using NVDA master-14625,b6c4d81f. The chat field has dev info of:
name: u'Input chat text Type message here\u2026'
role: ROLE_EDITABLETEXT
states: STATE_FOCUSED
isFocusable: False
hasFocus: True
Python object: <NVDAObjects.IAccessible.IAccessible object at 0x0551A950>
Python class mro: (<class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: None
location: (1045, 662, 287, 58)
value: None
appModule: <'appModuleHandler' (appName u'zoom', process ID 26136) at address 519ef90>
appModule.productName: u'Zoom Meetings'
appModule.productVersion: u'4,0,36525,0619'
TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'>
windowHandle: 2622586
windowClassName: u'ZPConfChatWndClass'
windowControlID: 0
windowStyle: 1442840576
windowThreadID: 39736
windowText: u'Zoom Group Chat'
displayText: u'Type message here\u2026'
IAccessibleObject: <POINTER(IAccessible) ptr=0x82b5e10 at 55e8f80>
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=2622586, objectID=156045680, childID=0
IAccessible accName: u'Input chat text Type message here\u2026'
IAccessible accRole: ROLE_SYSTEM_TEXT
IAccessible accState: STATE_SYSTEM_FOCUSED, STATE_SYSTEM_VALID (4)
IAccessible accDescription: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
IAccessible accValue: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
Looks like this is something that could be improved, we will gladly accept a pull request to fix this.
I got myself a new computer, and it has Windows 10. Narrator reads notifications in Zoom. Not sure about edit fields. 
@govsta1981  @feerrenrut  @ehollig  I am the Engineer from ZOOM, sometimes we need proactively announce some text in the meeting, something like "your audio is muted" "xxx started screen share", here are the method we used to realize this feature: NotifyWinEvent(EVENT_SYSTEM_ALERT, m_hWnd, (LPARAM)pAccControl, CHILDID_SELF);		pAccControl is an control that implemented IAccessible interface and his role type is ROLE_SYSTEM_ALERT. it works on both JAWS and Narrator, but can't work on NVDA, we will really appreciate if you guys can also support it.
Hi, NVDA does support UIA-based system alert event. I suggest using either a live region changed event or a name changed event so someone who plans to write an app module or a support class for this can implement this easily. Thanks.

 

From: richardweiym [mailto:notifications@github.com] 
Sent: Friday, February 23, 2018 12:26 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Subject: Re: [nvaccess/nvda] Zoom Does Not work As Expected with NVDA (#7754)

 

@govsta1981 <https://github.com/govsta1981>  @feerrenrut <https://github.com/feerrenrut>  @ehollig <https://github.com/ehollig>  I am the Engineer from ZOOM, sometimes we need proactively announce some text in the meeting, something like "your audio is muted" "xxx started screen share", here are the method we used to realize this feature: NotifyWinEvent(EVENT_SYSTEM_ALERT, m_hWnd, (LPARAM)pAccControl, CHILDID_SELF); pAccControl is an control that implemented IAccessible interface and his role type is ROLE_SYSTEM_ALERT. it works on both JAWS and Narrator, but can't work on NVDA, we will really appreciate if you guys can also support it.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <https://github.com/nvaccess/nvda/issues/7754#issuecomment-367942314> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkB-ryNVBPe1Rwvyckn1vcs6hcMq5ks5tXnYrgaJpZM4QcBrX> .


@josephsl , it is not UIA-based, it is just legacy IAccessible solution...
Hi, I see. As I said, because NVDA only supports UIA-based system alert, the best case for now is using generic IAccessible name change or live region event. If this is a standalone app, maybe it might be possible to let NVDA respond to an alert event. If it’s a web-based solution, perhaps a live region could work. Thanks.

 

From: richardweiym [mailto:notifications@github.com] 
Sent: Friday, February 23, 2018 12:34 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] Zoom Does Not work As Expected with NVDA (#7754)

 

@josephsl <https://github.com/josephsl>  , it is not UIA-based, it is just legacy IAccessible solution...

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <https://github.com/nvaccess/nvda/issues/7754#issuecomment-367943879> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkDPXWzZ6--xavNK7aQMKi4oa2AwIks5tXngDgaJpZM4QcBrX> .


I think this needs a little more investigation before we can say we 
won't do it.
Thanks for the report. We will look into it soon.


@josephsl  Thanks for you advice, it is standalone Windows Desktop Application. As we tested,  "name change event" can't lead Screen Reader to announce the new text, so we uses this solution..

@michaelDCurran  Thanks
@richardweiym Another thing I noticed is that some checkboxes don't announce their state change. Are you guys notifying WinEvent?
@derekriemer  thanks for your remind, we will fix it
I am about to investigate EVENT_SYSTEM_ALERT.
I'll let you know if I can get NVDA to notify these.
Okay, I know how to make this work. After a few more possible flood mitigations are worked out, I'll open a pull request, and NVDA will work better with zoom's alerts.
@derekriemer  thank you so much, looking forward to your fix release.

Resolving patch:
diff --git a/source/appModules/zoom.py b/source/appModules/zoom.py
new file mode 100644
--- /dev/null
+++ b/source/appModules/zoom.py
@@ -0,0 +1,15 @@
+#appModules/zoom.py:
+#A part of NonVisual Desktop Access (NVDA)
+#Copyright (C) 2018 NV Access Limited,Derek Riemer
+#This file is covered by the GNU General Public License.
+#See the file COPYING for more details.
+
+import appModuleHandler
+import eventHandler
+
+class AppModule(appModuleHandler.AppModule):
+	def __init__(self,*args,**kwargs):
+		super(AppModule,self).__init__(*args,**kwargs)
+		#Explicetly allow alert events for zoom's chat windows.
+		#Zoom alerts are not  zoom foreground window descendants.
+		eventHandler.requestEvents("alert",processId=self.processID,windowClassName="zoom_acc_notify_wnd")


Test patch: