Repository: nvaccess/nvda

Issue:
Upload NVDA symbols to Mozilla crash-stats
We want to upload NVDA symbols to Mozilla crash-stats so that Mozilla crash reports have useful info if a crash occurs somewhere in our in-process code. This will make debugging such crashes a great deal easier, since we might not have to reproduce it ourselves and/or get a crash dump.

* We can make this part of our AppVeyor builds.
* We already have an account for sysadmin@nvaccess.org on crash-stats with permission to upload symbols ([Mozilla bug 1322632](https://bugzilla.mozilla.org/show_bug.cgi?id=1322632).
* We need to generate an auth token for crash-stats and encrypt it for use with AppVeyor.
* We'll need a copy of dump_syms.exe to convert the symbols to Breakpad format. We could build this ourselves, but we could also [just grab it from Mozilla's repository](https://github.com/mozilla/gecko-dev/raw/master/toolkit/crashreporter/tools/win32/dump_syms_vc1800.exe) for now.
* We can write a Python script to do this based on [these scripts](https://gist.github.com/luser/2ad32d290f224782fcfc).
* We need to upload symbols for IAccessible2Proxy, minHook, nvdaHelperRemote, VBufBackend_gecko_ia2 and vbufBackend_adobeFlash, both 32 and 64 bit versions. The other dlls/exes aren't injected into Firefox.
* [More info about how to pack and upload symbols](http://socorro.readthedocs.io/en/latest/symbols.html)
* We should manually upload symbols for these files in NVDA 2016.4 so we can get more info for crashes from that too.

P2 because anything that can help us resolve crashes is a good thing, especially while we're collaborating with Mozilla on multi-process Firefox. If someone does hit a crash and we don't have this, it's really, really tedious to debug it.


Discussion:
Don't forget wx symbols :-)


On 12/15/2016 10:02 PM, James Teh wrote:
>
> We want to upload NVDA symbols to Mozilla crash-stats so that Mozilla 
> crash reports have useful info if a crash occurs somewhere in our 
> in-process code. This will make debugging such crashes a great deal 
> easier, since we might not have to reproduce it ourselves and/or get a 
> crash dump.
>
>   * We can make this part of our AppVeyor builds.
>   * We already have an account for sysadmin@nvaccess.org
>     <mailto:sysadmin@nvaccess.org> on crash-stats with permission to
>     upload symbols (Mozilla bug 1322632
>     <https://bugzilla.mozilla.org/show_bug.cgi?id=1322632>.
>   * We need to generate an auth token for crash-stats and encrypt it
>     for use with AppVeyor.
>   * We'll need a copy of dump_syms.exe to convert the symbols to
>     Breakpad format. We could build this ourselves, but we could also
>     just grab it from Mozilla's repository
>     <https://github.com/mozilla/gecko-dev/raw/master/toolkit/crashreporter/tools/win32/dump_syms_vc1800.exe>
>     for now.
>   * We can write a Python script to do this based on these scripts
>     <https://gist.github.com/luser/2ad32d290f224782fcfc>.
>   * We need to upload symbols for IAccessible2Proxy, minHook,
>     nvdaHelperRemote, VBufBackend_gecko_ia2 and
>     vbufBackend_adobeFlash, both 32 and 64 bit versions. The other
>     dlls/exes aren't injected into Firefox.
>   * More info about how to pack and upload symbols
>     <http://socorro.readthedocs.io/en/latest/symbols.html>
>   * We should manually upload symbols for these files in NVDA 2016.4
>     so we can get more info for crashes from that too.
>
> P2 because anything that can help us resolve crashes is a good thing, 
> especially while we're collaborating with Mozilla on multi-process 
> Firefox. If someone does hit a crash and we don't have this, it's 
> really, really tedious to debug it.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub 
> <https://github.com/nvaccess/nvda/issues/6646>, or mute the thread 
> <https://github.com/notifications/unsubscribe-auth/AFGivZlHibPG0XqhLhg6Cd2IFd1ErLv2ks5rIhtcgaJpZM4LO2dE>.
>

-- 
------------------------------------------------------------------------


    Derek Riemer

  * Department of computer science, third year undergraduate student.
  * Proud user of the NVDA screen reader.
  * Open source enthusiast.
  * Member of Bridge Cu
  * Avid skiier.

Websites:
Honors portfolio <http://derekriemer.com>
Awesome little hand built weather app! 
<http://django.derekriemer.com/weather/>

email me at derek.riemer@colorado.edu <mailto:derek.riemer@colorado.edu>
Phone: (303) 906-2194


On the contrary, wx symbols are irrelevant here. We don't inject wx into any process (including Firefox), so a Firefox crash involving NVDA will never reference wx symbols.

Resolving patch:
diff --git a/appveyor/mozillaSyms.py b/appveyor/mozillaSyms.py
new file mode 100644
--- /dev/null
+++ b/appveyor/mozillaSyms.py
@@ -0,0 +1,105 @@
+# Based on code from https://gist.github.com/luser/2ad32d290f224782fcfc
+
+"""Script to convert and upload appropriate NVDA debug symbols to Mozilla crash-stats.
+This should just be run as a script with no arguments.
+It expects the crash-stats auth token to be placed in the mozillaSymsAuthToken environment variable.
+To update the list of symbols uploaded to Mozilla, see the DLL_NAMES constant below.
+"""
+
+from __future__ import print_function
+import argparse
+import os
+import subprocess
+import sys
+import zipfile
+import requests
+
+SCRIPT_DIR = os.path.abspath(os.path.dirname(__file__))
+DUMP_SYMS = os.path.join(SCRIPT_DIR, "dump_syms.exe")
+NVDA_SOURCE = os.path.join(os.path.dirname(SCRIPT_DIR), "source")
+NVDA_LIB = os.path.join(NVDA_SOURCE, "lib")
+NVDA_LIB64 = NVDA_LIB + "64"
+ZIP_FILE = os.path.join(SCRIPT_DIR, "mozillaSyms.zip")
+URL = 'https://crash-stats.mozilla.com/symbols/upload'
+
+# The dlls for which symbols are to be uploaded to Mozilla.
+# This only needs to include dlls injected into Mozilla products.
+DLL_NAMES = [
+	"IAccessible2Proxy.dll",
+	"minHook.dll",
+	"nvdaHelperRemote.dll",
+	"VBufBackend_adobeFlash.dll",
+	"VBufBackend_gecko_ia2.dll",
+]
+DLL_FILES = [f
+	for dll in DLL_NAMES
+	# We need both the 32 bit and 64 bit symbols.
+	for f in (os.path.join(NVDA_LIB, dll), os.path.join(NVDA_LIB64, dll))]
+
+class ProcError(Exception):
+	def __init__(self, returncode, stderr):
+		self.returncode = returncode
+		self.stderr = stderr
+
+def check_output(command):
+	proc = subprocess.Popen(command,
+		stdout=subprocess.PIPE,
+		stderr=subprocess.PIPE)
+	stdout, stderr = proc.communicate()
+	if proc.returncode != 0:
+		raise ProcError(proc.returncode, stderr)
+	return stdout
+
+def processFile(path):
+	try:
+		stdout = check_output([DUMP_SYMS, path])
+	except ProcError as e:
+		print('Error: running "%s %s": %s' % (DUMP_SYMS, path, e.stderr))
+		return None, None, None
+	bits = stdout.splitlines()[0].split(' ', 4)
+	if len(bits) != 5:
+		return None, None, None
+	_, platform, cpu_arch, debug_id, debug_file = bits
+	# debug_file will have a .pdb extension; e.g. nvdaHelperRemote.dll.pdb.
+	# The output file format should have a .sym extension instead.
+	# Strip .pdb and add .sym.
+	sym_file = debug_file[:-4] + '.sym'
+	filename = os.path.join(debug_file, debug_id, sym_file)
+	debug_filename = os.path.join(debug_file, debug_id, debug_file)
+	return filename, stdout, debug_filename
+
+def generate():
+	count = 0
+	with zipfile.ZipFile(ZIP_FILE, 'w', zipfile.ZIP_DEFLATED) as zf:
+		for f in DLL_FILES:
+			filename, contents, debug_filename = processFile(f)
+			if not (filename and contents):
+				print('Error dumping symbols')
+				raise RuntimeError
+			zf.writestr(filename, contents)
+			count += 1
+	print('Added %d files to %s' % (count, ZIP_FILE))
+
+def upload():
+	r = requests.post(URL,
+		files={'symbols.zip': open(ZIP_FILE, 'rb')},
+		headers={'Auth-Token': os.getenv('mozillaSymsAuthToken')},
+		allow_redirects=False
+	)
+	if r.status_code >= 200 and r.status_code < 300:
+		print('Uploaded successfully!')
+	elif r.status_code < 400:
+		print('Error: bad auth token? (%d)' % r.status_code)
+		raise RuntimeError
+	else:
+		print('Error: %d' % r.status_code)
+		print(r.text)
+		raise RuntimeError
+	return 0
+
+if __name__ == '__main__':
+	try:
+		generate()
+		upload()
+	except RuntimeError:
+		sys.exit(1)


Test patch: