Patent SG-11201909175X-A (SG)
Title: Neural network processor using compression and decompression of activation data to reduce memory bandwidth utilization
Inventor: CORKERY JOSEPH LEON
IPC: H03M7/30
Filed: 2018-04-16
Abstract: BYTES .... DATA 01110 01111 • • • 0000 ACTIVATION DATA (UNCOMPRESSED) 210 (12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property Organization International Bureau (43) International Publication Date 25 October 2018 (25.10.2018) WIP0 I PCT mu °million °nolo ommoimmin En oimIE (10) International Publication Number WO 2018/194998 Al (51) International Patent Classification: GO8B 13/196 (2006.01) H03111 7/30 (2006.01) GO6N 3/063 (2006.01) H03111 7/46 (2006.01) (21) International Application Number: PCT/US2018/027840 (22) International Filing Date: 16 April 2018 (16.04.2018) (25) Filing Language: English (26) Publication Language: English (30) Priority Data: 62/486,432 17 April 2017 (17.04.2017) US 15/953,356 13 April 2018 (13.04.2018) US (71) Applicant: MICROSOFT TECHNOLOGY LI- CENSING, LLC [US/US]; Attn: International Patents, One Microsoft Way, Redmond, Washington 98052-6399 (US). (72) Inventors: CORKERY, Joseph Leon; c/o Microsoft Technology Licensing, LLC, One Microsoft Way, Red- mond, Washington 98052 (US). LUNDELL, Benjamin Eliot; c/o Microsoft Technology Licensing, LLC, One Mi- crosoft Way, Redmond, Washington 98052 (US). WALL, Larry Marvin; c/o Microsoft Technology Licensing, LLC, One Microsoft Way, Redmond, Washington 98052 (US). MCBRIDE, Chad Balling; c/o Microsoft Technology Li- censing, LLC, One Microsoft Way, Redmond, Washing- ton 98052 (US). AMBARDEKAR, Amol Ashok; c/o Mi- crosoft Technology Licensing, LLC, One Microsoft Way, Redmond, Washington 98052 (US). PETRE, George; c/ o Microsoft Technology Licensing, LLC, One Microsoft Way, Redmond, Washington 98052 (US). CEDOLA, Kent (54) Title: NEURAL NETWORK PROCESSOR USING COMPRESSION AND DECOMPRESSION OF ACTIVATION DATA TO REDUCE MEMORY BANDWIDTH UTILIZATION 64 BYTES /202 8 BYTES • MASK 208 0 0 113 121 • • • 2 0 0 0 • • • 0 • • • 121 = 0 RETAINED TRUNCATED 2= RETAINED 0 0 0 0 113 = 0 0 RETAINED TRUNCATED TRUNCATED 1-1 co ot ot ot 00 O C ACTIVATION DATA COMPRESSION FIG. 3 (57) : A deep neural network (\"DNN\") module can compress and decompress neuron-generated activation data to reduce the utilization of memory bus bandwidth. The compression unit can receive an uncompressed chunk of data generated by a neuron in the DNN module. The compression unit generates a mask portion and a data portion of a compressed output chunk. The mask portion encodes the presence and location of the zero and non-zero bytes in the uncompressed chunk of data. The data portion stores truncated non-zero bytes from the uncompressed chunk of data. A decompression unit can receive a compressed chunk of data from memory in the DNN processor or memory of an application host. The decompression unit decompresses the compressed chunk of data using the mask portion and the data portion. This can reduce memory bus utilization, allow a DNN module to complete processing operations [Continued on next page] WO 2018/194998 Al MIDEDIMOMOIDEIREEMOMOIROMEIMMOVOIS D.; c/o Microsoft Technology Licensing, LLC, One Mi- crosoft Way, Redmond, Washington 98052 (US). BO- BROV, Boris; c/o Microsoft Technology Licensing, LLC, One Microsoft Way, Redmond, Washington 98052 (US). (74) Agent: HOPE, Leonard J.; Newport IP, LLC, 1400 112th Ave. SE, Suite 100, Bellevue, Washington 98004 (US). (81) Designated States (unless otherwise indicated, for every kind of national protection available): AE, AG, AL, AM, AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, BZ, CA, CH, CL, CN, CO, CR, CU, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, HN, HR, HU, ID, IL, IN, IR, IS, JO, JP, KE, KG, KH, KN, KP, KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, ME, MG, MK, MN, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, OM, PA, PE, PG, PH, PL, PT, QA, RO, RS, RU, RW, SA, SC, SD, SE, SG, SK, SL, SM, ST, SV, SY, TH, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, ZA, ZM, ZW. (84) Designated States (unless otherwise indicated, for every kind of regional protection available): ARIPO (BW, GH, GM, KE, LR, LS, MW, MZ, NA, RW, SD, SL, ST, SZ, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, RU, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, LV, MC, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Declarations under Rule 4.17: as to applicant's entitlement to apply for and be granted a patent (Rule 4.17(11)) as to the applicant's entitlement to claim the priority of the earlier application (Rule 4.17(iii)) Published: — with international search report (Art. 21(3)) more quickly, and reduce power consumption.