Is Microchip violating the GPL? - Page 1
Forum section: electronics / general

author: Karel
author_rank: Super Contributor
timestamp: August 20, 2016, 12:09:38 pm
content: I tried to download the sourcecode of the latest (1.42) PIC32 C compiler from the Microchip website.I couldn't find it at their page:http://www.microchip.com/mplab/compilerstab: download archivessection: source archivescolumn: MPLAB XC32Instead, I found it at their page:http://www.microchip.com/mplab/mplab-x-idetab: download archivessection: source archivescolumn: MPLAB XC32So, I downloaded the sourcecode and had a look at the file xc32-tools-xc32-XC32-release-1_42-src/gcc/gcc/config/mchp-cci/mchp_sha.hTo my surprise, I found a wrong sha256 hash at line 30. This hash doesn't match the hash used in the binary/opt/microchip/xc32/v1.42/bin/bin/gcc/pic32mx/4.8.3/cc1To be sure, I did a grep -I -R <sha256hash> * to see if the hash used in the cc1 binary is present in the sourcecode.It isn't.So, apparently, the sourcecode provided by Microchip is not the sourcecode used to create the binary.This would mean that Microchip violates the GPL.My question is, am I overlooking something?

author: Marco
author_rank: Super Contributor
timestamp: August 20, 2016, 02:12:34 pm
content: AFAICS only if they deny you a request for the actual source code.More backgroundhere.The bigger GPL violation in my opinion is that the IDE and the compiler are clearly a combined work, assuming they distribute them together.

author: Karel
author_rank: Super Contributor
timestamp: August 20, 2016, 02:42:19 pm
content: Quote from: Marco on August 20, 2016, 02:12:34 pmThen the question arises, what's the purpose of the link to the sourcecode with the wrong hash?It's labeled as beeing the sourcecode voor version1.42.

author: dannyf
author_rank: Super Contributor
timestamp: August 20, 2016, 02:59:05 pm
content: Quotethey are definitely wrong.mortgage your house, your retirement and your kids college funds and sue those bastards.we are right behind you,

author: Ian.M
author_rank: Super Contributor
timestamp: August 20, 2016, 03:14:05 pm
content: Quote from: Marco on August 20, 2016, 02:12:34 pmThey don't. MPLAB X is a totally separate download from the XC compiler suites. There may have been some legacy violations with the MPLAB 8 suite and C30/C32.

author: jesuscf
author_rank: Frequent Contributor
timestamp: August 20, 2016, 03:37:28 pm
content: Quote from: Karel on August 20, 2016, 12:09:38 pmThe correct sha256 is probably computed and included at build time. That is the way I would do it. The only way to know is by looking at the building script...

author: amyk
author_rank: Super Contributor
timestamp: August 21, 2016, 01:27:27 am
content: Quote from: Karel on August 20, 2016, 12:09:38 pmNo, it could be a different compiler they used to compile the compiler, or different compilation settings, and various other environment configurations.Anyway, unless you're some pedantic lawyer, it doesn't really matter. Even the GPL doesn't mean they have to essentially give away their whole build environment.

author: Karel
author_rank: Super Contributor
timestamp: August 21, 2016, 06:41:36 am
content: Quote from: amyk on August 21, 2016, 01:27:27 amI wasn't talking about a build environment. I was talking about the GCC compiler which is GPL licensed.More specific, a binary with the name cc1.The GPL license says that if you distribute a modified version, you have to distribute (or make available) the exact sourcecode of thatmodified version. There are no excuses.If they compute and build the sha256 hash at buildtime, as jesuscf suggestst, it's still not an excuse to not providethe sourcecode of the cc1 binary after building that binary.

author: helius
author_rank: Super Contributor
timestamp: August 21, 2016, 07:17:04 am
content: Quote from: Karel on August 21, 2016, 06:41:36 amI hope you are aware that modifying source code is not the only way that changes to a binary may be made.https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdfThis issue has never been addressed by the FSF: the GPL only requires that changes to the source be shared, but those are not the whole extent of what determines binary output. The compiler, other tools, environment variables, makefiles, and libraries all affect the output in a complex way. And most organizations do not have areproducible buildprocess in place.

author: voltsandjolts
author_rank: Unknown
timestamp: August 21, 2016, 08:42:25 am
content: If Microchip have included a build date/time string in the binary (perhaps not a good thing to do), re-building the binary would never give the same hash.Have you tried doing a make clean and rebuild to see if your hash is consistent?

author: dannyf
author_rank: Super Contributor
timestamp: August 21, 2016, 11:33:56 am
content: Quoteif you are so convinced of your righteousness, why don't you put your money where your mouth is and sue Microchip? Then you may find out where you may not have done your due diligence.

author: andersm
author_rank: Super Contributor
timestamp: August 21, 2016, 11:38:02 am
content: If you want a legal opinion, ask a lawyer. If you believe a violation has occurred, contact Microchip or the Free Software Foundation.

author: amyk
author_rank: Super Contributor
timestamp: August 21, 2016, 01:37:58 pm
content: Quote from: voltsandjolts on August 21, 2016, 08:42:25 amActually that is the default behaviour of a lot of compilers, they will embed the version of the compiler used to compile into the binary too. It's useful for determining when and where it was built, for example.

author: Karel
author_rank: Super Contributor
timestamp: August 21, 2016, 02:03:55 pm
content: Quote from: voltsandjolts on August 21, 2016, 08:42:25 amI wasn't talking about the hashofthe binary. I was talking about a hashinthe binary.It works as follows. When the compiler is called, it calls another (closed source) program that checks your license.This program returns a value to the compiler so that it knows if it can use the higher optimization settings yes or no.In the past, people have replaced this program with another simple program that always return the value that allows thecompiler to use all optimization settings.Now, the compiler performs a sha256 sum of the program that checks for the license. If it detects that the hash of that programis different, it refuse to compile.The hash value is hardcoded in a headerfile and because of that, clearly visible when you open the compiler executable witha hexeditor.Have a look here:https://www.eevblog.com/forum/microcontrollers/pic32-evolution/msg1007099/#msg1007099and here:http://www.jubatian.com/articles/turning-on-optimizations-in-microchips-xc32/

author: helius
author_rank: Super Contributor
timestamp: August 21, 2016, 03:06:18 pm
content: Surely a complete fool's errand? Anyone who compiles from source can simply comment out the code that checks if it is equalBut if Microchip had any brains (fat chance that), they would realize that they could have just made their proprietary program communicate over RPC. Then when they wanted to add some code to the GPL'd base (gcc), they would simply put it in the new program instead, and leave a rpc-call stub in gcc. No GPL violation, but the feature won't work unless you buy it from Microchip.

author: Marco
author_rank: Super Contributor
timestamp: August 21, 2016, 03:13:00 pm
content: RPC is likely to open them up to the claim that the calling program is a derivative work.

author: Karel
author_rank: Super Contributor
timestamp: August 21, 2016, 03:14:36 pm
content: Quote from: helius on August 21, 2016, 03:06:18 pmDespite being it theoretically possible, many modifications Microchip made are very much interwoven with the original code.Because of that it's not practical.

author: AlfBaz
author_rank: Super Contributor
timestamp: August 21, 2016, 03:54:58 pm
content: Quite some versions of C32 ago, I got my hands on on the source package for C32. I didn't know about hash values but used CC1's source in conjunction with it's disassembly to hex-hack CC1's exe by placing jumps and nop's around the offending code. Took about an hour to do.From memory i changed about three bytes in the exe and got a fully optimizing C32. At the time I remember thinking "Who the hell releases source to the code that protects their IP?"

author: Karel
author_rank: Super Contributor
timestamp: August 21, 2016, 04:08:23 pm
content: Quote from: AlfBaz on August 21, 2016, 03:54:58 pmThe people who want to use GPL licensed code?

author: hans
author_rank: Super Contributor
timestamp: August 21, 2016, 04:15:55 pm
content: Quote from: AlfBaz on August 21, 2016, 03:54:58 pmThe hash counter measure was only introduced at a later date.Before you could just replace xlcm with a return statement and it would activate appropriately.Conveniently, it could also make compiling much faster because it doesn't have to check at the server what the actual status is. But I believe they fixed this bug.They added the hashing later, but being an open source product that has little effect to people that want to figure this stuff out.I believe you can also just hex edit the target file and invert the nullify option if expressions. Indeed, with 1 hour spare time and a cup of coffee you could have this sorted.Of course you could also recompile from source, but doing that on a windows box it's small little hell IMO.

author: jesuscf
author_rank: Frequent Contributor
timestamp: August 21, 2016, 04:40:47 pm
content: Allow me to show results from the *gcc -v of several gcc compilers I have installed in my computer:Code:C:\>gcc --vUsing built-in specs.Target: x86_64-w64-mingw32Configured with: ../gcc44-svn/configure --target=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --disable-multilib --disable-nls --disable-win32-registry --prefix=/mingw64 --with-gmp=/mingw64 --with-mpfr=/mingw64 --enable-languages=c,c++Thread model: win32gcc version 4.4.3 (GCC)Code:C:\>avr-gcc -vUsing built-in specs.Reading specs from c:/programs/avr8-gnu-toolchain/bin/../lib/gcc/avr/4.9.2/device-specs/specs-avr2COLLECT_GCC=avr-gccCOLLECT_LTO_WRAPPER=c:/programs/avr8-gnu-toolchain/bin/../libexec/gcc/avr/4.9.2/lto-wrapper.exeTarget: avrConfigured with: /home/toolsbuild/workspace/avr8-gnu-toolchain/src/gcc/configure LDFLAGS=-L/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86-hostlibs/lib CPPFLAGS= --target=avr --host=i686-w64-mingw32 --build=x86_64-pc-linux-gnu --prefix=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86 --libdir=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86/lib --enable-languages=c,c++--with-dwarf2 --enable-doc --disable-shared --disable-libada --disable-libssp --disable-nls --with-avrlibc=yes --with-mpfr=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86-hostlibs --with-gmp=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86-hostlibs --with-mpc=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86-hostlibs --enable-win32-registry=avrtoolchain --with-pkgversion=AVR_8_bit_GNU_Toolchain_3.5.3_1700 --with-bugurl=http://www.atmel.comThread model: singlegcc version 4.9.2 (AVR_8_bit_GNU_Toolchain_3.5.3_1700)Code:C:\>arm-none-eabi-gcc -vUsing built-in specs.COLLECT_GCC=arm-none-eabi-gccCOLLECT_LTO_WRAPPER=c:/programs/gnu\ tools\ arm\ embedded/5.4\ 2016q2/bin/../lib/gcc/arm-none-eabi/5.4.1/lto-wrapper.exeTarget: arm-none-eabiConfigured with: /home/build/work/GCC-5-0-build/src/gcc/configure --build=i686-linux-gnu --host=i686-w64-mingw32 --target=arm-none-eabi --prefix=/home/build/work/GCC-5-0-build/install-mingw --libexecdir=/home/build/work/GCC-5-0-build/install-mingw/lib --infodir=/home/build/work/GCC-5-0-build/install-mingw/share/doc/gcc-arm-none-eabi/info --mandir=/home/build/work/GCC-5-0-build/install-mingw/share/doc/gcc-arm-none-eabi/man --htmldir=/home/build/work/GCC-5-0-build/install-mingw/share/doc/gcc-arm-none-eabi/html --pdfdir=/home/build/work/GCC-5-0-build/install-mingw/share/doc/gcc-arm-none-eabi/pdf--enable-languages=c,c++ --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-headers=yes --with-newlib --with-python-dir=share/gcc-arm-none-eabi --with-sysroot=/home/build/work/GCC-5-0-build/install-mingw/arm-none-eabi --with-libiconv-prefix=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-gmp=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-mpfr=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-mpc=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-isl=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-cloog=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-libelf=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors' --with-multilib-list=armv6-m,armv7-m,armv7e-m,armv7-r,armv8-m.base,armv8-m.mainThread model: singlegcc version 5.4.1 20160609 (release) [ARM/embedded-5-branch revision 237715] (GNU Tools for ARM Embedded Processors)Code:C:\>xc32-gcc -vMicrochip Language Tool Shell Version 1.42 (Build date: Jun 1 2016).Copyright (c) 2012 Microchip Technology Inc. All rights reserved*** Executing: "C:\Programs\xc32\v1.42\bin\bin/xc32-gcc.exe" "-v"Using built-in specs.COLLECT_GCC=C:\Programs\xc32\v1.42\bin\bin/xc32-gcc.exeCOLLECT_LTO_WRAPPER=c:/programs/xc32/v1.42/bin/bin/gcc/pic32mx/4.8.3/lto-wrapper.exeTarget: pic32mxConfigured with:Thread model: singlegcc version 4.8.3 MPLAB XC32 Compiler v1.42 (Microchip Technology)Did you notice something missing from the output of the XC-32 compiler that is present in the other ones? Maybe I am wrong, but it looks to me that Microchip is making an extra effort to prevent people from building XC-32 from sources. But wait, there is more. Look what happens when you do the same with XC-32 v1.40:Code:C:\Programs\xc32\v1.40\bin>xc32-gcc -vMicrochip Language Tool Shell Version v1.40 (Build date: Jun 3 2015).Copyright (c) 2012 Microchip Technology Inc. All rights reserved*** Executing: "C:\Programs\xc32\v1.40\bin\bin/xc32-gcc.exe" "-v"Using built-in specs.COLLECT_GCC=C:\Programs\xc32\v1.40\bin\bin/xc32-gcc.exeCOLLECT_LTO_WRAPPER=c:/programs/xc32/v1.40/bin/bin/gcc/pic32mx/4.8.3/lto-wrapper.exeTarget: pic32mxConfigured with: ../../src48x/gcc/configure --build=i386-apple-darwin10 --host=i586-mingw32 --prefix=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_ranch/win32-image --includedir=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_branch/win32-image/pic32mx/include --bindir=/Volumes/home/c11067/xc32/uild/xc32-XC32_v1_40_branch/win32-image/bin/bin --libexecdir=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_branch/win32-image/bin/bin --disable-shaed --with-libelf=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_branch/win32-build/mingw32-libs --with-gmp=/Volumes/home/c11067/xc32/build/xc32-XC32v1_40_branch/win32-build/mingw32-libs --without-isl --without-cloog --target=pic32mx --program-prefix=xc32- --with-bugurl=http://www.microchip.com/suport --with-dwarf2 --with-newlib --with-gnu-as --with-gnu-ld --enable-cxx-flags=-mno-smart-io --enable-lto --enable-fixed-point --enable-gofast --enale-languages=c,c++ --enable-sgxx-sde-multilibs --enable-sjlj-exceptions --enable-obsolete --disable-hosted-libstdcxx --disable-libstdcxx-pch --disabl-libssp --disable-libmudflap --disable-libffi --disable-libfortran --disable-bootstrap --disable-shared --disable-__cxa_atexit --disable-nls --disabl-libgomp --disable-threads --disable-sim --disable-decimal-float --disable-libquadmath --without-headers 'XGCC_FLAGS_FOR_TARGET=-frtti -fexceptions -no-enforce-eh-specs -mno-smart-io' --disable-checking --enable-poison-system-directoriesThread model: singlegcc version 4.8.3 MPLAB XC32 Compiler v1.40 (Microchip Technology)Did they suddenly realized that displaying the 'Configured with' was helping those who tried to build XC-32 from sources and then they removed it?

author: Eturtle
author_rank: Newbie
timestamp: August 21, 2016, 05:30:08 pm
content: Hello. I'm new to this forum and my English isn't the best, so be patient.Microchip took the free gcc as base for XC16 and XC32. Everyone can download the modified version even without registration. This satifies the GPL, IMO. But the building of XC?? is very tricky, otherwise they couldnt sell it. To check for a valid license some parts of the modified gcc (cc1, cc1plus and lto1) call the external program "xclm". Some figured out it was easy to cheat gcc by replacing xclm with a simple "return 2". MCHP counters back by checking the sha256 of xclm. Cause xclm isnt GPL it's totally no matter how the checksum is.Seems XC was build canadian cross with a macintosh. Tried myself and got really trouble on an actual 64bit linux. But one day i got some success with my old 32bit desktop from 2009. Build XC16 v1.25 for linux which dont do the xclm-check. You can download it fromhttps://www.mikrocontroller.net/topic/400756,copy the binaries and enjoy O2, O3 and Os for your PIC24. Unfortunately i did not get any feedback if its working.

author: dannyf
author_rank: Super Contributor
timestamp: August 21, 2016, 05:40:21 pm
content: QuoteThat's the basic flaw in GPL: it assumes the only thing that matters is the source code.Couldn't have been further from the truth.

author: Karel
author_rank: Super Contributor
timestamp: August 21, 2016, 05:46:16 pm
content: Quote from: Eturtle on August 21, 2016, 05:30:08 pmAnd this is exactly the point why I started this thread. They did not. (at least that's what I think)The hardcoded hash in the header file is not the same as the hash inside the executable.Correct me if I'm wrong but to me this means that the sourcecode they provide is not the sameas used to produce the executable.

author: grumpydoc
author_rank: Super Contributor
timestamp: August 21, 2016, 05:49:14 pm
content: Quote from: dannyf on August 21, 2016, 05:40:21 pmNo it doesn't - the GPL V2.0 contains the following definition"The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable."V3 contains a similar definition.Stallman was well aware that you need more than just the "C" files.

author: Eturtle
author_rank: Newbie
timestamp: August 21, 2016, 06:32:45 pm
content: Quote from: Karel on August 21, 2016, 05:46:16 pmHardcoded sha is only for obfuscation. The build_XC16_451 script containsCode:if [ "$BUILD_SHA" = "1" ]; thenecho "*** Creating XCLM SHA"gcc -o ${TMP:-/tmp}/build_sha \$SRC_DIR/gcc/gcc/config/mchp-cci/mchp_sha.c \$SRC_DIR/gcc/gcc/config/mchp-cci/sha256.c -D_BUILD_SHA_CHECKSUM_BUILT_SHA=`${TMP:-/tmp}/build_sha $INSTALL_DIR/xclm/install/client/bin/xclm${EXE_SUFFIX} 2>&1`if ( echo $BUILT_SHA | grep -q -i "Error:" ); thenecho "$BUILT_SHA"exitfiBUILD_SHA="-DMCHP_XCLM_SHA256_DIGEST_VAL=$BUILT_SHA"fimeaning the sha is taken during building of the compiler.Maybe the building was easy if someone has a "--build=i386-apple-darwin10" environment.

author: filssavi
author_rank: Frequent Contributor
timestamp: August 21, 2016, 06:36:46 pm
content: If the binary they distribuite and the source code archive built with default configuration (or whatever they specify in the documentazione) don't match bit by bit than they are violaing GPL for more on the topic you can read thishttp://cormander.com/2011/08/what-some-many-people-dont-understand-about-the-gpl/Bottomline open source doesn't only mean giving out .c/.h files you must also give out makefiles/Shell script and anything that goes with the build system.As for enforcing it might be not as hard as you think, since gcc is a GNU's project members and RMS is quite a full on open source zealot (for better or for worse) and Microchip is a very High profilo target so you might only have to reportage the thing to the gcc/gnu guys on the mailing list ant they will go after them (they have a dedicated legal team for open source violation enforcement)

author: voltsandjolts
author_rank: Unknown
timestamp: August 21, 2016, 07:30:16 pm
content: So, in theory, I could compile this program:int main (void) { return 2; }and name it XCLM.EXEThen during build, its sha256 hash is calulated by the build script and added to XC16 binary.So, no need to go changing source code in all locations where XCLM is called?Just the simple matter of setting up the build environment then.....eek, if only I was a bit smarter.

author: Karel
author_rank: Super Contributor
timestamp: August 21, 2016, 07:47:47 pm
content: Quote from: voltsandjolts on August 21, 2016, 07:30:16 pmMaybe, apart from that your program must return 6, not 2.But changing the hash with a hexeditor is much quicker than trying to setup the build environment,as explained here:https://www.eevblog.com/forum/microcontrollers/pic32-evolution/msg1007099/#msg1007099

author: helius
author_rank: Super Contributor
timestamp: August 21, 2016, 08:34:40 pm
content: Quote from: grumpydoc on August 21, 2016, 05:49:14 pmYou can still drive a truck through it. Nothing in nature requires that the commands used to compile a program be entered into a script.It also assumes that the compiler is a platonic unchanging ideal, so he must have been unaware of Thompson's Turing Award lecture. It's obvious to anyone who has read it that if a password check can be changed by invisible code lurking inside the compiler, then any change whatsoever can be made in the same way; it doesn't even matter whether the compiler's source code is also published.His blindness on this point may be due to an assumption that he controls the compiler, buteven GNU developers think that GCC is doomed.

author: grumpydoc
author_rank: Super Contributor
timestamp: August 21, 2016, 09:35:06 pm
content: Quote from: helius on August 21, 2016, 08:34:40 pmI'd argue that the definition in the GPL would cover command lines, even if not contained in scipts per se.And human nature being what it is no-one would want to typegcc -o prog <list of 15 object files> <list of 6 libraries>more than once or twice without putting it into a file.QuoteI doubt he was unaware but probably did not think it relevant because the GPL dates from a time where you would generally assume other people on the 'net - which was nascent, naive and mostly restricted to academia - would "do the right thing". I don't think it was because he thought he controlled the compiler. In fact the GNU project barelyhada compiler at the time.But I think you do raise a valid point which is that few of the open source projects specify the build environment well enough, or even at all. Quite often interesting looking projects which compiled fine with gcc 2.x have not really been developed since and fail in later versions, or against later versions of libraries. Sometimes spectacularly in the case of libraries and don't get me started on trying to find the right version of auto<crap> for an old project

author: andersm
author_rank: Super Contributor
timestamp: August 21, 2016, 09:44:20 pm
content: Quote from: filssavi on August 21, 2016, 06:36:46 pmDoes the GPL require me to provide source code that can be built to match the exact hash of the binary I am distributing?

author: filssavi
author_rank: Frequent Contributor
timestamp: August 21, 2016, 11:38:04 pm
content: Quote from: andersm on August 21, 2016, 09:44:20 pmYou never stop learning... (I'm sincere not sarcastic or joking)I only wrote few trivial makefiles so i'm not up to Speed with all the depths of build systems (i'd like to learn but too difficult and too Little practical utility for me)And from the height of my inenuity i really thought that builds were reproducibile (given the right cli arguments of course) and i fail to find a use case where having a system specific build system is useful (as far as te finalmente binary goes, od course the linker must know where libraries, dll's ecc are)