makefile compile only modified files - Page 1
Forum section: electronics / general

author: maurosmartins
author_rank: Regular Contributor
timestamp: October 02, 2021, 08:44:57 pm
content: Hello all,I'm using some stm8 microcontrollers with cosmic compiler and STVD. STVD is not working great for me as it constantly crashes. I though I could use Visual Code my means of a makefile and it works (more or less) as it always compiles every file and not only files with modifications.here is the makefile I've made:Code:PROJ_NAME=workingprojectMICRO=STM8L15xC8PROGRAMMER=STVP_CmdLineODIR=Release# CompilerCC=cxstm8# Flags for compilerCFLAGS=+mods0 -pp -dSTM8L15X_HD -iinc -isrc -i"C:\Program Files (x86)\COSMIC\FSE_Compilers\Hstm8" -clRelease\ -coRelease\# .c filesC_SOURCE=$(wildcard *.c)C_SOURCE+=$(wildcard src/*.c)# .h filesH_SOURCE=$(wildcard *.h)H_SOURCE+=$(wildcard inc/*.h)# Object filesobjects=$(C_SOURCE:c=o)obj = $(patsubst %,$(ODIR)/%,$(objects))TARGET_S19=$(ODIR)/$(PROJ_NAME).s19$(TARGET_S19) : $(obj)@echo $(TARGET_S19)clnk -l"C:\Program Files (x86)\COSMIC\FSE_Compilers\Lib" -o Release\$(PROJ_NAME).sm8 Release\eso.lkfchex -o Release\$(PROJ_NAME).s19 Release\$(PROJ_NAME).sm8./$(ODIR)/%.o: ./%.c@echo $@@echo $<$(CC) $(CFLAGS) $<.PHONY : loadload:@echo Uploading $(PROJ_NAME).s19 to target microcontroller...STVP_CmdLine -BoardName=ST-LINK -Port=USB -ProgMode=SWIM -Device=$(MICRO) -no_log -no_loop -no_warn_protect -verif -FileProg=Release\$(PROJ_NAME).s19can someone please tell me what can I do to prevent this behavior compiling only the files that have been changed?looking forward your reply,Best regards, Mauro.

author: brucehoult
author_rank: Super Contributor
timestamp: October 02, 2021, 10:27:35 pm
content: Quote from: evb149 on October 02, 2021, 10:17:16 pmThat does the opposite of what is being asked.

author: brucehoult
author_rank: Super Contributor
timestamp: October 02, 2021, 10:47:11 pm
content: Quote from: maurosmartins on October 02, 2021, 08:44:57 pmIt looks more or less right, but I'm not sure about some details.First: you don't say what "make" you are using. I'm assuming GNU make aka gmake.You're being trickier with output directories than I normally am with makefiles I write myself.The "./" at the start of the pattern should not be necessary and it doesn't match your substitution to make the "obj" variable. This might not matter."$(CC) $(CFLAGS) $<" is not what I'd normally do. I'd expect more like "$(CC) $(CFLAGS) $(CPPFLAGS) -c -o $@ $<". If using gcc your version would 1) write the object file to the source directory, not the Release directory and 2) try to link every source file.Maybe cxstm8 behaves differently but I don't know how you expect it to write the .o to the ODIR because you're not telling it to.You also don't have anything in your makefile to cause a .o to be rebuilt if a header it includes changes.Check carefully where the .o files are actually being written to. The link step looks like it will be taking them from the Release directory, but the compile step looks like it's putting them into the source directory. Maybe you have some old copies in the Release directory and that's making the linker happy (but not picking up any changes you make in the C source!)Look for and delete all .o files everywhere before running your make, so see what it's really doing. (for debugging, not as a matter of course)

author: retiredfeline
author_rank: Frequent Contributor
timestamp: October 02, 2021, 10:49:39 pm
content: As OP is not a native English speaker, they should clarify what desired. There are two sentences in his post that seem to conflict, and I couldn't decide which was the intended meaning. My money is on less compiles though.>it always compiles every file and not only files with modifications.Ok so they don't want it to compile every file just the ones that have changed?>to prevent this behavior compiling only the files that have been changed?Wait, they don't want to compile only the files that changed? Is there a missing "by" between behaviour and compiling?

author: brucehoult
author_rank: Super Contributor
timestamp: October 02, 2021, 10:59:28 pm
content: Or a comma. "to prevent this behavior, compiling only the files that have been changed"I'm sure that's what is meant.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 02, 2021, 10:59:53 pm
content: I may have missed it in the Makefile the OP posted, but what seems missing to me are the dependencies for each object file. This is probably why it compiles everything every time make is invoked.A number of ways to generate the dependencies, the one that I use being calling gcc with the -MM option.Edit: as Bruce noted, the main culprit is probably the compile flags in the pattern rule. You need to add "-o $@" to the command line. As it is, the object files will be created at the same location as the .c files (or rather the same location as the Makefile), while this Makefile is written so that object files should be written in the $(ODIR) subdirectory. Make will look for object files in this subdirectory every time it is invoked, and since, as the Makefile is here, it will always be empty, then it will trigger a recompile of all .c files.The Makefile is doubly borked so that the link step (that I don't understand, I don't know this 'clnk' tool) probably uses object files in the current directory, while it should look for the in the $(ODIR) subdirectory to be consistent. So it's just a mix of things put together that don't match...As to my first point, the pattern rule "./$(ODIR)/%.o: ./%.c" is an implicit rule that will only consider the .c files' date compared to the .o files' one. It's not enough. It was not the reason for your problem; the point just above is. But it's still incomplete dependencies: it won't take any included files in the .c files as dependencies - so any modification of included files (such as .h) wouldn't retrigger a compile - which is bad. For this, look up how to generate complete dependencies - as I said, the most common way is the use the compiler. GCC can do this with the -MM option, but for the compiler the OP uses, I don't know.* So: first things to fix are:- Add "-c -o $@" to the $(CC) command line;- Probably modify the link step so that it looks for the object files in the $(ODIR) instead.* As to full dependencies, I don't know if the cxstm8 compiler can generate them as GCC can. You should look this up. If not, that'll be more annoying; you'll need to write the dependency rules manually (and maintaining them is a pain.) If you don't care about modifications of included files not triggering a recompile, you can ignore this point. But this is risky IMO.

author: brucehoult
author_rank: Super Contributor
timestamp: October 02, 2021, 11:24:30 pm
content: Quote from: SiliconWizard on October 02, 2021, 10:59:53 pmYes, as I noted, there is nothing there for dependencies on headers. But that will potentially fail to do needed compiles, not cause extra ones.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 02, 2021, 11:25:04 pm
content: Quote from: brucehoult on October 02, 2021, 11:24:30 pmyes - see my Edit

author: brucehoult
author_rank: Super Contributor
timestamp: October 02, 2021, 11:42:54 pm
content: Quote from: SiliconWizard on October 02, 2021, 10:59:53 pmA surprising number of projects seem to get by without dependencies on included files. Often headers are stable enough that doing a full build on the rare occasion they change is not a big deal. But forgetting to do that when needed *is* a big deal and a source of hard to find bugs.gcc conveniently outputs dependencies in a form that can be directly used in a makefile. That's not always the case but most compilers I've come across can output a list of the included files *somehow*. You can then use perl or python or whatever (automated in the makefile) to massage them into a form that can be included into the make process.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 03, 2021, 12:09:03 am
content: Quote from: brucehoult on October 02, 2021, 11:42:54 pmYes. That's a very bad idea. Modifying things just in a header file - for instance a constant - is pretty common, especially for embedded development. If it doesn't retrigger a compile, it's pretty bad.If you're triggering the build manually, you will at least notice nothing gets recompiled (but some less experienced devs may think that somehow the build tool determined it didn't need to... believe me, I've seen this!) But of course it's way worse if you're automating your builds.Quote from: brucehoult on October 02, 2021, 11:42:54 pmIf your compiler doesn't have this feature, you can always use GCC just to generate the dependencies. You just need GCC for any target, doesn't matter. Second condition is that your source code doesn't contain anything with specific compiler extensions that GCC would choke on...Anyway, for more reading about this:https://make.mad-scientist.net/papers/advanced-auto-dependency-generation/

author: brucehoult
author_rank: Super Contributor
timestamp: October 03, 2021, 12:52:09 am
content: Quote from: SiliconWizard on October 03, 2021, 12:09:03 amI was going to mention this but my post was already long enough :-)Note that you only need the preprocessor from GCC (-E), so the actual C code and any extensions in it don't matter at all. And you need to point gcc at the correct system headers for your target, not the host.

author: bson
author_rank: Unknown
timestamp: October 03, 2021, 02:40:51 am
content: Quote from: maurosmartins on October 02, 2021, 08:44:57 pmTry:Quote

author: bson
author_rank: Unknown
timestamp: October 03, 2021, 02:49:32 am
content: Also, you have sources in src/*.c but your dependency is on "./%.c"

author: maurosmartins
author_rank: Regular Contributor
timestamp: October 03, 2021, 09:00:24 am
content: Hello all,thank you for your replies and sorry for the confusion.Quote from: brucehoult on October 02, 2021, 10:59:28 pmYou are totally correct, I want to prevent the compilation of all files when I've only modified one, this reduces the compile time.Quote from: SiliconWizard on October 02, 2021, 10:59:53 pmI've checked this and in this compiler -c doesn't exist and -o is for optimizations:according to the documentation to generate the .o file it only takes the following :I'm trying to achieve this through the implicit rule and the options passed to the compiler (flags -cl and -co that were previously hardcoded)Code:./$(ODIR)/%.o: ./%.c $(DEPS)@echo $@@echo $<$(CC) $(CFLAGS) $<I agree with the missing dependencies and I didn't knew the possibility to generate them automatically. After checking the documentation I've found this:when executed for main.c I get the following:Nevertheless I don't know how to integrate this into the makefile, any help?Quote from: bson on October 03, 2021, 02:40:51 amI've tested it with same results it compiles everything.looking forward your replies,Best regards, Mauro.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 03, 2021, 05:07:47 pm
content: Quote from: maurosmartins on October 03, 2021, 09:00:24 amAlright, you'll need to find another way. Hopefully you understood what the main issue is, but I'm not too sure. Not sure you're very familiar with Makefiles.The issue is the pattern rule "./$(ODIR)/%.o: ./%.c" will expect to find .o files in the '$(ODIR)' directory (which, from your Makefile, is defined as 'Release') as I explained. But your compiler will NOT write them in this directory. So everytime you run make, it finds no object file and thus determines it must rebuild them all.Just take a look at where the .o files are written, and report back.From your screenshot, your compiler doesn't support the '-o' option (which defines the ouput file name, including its path), but it has a '-co' option which apparently allows to define the output directory where it will write the object files. So just pass '-co $(ODIR)' , which should do the trick.Code:$(CC) $(CFLAGS) -co $(ODIR) $<You will probably additionally need to modify the link step, as I'm pretty sure it looks for objects files in the current directory, and not in '$(ODIR)' .Now a simpler way would be to get rid of this '$(ODIR)' subdirectory and just let the compiler/linker put everything in the current directory - but that's messier. In that case, the pattern rule would just become:Code:./%.o: ./%.cbut you'll also need to modify the 'obj' variable.