Arduino and SD Cards, weird behaveour - Page 1
Forum section: electronics / beginners

author: Dajgoro
author_rank: Frequent Contributor
timestamp: March 24, 2014, 12:10:42 am
content: Hi!This is the first time I've tried building a project with an Arduino.I've been avoiding it since it came out, I always used discrete microcontrollers for my projects, but now I have to do a job, so I am stuck with it.I have to build a data logger, that would log some analog values to the sd-card.I quickly found the data logger example, and it worked, so I went to modify it to suit the project needs.The problem is that it fails to write to files kinda often.It sorta works, but then it doesn't work.Then I found out that I can't have a expression in a function call, but only a single value?This won't work:Code:Serial.println("Wrote: "+String(something)+" Min= "+String(bmin)+" Max= "+String(bmax)+" T= "+String(t)+" NND= "+nnd+"\r\n");And this does work (sorta):Code:String s;...s="Wrote: "+String(something)+" Min= "+String(bmin)+" Max= "+String(bmax)+" T= "+String(t)+" NND= "+nnd+"\r\n";Serial.println(s);The data logger needs to be able to log some peak values when they occur, and log the value of the ad converters every few sec.I tested all of the code with Serial.println to check if all of the data which was supposed to be written is fine, so the code regarding the data collection is fine.The code that registers the peak values in a separate file works fine, but the code that periodically logs the values sorta doesn't.Sometimes it will fail the first write, first two, or every write until the value on the ad-converter changes.The sketch does use a timer that causes interrupts, but I disable the interrupts prior to writing to the sd-card.I can't figure out what causes it, but I've noticed that I try to write a static string it will always works.I've even tried adding extra power to the circuit, but that wasn't the case, also the demo examples work fine.I also tried adding a loop that retries to write to the file for n times if it fails, but if it failed once, it will continue to fail until the next function call.Data logger function:Code:File dataFile;char logData(String dataString,char file){for(wa=10;wa>0;wa--){File dataFile = SD.open(file, FILE_WRITE);if (dataFile) {dataFile.print(dataString);dataFile.close();wa=0;}else{if(wa==1)return 0;}}return 1;}The log function is being called by(periodically):Code:val=analogRead(0);u="Value= "+String(val);val=analogRead(1);u+=" Temp= "+String(val)+" T= "+String(t)+"\r\n";noInterrupts();logData(u,"log.txt");interrupts();And(peak):Code:s="Min= "+String(bmin)+" Max= "+String(bmax)+" T= "+String(t)+"\r\n";noInterrupts();logData(s,"peaks.txt");interrupts();So what could be the cause of such issues?

author: matts-uk
author_rank: Contributor
timestamp: March 24, 2014, 08:05:40 am
content: Quote from: Dajgoro on March 24, 2014, 12:10:42 amYou can have expressions in function calls, provided the expression evaluates to the type of the argument the function is expecting. The code you posted fails for a different reason. Don't take it the wrong way but it is clear you are yet to master how type declaration and memory allocation work in C/C++.For example...Code:Serial.println("Wrote" +String(something));/*"Wrote" is a string literal, so a constant value. The compiler can notpossibly add 'something' to a constant.*/Generally speaking. The String library uses RAM inefficiently and as as an MCU doesn't have much SRAM, you should avoid using String.h (like the plague). You can get closer to what you are trying to express by using the Streaming.h library and the C++ streaming operator <<.For example...Code:#include <Streaming.h>char stringBuffer[] = "times";void setup(void) {Serial.begin(9600);for (uint8_t i = 1; i < 10; i++) {Serial << "Streaming.h mixes literals and variables " << i << " " << stringBuffer << "\r\n";}}QuoteDid you check whether the SD.h functions are compatible with interrupts being disabled? SD card operations are relatively slow, so I would guess SD card functions are not compatible with interrupts being disabled.Generally speaking. You should not call 'high level' functions with interrupts disabled, as high level functionality is often dependant on interrupts or is so slow, disabling interrupts causes interrupt events to be missed, which causes odd things to occur elsewhere.Typically, interrupts should only be disabled within interrupt service routines (ISR) and ISRs should not be doing much at all. If you _need_ to drive your code from an interrupt, declare a global flag as volatile, set the flag within the ISR and use the flag to trigger the higher level functionality outside of the ISR.For example...Code:volatile boolean flagDoAction = false;void loop(void) {int someValue = Sensor.read();if (flagDoAction) then{sdWrite(someValue);flagDoAction = false;}}void myISR(void) {noInterrupts();flagDoAction = true;interrupts();}

author: Dajgoro
author_rank: Frequent Contributor
timestamp: March 24, 2014, 03:02:56 pm
content: Code:Typically, interrupts should only be disabled within interrupt service routines (ISR) and ISRs should not be doing much at all. If you _need_ to drive your code from an interrupt, declare a global flag as volatile, set the flag within the ISR and use the flag to trigger the higher level functionality outside of the ISR.The interrupt routine is very short. I've also made another version of the code which doesn't use timer interrupts, but it has a static code, and it is even worse...I never wrote stuff in C++, or used String.h, I either used pure C, or some other high level language(pc only).I'll try replacing the string stuff, and see how it behaves.

author: Dajgoro
author_rank: Frequent Contributor
timestamp: March 24, 2014, 03:20:13 pm
content: I get "no match for 'operator <<'", so that doesn't work.I tried to replace everything with char arrays and sprintf, and it crashes immediately.Edit: I removed the string parameter form the function, and I made a global variable, and I use it to pass the string value that needs to be written. And it works, unless if the value on the ad converter is 0, then it crashes sometimes.

author: matts-uk
author_rank: Contributor
timestamp: March 24, 2014, 05:47:53 pm
content: Quote from: Dajgoro on March 24, 2014, 03:20:13 pmDid you download Streaming.h and install it in your <sketchbook>/libraries folder?There is a download link about half way down this pagehttp://arduiniana.org/libraries/streaming/QuoteHmm. I do hope this is not your homeworkCode:#include <Streaming.h>#include <SPI.h>#include <SD.h>char logFileName[] = "logFile.txt";char* stringBuffer = NULL;void setup() {Serial.begin(9600);stringBuffer = (char*) malloc(64);if (stringBuffer == NULL) {Serial << F("Failed to allocate buffer");}pinMode(10, OUTPUT);if(SD.begin()) {Serial << F("SD card open\r\n");}else {Serial << F("SD card failed to open\r\n");}}uint8_t val0Max = 0;int16_t val1Max = 0;uint16_t startPeriod = 0;const uint16_t samplePeriod = 5000;void loop(){uint16_t now = millis();boolean writeMax = false;uint8_t val0 = analogRead(A0);int16_t val1 = analogRead(A1);if (val0 > val0Max) {val0Max = val0;writeMax = true;}if (val1 > val1Max) {val1Max = val1;writeMax = true;}if(writeMax) {stringBuffer = 0;sprintf(stringBuffer, "maxValue= %u maxTemp= %d T= %u\r\n", val0Max, val1Max, now);if(writeToFile(stringBuffer, logFileName)) {Serial << F("Write to ") << logFileName << " : " << stringBuffer;}else {Serial << F("Write failed!\r\n");}}if(now - startPeriod > samplePeriod) {startPeriod = now;stringBuffer = 0;sprintf(stringBuffer, "Value=%u Temp= %d T= %u\r\n", val0, val1, now);if (writeToFile(stringBuffer, logFileName)) {Serial << F("Write to ") << logFileName << " : " << stringBuffer;}else {Serial << F("Write failed!\r\n");}}}uint16_t writeToFile(char* data, char* fileName) {uint16_t numBytes = 0;File logFile = SD.open(fileName, FILE_WRITE);if(logFile) {numBytes = logFile.write((byte*) data, strlen(data));logFile.close();}return numBytes;}

author: Dajgoro
author_rank: Frequent Contributor
timestamp: March 24, 2014, 08:33:47 pm
content: QuoteI included it, downloaded and added to the library folder, but I still get errors:error: invalid operands of types '<unresolved overloaded function type>' and 'const char ' to binary 'operator<<'QuoteNo, it is not.I modified the code to only have a single global string variable, and I use it to pass the value to the function, and it seems that it works now. In the case of getting another write error, i added a crash function that will get stuck in an infinite loop, and trigger the watchdog.I left the thing for a while and some of the data got kinda corrupted, but that might be an issue with the breadboard and emi related to it, the final circuit should be better in that regard.So why having a string parameter in a function causes such weird issues with the arduino compiler???And why does it completely crash with char arrays?What is going on under the hood?Edit:I sent the string(the one used to pass the value to the string after it was written to the sd card, and at some point I got this:Code:@h O! t­â?h×@:?ýÖPâ`h×@:?ýÖPâ`h×@:?ýÖPâEdit2:The string gets corrupted after it has been written to the sd-card.Looks like the sd .print method breaks the string that is being received.Edit3:This code causes the write function to crash:Code:short len;byte charBuf;...len=dataString.length();dataString.getBytes(charBuf, len);When I have this at the beginning of my write function, the write functions just creates an empty file, and fails to write anything.I don't use the chaBuf or len anywhere else in my code.I am having a feeling like the memory is leaking all around the place in this device. /:

author: matts-uk
author_rank: Contributor
timestamp: March 25, 2014, 07:39:59 am
content: Quote from: Dajgoro on March 24, 2014, 08:33:47 pmSuggests the Streaming.h library is not installed, not included correctly or perhaps you are passing a type which needs casting, although I can't think what. You could try turning on the verbose compiler output option, in preferences. And the include is case sensitive. Alternatively, just don't use the Streaming operator. I just find it much easier than the .print and println methods, myself.QuoteThe bug is far more likely to be down to an unterminated, poorly allocated or poorly referenced string, in your code.QuoteThere is nothing wrong with the compiler. The issue is almost certainly down to an unterminated, poorly allocated or poorly referenced string, in your code.Quotea feature of the Arduino's Harvard architecture is, if you fail to manage the stack correctly, the stack pointer will run into the memory space which contains the program instructions. Thereafter, calling a function will cause the program to become corrupted. By far the most common cause of such corruption, is an unterminated string or poorly referenced string or other object.QuoteIs the behaviour typical of an unterminated or poorly allocated or referenced string.Hopefully you understand why I keep repeating myself.QuoteThere is nothing wrong with the compiler and the devices do not leak memory. The example sketch I posted yesterday afternoon, took about 20 minutes to write and has been running on my UNO without error, all night.The behaviour you are seeing, is almost certainly (about 99.9%), caused by a bug in your code. The other .1% is, I can not rule out a corruption of the SD card.Are you willing to post your entire sketch? If I can see how you are using strings, in the full context, I suspect I can very quickly show you where you are going wrong.

author: baljemmett
author_rank: Unknown
timestamp: March 25, 2014, 04:11:56 pm
content: Quote from: matts-uk on March 25, 2014, 07:39:59 amThat doesn't sound much like a Harvard architecture…

author: Dajgoro
author_rank: Frequent Contributor
timestamp: March 25, 2014, 11:06:38 pm
content: QuoteI tried loading that code that you posted, and it fails.It doesn't even create the file.And yet, the datalog example works flawlessly every time!

author: Dajgoro
author_rank: Frequent Contributor
timestamp: March 25, 2014, 11:46:47 pm
content: Taking a closer look at the datalog example, I noticed that the string is declared inside the loop, not as a global variable. So I did that, no function calls at all, and string declared at the start of the loop. And it works, no corruption.Later I tried moving that declaration just outside the loop, to be global, and it gets corrupted upon writing.So that was it, if I want it to work correctly, it must be a single string, declared inside the loop, otherwise it fails!?I even borrowed another arduino from the local makerspace, and it is the same thing!What is going on here?

author: matts-uk
author_rank: Contributor
timestamp: March 26, 2014, 03:02:49 pm
content: Quote from: baljemmett on March 25, 2014, 04:11:56 pmI guess for the pedants, Modified Harvard would be more precise.IIRC. The program is stored top down in memory. The data segment growsbottom up. The stack sits above the data segment. If you fail to manage the stack, it grows upwards and eventually overwrites your executable instructions.Similarly, if you send an unterminated string to a .print method, the method just keeps on sending the next byte of memory to the serial port, until it comes across a zero. Hence, garbage appearing when you .print(someString) is almost certainly down to an unterminated string or some other memory issue.