Arduino Ethernet wierdness - Page 1
Forum section: electronics / general

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 01:05:11 am
content: I'm stumped. I'm hoping someone here as some insight into why I'm experiencing what I'm experiencing.The heck am I trying to do?Part one:I have an array of sensors that I'm reading. Temp, light, and various other analog sensors. I'm taking those values and serving up a webpage to display them. You can see it live here:http://garden.dagobah-system.com/tadaa! it's works see. Happy webpage. Refresh manually and you can see the data update. For the record, that is being served up directly from my Arduino. That webpage lives on the arduino. I'm not copying txt files to an Apache or IIS server or anything. That's all Arduino right there.Part two:Part two of this little gizmo, goes out to a "real" webserver (IIS, server in my house, on the network, etc..) and looks at a page that looks like this.Yup. It's a webpage that displays the number zero. It's literally a txt file with 1 character in it, saved as a .html file. This file lives on a local IIS server.OH, sometimes it looks like this:The number 1. Same file, just the content changes. It changes from a 1 to a 0. Right now I'm changing it manually for testing. eventually it's state will be changed by a.....blah blah blah. I'll share when I'm done and it works.The code on the arduino has 4 functions.1. read the sensors: - Check! Works great.2. Serve up that webpage: - Check! works great.3. READ that 1 or 0 from that other page. -Check! Works great4. Turn on an LED (place holder for a 12V water valve) if function 3 =1, turn it off if function 3 = 0. Check. Works great!I can let this run, and change that 1 to a 0 for days and it just continues to work. LED turns on and off, the sensors are all read properly and verified through the webpage and the serial monitor debug function I added.The problem:If I view that first webpage (the one with all the temps on it-served up from the Arduino directly) from a PC inside my network, the Arduino freaks the HECK out. For some reason, it locks up and will no longer READ the 1 or 0 webpage. It continues to read and display the temps, but it stops properly reading the 1 or 0 site served up on my local IIS server. If I reset the ethernet shield connected to the arduino, it starts working again....untill I view and refresh the data page served up by the arduino directly.The kicker, is if I view the page from outside my network (RDP'd to a server at my office or on my cellphone for example) the Arduino is fine. It's ONLY if I view/refresh the data page from inside my network. Crazy.Anyway, I've been chasing this problem for months. I'm happy to post code, in follow-up posts once I've properly commented it, but this seems like something characteristic of the Ethernet Shield. I want SOOO badly to blame this on IIS, but I don't think it's the issue. I think it's something related to the Ethernet shield getting a request while it's trying to READ a page... but why only on the local network is this an issue? Anyway, I'm completely stumped.Anyone have any ideas? I REALLY don't want to use 2 arduinos with 2 Ethernet shields.Thanks,Brian

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 01:45:23 am
content: The code...with it mostly commented. I'm NOT...NOT much of a coder, so please be gentle.Code:#include <SPI.h>#include <Ethernet.h>//so, most of this first part is direct from the Arduino webserver example.byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };IPAddress ip(10,10,27,20);IPAddress dnserver(10,10,27,5);IPAddress gateway(10,10,27,1);char serverName[] = "master-yoda.dagobahsystem.local";EthernetClient client;//global Variables I setup.int analogChannel0 = 0;int analogChannel1 = 1;int analogChannel2 = 2;char valve_status_1;float tempF0;float tempF1;float light1;EthernetServer server(80);void setup() {// Open serial communications and wait for port to open:Serial.begin(9600);//more stuff from the Arduino Example.// start the Ethernet connection and the server:Ethernet.begin(mac, ip, dnserver, gateway);server.begin();Serial.print("Garden IP Address is: ");Serial.println(Ethernet.localIP());}void loop(){//the core functions of this whole sketch.ReadSensors(); //reads all the sensors attached to the ArduinoServPage(); //displays the values from the first function on a webpageDebugSerial(); //this just sends a whole bunch of nerdy stuff to the serial monitor so I can see what the heck is going on.ReadCtrlPage(); // this goes out and reads the 1 or 0 from my IIS server page. The function does stuff, but that stuff will be moved to the dostuff() function - still troubleshooting//DoStuff(); //this WILL be the function that carrys out all the tasks that happen as a result of the information from the first 4 functions... and some external sources as well.}void ReadSensors(){//This function reads all the sensors, does some math and stores the results in global variables.//Read Temp Sensor 0 on pin A0long sum = 0; // *****float voltage; // Localfloat temperature; // Variablesint lsbCount; // *****float lsb = 1.23432/1023; // This is a varable associated with a precision voltage regulator that I don't have connected yet.for (int i = 1; i <= 10; i++)sum += analogRead(analogChannel0); // sum 10 readingslsbCount = (sum + 5) / 10; // and averagevoltage = lsbCount * lsb;temperature = (voltage / 0.01) - 10.0; //temp in CtempF0 = ((temperature * 1.8) + 32); // Converts to F//Read Temp Sensor 1 on pin A1sum = 0;for (int i = 1; i <= 10; i++)sum += analogRead(analogChannel1); // sum 10 readingslsbCount = (sum + 5) / 10; // and averagevoltage = lsbCount * lsb;temperature = (voltage / 0.01) - 10.0; //same as the previous functiontempF1 = ((temperature * 1.8) + 32); //same as the previous functionsum = 0;for (int i = 1; i <= 10; i++)sum += analogRead(analogChannel2); // sum 10 readingslsbCount = (sum/10); // and averagevoltage = lsbCount * 5.0;voltage /= 1024.0;light1 = voltage; //I have no idea how to conver this to Lux or candle power or lumins or whatever. so I'm leaving it volts for now.}void ServPage(){//this function generates the webpage at http://garden.dagobah-system.com//The next 20 lines are pretty much copy/paste from the Arduino IDE webserver example.EthernetClient client = server.available();if (client) {Serial.println("new client");// an http request ends with a blank lineboolean currentLineIsBlank = true;while (client.connected()) {if (client.available()) {char c = client.read();Serial.write(c);// if you've gotten to the end of the line (received a newline// character) and the line is blank, the http request has ended,// so you can send a replyif (c == '\n' && currentLineIsBlank) {// send a standard http response headerclient.println("HTTP/1.1 200 OK");client.println("Content-Type: text/html");client.println("Connnection: close");client.println();client.println("<!DOCTYPE HTML>");client.println("<html>");//I took out the meta refresh because it kept breaking my Arduino... so now I refresh manually for testing.. :(// add a meta refresh tag, so the browser pulls again every 5 seconds://client.println("<meta http-equiv=\"refresh\" content=\"5\">");//*********************************************************************//* LETS DISPLAY SOME STUFF!!!!! *//*********************************************************************//the next like million lines of code is me doing HTML codeing via client.print(). I know I can consolidate this a little bit but the way I have it here makes it easier to read/edit I think.client.print("<table bgcolor=#55FFAA border=1>");client.print("<tr>");client.print("<td width=267><b><center>INPUT SENSOR DATA</center></b></td>");client.print("</tr>");client.print("</table>");//Table Setup Format// Sensor Name, Value, Unit, ADC Channelclient.print("<table border=1 >");client.print("<tr>");client.print("<td>");client.print("<b>Sensor Name</b>");client.print("</td>");client.print("<td>");client.print("<b>Value</b>");client.print("</td>");client.print("<td>");client.print("<b>Unit</b>");client.print("</td>");client.print("<td>");client.print("<b>ADC Channel</b>");client.print("</td>");client.print("</tr>");//DATAclient.print("<tr>");client.print("<td>");client.print("Shady Temp ");client.print("</td>");client.print("<td>");client.print(tempF0);client.print("</td>");client.print("<td>");client.print(" F");client.print("</td>");client.print("<td>");client.print(analogChannel0);client.print("</td>");client.print("</tr>");client.print("<tr>");client.print("<td>");client.print("Sunny Temp ");client.print("</td>");client.print("<td>");client.print(tempF1);client.print("<td>");client.print(" F");client.print("</td>");client.print("<td>");client.print(analogChannel1);client.print("</td>");client.print("</tr>");client.print("<tr>");client.print("<td>");client.print("Ambiant Light ");client.print("</td>");client.print("<td>");client.print(light1);client.print("</td>");client.print("<td>");client.print(" V");client.print("</td>");client.print("<td>");client.print(analogChannel2);client.print("</td>");client.println("<tr/>");client.println("</table>");client.print("<table bgcolor=#55FFAA border=1>");client.print("<tr>");client.print("<td width=267><b><center>OUTPUT CONTROL STATUS</center></b></td>");client.print("</tr>");client.print("</table>");client.print("<table border=1 >");client.print("<tr>");client.print("<td>");client.print("<b>Output Name</b>");client.print("</td>");client.print("<td>");client.print("<b>Value</b>");client.print("</td>");client.print("<td>");client.print("<b>Unit</b>");client.print("</td>");client.print("<td>");client.print("<b>I/O Channel</b>");client.print("</td>");client.print("</tr>");client.print("<tr>");client.print("<td>");client.print("Valve 1");client.print("</td>");client.print("<td>");client.print(valve_status_1);client.print("</td>");client.print("<td>");client.print("1/0");client.print("</td>");client.print("<td>");client.print(analogChannel2);client.print("</td>");client.println("<tr/>");client.print("</table>");//*********************************************************************//* ...AND WE'RE DONE DISPLAYING *//*********************************************************************//The rest of this function is right out of the Arduino Example too. I kinda know what's going on, but... it's best if I leave it alone.client.println("</html>");break;}if (c == '\n') {// you're starting a new linecurrentLineIsBlank = true;}else if (c != '\r') {// you've gotten a character on the current linecurrentLineIsBlank = false;}}}// give the web browser time to receive the datadelay(1);// close the connection:client.stop();Serial.println("client disonnected from browser request");delay(1000);}}void ReadCtrlPage(){//This function goes and reads that IIS page looking for a 1 or 0.client.stop(); //I put this in here for troubleshooting to make sure that the ethernet shield is not doing anything until I tell it to.delay(1000); //again, more trouble shooting. figured I'd give it second, to close any open connections. I dunno. Seemed reasonable.Serial.println("connecting to control page..."); //just for the serial monitor. Let'n yall know what's going on.if (client.connect(serverName, 80)) { //connects to the IIS server on port 80Serial.println("connected");// Make a HTTP request:client.println("GET http://master-yoda.dagobahsystem.local/do.html"); //sends the request via the client function to the IIS web server looking for a page called do.htmlSerial.println("GET http://master-yoda.dagobahsystem.local/do.html"); //displays the same thing to the serial monitor.client.println("HTTP/1.1");Serial.println("HTTP/1.1");client.println("Host: master-yoda.dagobahsystem.local");Serial.println("Host: master-yoda.dagobahsystem.local");client.println();Serial.println();}else {// if you didn't get a connection to the server it just says this, then moves on.Serial.println("connection failed");Serial.println("We'll give it another go on the next go-around.");delay(2000); //so you can read it.}if (client.available()) {char d = client.read(); //reads the page, and stores the character it finds in the variable "d".Serial.println(d); //prints "d" to the serial monitor so I can verify what it found out there.Serial.println();if (d == '1'){ //this will be move to the "dostuff()" function eventually.digitalWrite(7,HIGH); //if d = 1 then turn on the LED on digital pin 7.}else if(d = '0'){ //if d = 0 turn the LED off.digitalWrite(7,LOW);}valve_status_1 = d; //takes the local variable "d" and stores it in the global variable "valve_status_1" so I can use it other places.client.stop(); //stops the client function.}// if the server's disconnected, stop the client:if (!client.connected()) {Serial.println();Serial.println("disconnecting.");client.stop();}}void DebugSerial(){//this is my debug function. I'll comment it out once everything is working, but for now I like having it.Serial.println("************* ");Serial.print("temp 0: ");Serial.println(tempF0);Serial.print("temp 1: ");Serial.println(tempF1);Serial.print("light 1: ");Serial.println(light1);Serial.println(valve_status_1);delay(3000);}

author: Psi
author_rank: Super Contributor
timestamp: January 07, 2014, 01:57:16 am
content: This is just a stab in the dark. but..It could be your web browser attempts to establish a HTTPS connection first before falling back to HTTP and the arduino doesn't the HTTPS request.When connecting from outside of your LAN the https port will be blocked by NAT so doesn't get to the arduino.Maybe try typing the full HTTP:// address into the browser and see if you get the same problem.(im assuming you've just been typing the DNS name or IP with no HTTP ?)

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 02:03:44 am
content: OH! I hope it's something simple like that... I'll try that and see if gets around the issue.Brian

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 02:06:57 am
content: Nope. It broke. So weird. I can refresh that page from my office 100 times if I want. Hit it once from inside and BZZZZZZZZ... broke.Brian

author: mrflibble
author_rank: Super Contributor
timestamp: January 07, 2014, 03:37:35 am
content: Maybe the difference is that having the 2nd PC in the local network is causing extra windows domain bullshit broadcasts, that are not being sent in the single server case. If your arduino barfs on the broadcasts then that would explain it.Well that, or having the 2nd PC and the arduino on the same IP address or even mac address ... but that would be really silly.

author: Psi
author_rank: Super Contributor
timestamp: January 07, 2014, 03:48:25 am
content: Do you have any old win98 computers you could test out on the local LAN and see if they have the problem?It might help prove something.You could also try using a smartphone over wifi.

author: IanJ
author_rank: Unknown
timestamp: January 07, 2014, 06:54:52 am
content: I seem to remember something similar................is this helpful?http://forum.arduino.cc/index.php/topic,114291.0.htmlIan.

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 01:35:53 pm
content: Quote from: mrflibble on January 07, 2014, 03:37:35 amI suppose it's possible. I assure you I don't have duplicate MACs or IPs, but the broadcast thing is curious enough. When I get home tonight, I'll put that Arduino on a separate VLAN. That layer 3 component should block any broadcasts. Probably not a bad idea to isolate my projects like that anyway. Some nerd will figure out how to hack my network through my Arduino.THanks,Brian

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 01:38:33 pm
content: Quote from: Psi on January 07, 2014, 03:48:25 amI could spin up a Win98 VM I suppose. I'll verify the smartphone thing, but I'm pretty sure that broke it too. I also have a Chromebook and an iPad I could try as well. Heck, maybe it is an OS thing. I'm willing to try anything at this point.Thanks,Brian

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 01:43:52 pm
content: Quote from: IanJ on January 07, 2014, 06:54:52 amThat's interesting, but I'm not sure we're having the same issue. Although! I'm not sure why I never thought to look at wireshark..https://www.eevblog.com/forum/Smileys/default/facepalm.gifDUH. This could REALLY give me some insight.Ok, well, when I get home tonight, I'm going to check out the traffic with WireShark, if that doesn't make the solution obvious, then I'll try the VLAN thing, if THAT doesn't work. I will watch TV and drink a beer.Brian

author: amyk
author_rank: Super Contributor
timestamp: January 07, 2014, 02:10:48 pm
content: I'm also a little surprised why no one else mentioned getting out the wireshark... as someone who has debugged alotof embedded networked systems, it is the #1 tool to use.And judging by your symptoms, I would look in more detail into ARP.

author: BrianDagobah
author_rank: Contributor
timestamp: January 07, 2014, 02:20:20 pm
content: I also use WireShark everyday. No idea why it didn't occur to me to use it on this... LOL. I guess it's the work/hobby disconnect. eh? I'm still looking for some way to generate SNMP data with my arduino. That's another project though.Brian

author: BrianDagobah
author_rank: Contributor
timestamp: January 12, 2014, 10:58:30 pm
content: OK. Sorry it took so long to reply to this. Stupid job/life getting in the way.So, WireShark yielded nothing useful that I could tell (dug out an old ethernet hub and connected it to the Arduino and my Wireshark laptop). However, creating another VLAN seemed to work. As long as there is a L3 "barrier" between the Arduino and the PC hitting that website, everything works fine. So, my hobby projects now have their own VLAN and associated ACL. How about that! It's GOT to be broadcasts or something. Not sure where the problem is, but meh... chalk it up to, weird crap happens I suppose. When I migrate this all over to my Linux VM I'll try putting it back on the original VLAN and see what happens. Maybe it is IIS...Next step: get SNMP data out of the Arduino ethernet shield.Thanks

author: westfw
author_rank: Super Contributor
timestamp: January 13, 2014, 10:01:49 am
content: QuoteBut what does it show? Does the Arduino respond to the ARP request from the local system? Does it (in turn) ARP FOR the address of the local system? Does it send IP packets for the local system to the proper ethernet address? Which side sends (or fails to send) the first "mistake"? Can you duplicate the bad behavior if you make a telnet connection to the web server address, or is it only a problem with a browser?In theory, I should be doing more with Ethernet Arduinos. In reality, I'm terrified of using a TCP/IP implementation that has no "internals visibility." (shudder.)