Control system web based frontend - Page 1
Forum section: electronics / projects

author: Harvs
author_rank: Super Contributor
timestamp: March 12, 2013, 12:14:19 am
content: Just after some advice.I'm putting together a control system for an outdoor hydroponics system. All pretty straight forward stuff from a uC perspective.There's nodes at a few locations controlling and sensing things all linked up by RS485 (that's the plan at this stage.)I'm thinking of using one of the RPi's I've got sitting around to provide a web based front end to the system and perform long term datalogging. I've linked one via the serial port to another uC to provide a bridge to the RS485 bus since the Pi doesn't seem to support 9bit addressing.So to my question. Any advice on what languages/tools to use on the Pi? The extent of my web based development has been a few lines of html and php many years ago, so whatever I use I'm going to have to learn something (I'm pretty competent in SQL and C is my main language of knowledge.)Obviously there's the web front end stuff that will be seen, but there's also the on-going collection of data and interfacing to the hardware via serial. So I'm guessing they should be split up and communicate somehow? (yes I'm a Linux noob as well.)Thanks

author: Thor-Arne
author_rank: Unknown
timestamp: March 12, 2013, 12:19:13 pm
content: C/C++ is the preferred language on Linux, make your source in one of the editors and compiler with gcc.For the web front-end it would probably be easiest using php and look-up in a mySQL database.If you're a Windows user I suggest gettingWinSCPand do all the work from there.That allows you to useNotepad++or any favorite tools to edit the files, the compile needs to be done in a shell window in this case.

author: varesa
author_rank: Contributor
timestamp: March 12, 2013, 05:52:16 pm
content: C/C++ for the hardware stuff, and php for the web stuff are the more "traditional" choices. Depending on the way you are interfacing to the microcontroller (is it easy to do it on the command line?) it could even be possible to set up simple bash scripts and for example a cron job to do the data collecting. If you need to do some more complex stuff, than to read something from a bus, I would propably choose something else than bash.Also depending on the thing you are doing python could also be a possibility for both of them. I prefer it in some situations because it is pretty easy and fast to write and prototype. It has libraries for eg. interfacing with devices through serial ports and for the web frontend there are some python-cgi libraries. The fact that they can both be done with the same language and tools doesn't mean you should try to integrate them to a single program though. (Unless you can do with polling data from the chip in real time when requested by the user)

author: Smokey
author_rank: Super Contributor
timestamp: March 12, 2013, 07:47:09 pm
content: If you have never used it before, check out python. It's killer for pretty much everything and really easy to learn compared to low level C/C++ stuff. You can do the front end and the back end in the same language, and there are probably libraries already for doing close to what you want to do. I've used it to talk to a bunch of uC projects, but always using pyserial. Even when you HAVE to use C, you can still use python:)http://www.adafruit.com/blog/2013/01/11/spi-python-hardware-spi-for-raspi-from-python-piday-raspberrypi-raspberry_pi/

author: Harvs
author_rank: Super Contributor
timestamp: March 13, 2013, 01:39:26 am
content: Thanks,Already knowing how to program in C (on uCs), I wrote a few small programs to try out comms with the uC using file read and write and it works a charm, much easier than I thought it was going to be. Also I've looked at a tutorial for the MySQL C API and it looks pretty trivial to do the basics of adding and retrieving data from a C program, so that's all good.The small amount of PHP I've done in the past included retrieving and storing to a MySQL database, so that's ok.But what I'm not sure about is how to pass messages back and forward between the web interface and the back end program. Obviously using the database for logged data, but for a simple example, say I wanted to pass a new set temperature value to the control system. It's possible to use the database as a shared memory and just have the web interface update the value in a table, but that would be pretty inefficient having the back end program polling the database periodically for changes, so surely there's a better way of doing it?

author: KuchateK
author_rank: Regular Contributor
timestamp: March 13, 2013, 05:13:04 am
content: If you know PHP I think that is everything you'll need for this project.Use PHP for web interface.I assume you'll use rs232 to rs485 converter to communicate with your hardware. Use PHP function exec to push data from your web interface to your hardware. You can basically write directly from your web to a serial port this way.Code:<?php exec('echo "something" > /dev/ttyS0');?>You can also get data from serial port. If I remember correctly I pulled data from my Arduino using something like this:Code:<?phpechoexec('head -n 1 /dev/ttyUSB0');?>http://php.net/manual/en/function.exec.phpFor data collection and scheduled events you can run PHP as a console script executed from cron. Recently I use it more PHP than Bash for small console scripts.http://php.net/manual/en/features.commandline.phpI used mix of above methods for a lot of projects, including hardware interaction with Arduino over serial port. I like to use PHP because I don't need to compile anything for linux and I can share code between web interface and background scripts.Most of the PHP written for linux will also work on Windows (WAMP) without any problems. The only major difference is with serial port access. I wrote couple powershell scripts to bridge serial port to PHP to get similar behavior to linux. I will post them when I'll find them, for now here is the website I used to write them.http://blogs.msdn.com/b/powershell/archive/2006/08/31/writing-and-reading-info-from-serial-ports.aspx

author: tld
author_rank: Regular Contributor
timestamp: March 13, 2013, 07:33:16 am
content: Quote from: Harvs on March 12, 2013, 12:14:19 amIf you're looking to *start* web development, and you're not already familiar with a specific language for that, I'd not think twice about picking python. Bunch of reasons:* It's a capable generic language (whereas PHP is more web-centric)* It's fast and comfortable to develop in (and fast enough when you run it)* "Batteries included". If you'd like to generate PDF-reports from a dataset at some future point, it's there. Almost everything you'd ever need is just there.I'd also like to point you towards the Flask microframework:http://flask.pocoo.org/I like it because it doesn't get in my way. I'll let me really comfortably define things like "This function handles /info", without me having to deal a lot with the framework. It helps me, but I don't have to deal with it beyond that.As for getting data off of serial, typical would be to have a daemon that reads your serial, then moves that data somewhere else, and only that. It can insert it into an SQL database (I recommend PostgreSQL), throw it at an enterprise bus (where other daemons can subscribe to it) and so on.By random chance, I set up a test-interface to show some of what you can do, and how easily, last weekend.It's sampling outside temperature, downloading some forcasts, logging both to PostgreSQL, and pulling it back out for a nice graph.It lives here, if you'd like to check it out:http://restdemo.keepquiet.net/The live code is available for viewing:http://restdemo.keepquiet.net/doc/code(this is just a mockup to show developers as an example, it's currently in slight violation of the terms of the weather service, I'm hoping they'd forgive me since it's just mucking about, and nothing really public).Being familiar with SQL, I'm sure you could get the gist of how it works simply be looking the code over.Note that the *whole* web page comes in at 131 lines, including SQL and comments. Including JSON/REST-administration API and documentation-browser, it's about 350 lines total.What I love about this is that (with one or two minor exceptions) every single line *does* something. Something specific to the task I'm trying to get done. There's very little "I need to write this code to get X to work, so I can do Y." There's just Y.Whoops, this turned into a bit of a long babbling thing... Let me know if you'd like me to expand on anything...tld

author: Harvs
author_rank: Super Contributor
timestamp: March 13, 2013, 07:48:38 am
content: QuoteInteresting, I see that's probably an easy way to deal with the problem.This is again a noob question, but will the system automatically de-conflict requests to write the serial port? e.g. say the cron job and the web interface were both going to send 10kB to the serial port at the same time, is there any chance they could end up interleaved in some way? Or are they guaranteed to always be complete separate packets?That would make the whole idea more convoluted because not only would the uC require packet error detection (which it will anyway), but both pieces of software would need to be informed their packet failed and to retry at different intervals. Not that difficult but it adds complication. And what if both pieces of software on the Pi were expecting a response at once? What happens if two processes both request a set number of bytes from the serial port at once? Who gets what and how would the uC know which one it's sending to (i.e. can a process put a lock on the serial port to complete its transaction?)I guess the model I was thinking, is if I were writing this as a GUI in .NET, I always have a separate async thread that's solely responsible for maintaining the hardware link. The GUI and any other parts of the program that need to communicate with the system do so by passing messages through one of the async methods to the thread that handles the interface. That way the interface is encapsulated and any changes to the interface, packet structure etc is contained within the one place.

author: tld
author_rank: Regular Contributor
timestamp: March 13, 2013, 08:16:26 am
content: Quote from: Harvs on March 13, 2013, 07:48:38 amYou'll often lock serial port access, so ods are that one of them would simply fail if the other holds the lock.Other than locking, there's no real protection, so you could end up mixing the streams.Quote from: Harvs on March 13, 2013, 07:48:38 amIt's actually rather easy to solve. The thing to do (in my book at least) would be to have a single daemon that "owns" the serial-port. Anything else can then talk through the serial port through that program, and you can apply all the logic you'd like for error-correction, not running multiple requests at once, etc.Quick example on interfacing with serial port (including USB-serial stuffs) using Python:Code:import serials = serial.Serial( "/dev/ttyU0" )s.baudrate = 9600for line in s.readline():print "got line: %s" % lineYou could treat the socket like you would treat say, a network socket.Quote from: Harvs on March 13, 2013, 07:48:38 amPretty similar. Just thinking put it in a separate process, so the rest of the system can be stopped, started, upgraded, added to etc, without having to down the serial-stuffs (unless that needs upgrade too).That way, a crontab can be started, do something throught the serial daemon, and shut down again. No problem.You could have command-line utilities and web-interfaces do the same thing, etc etc.tld

author: Harvs
author_rank: Super Contributor
timestamp: March 13, 2013, 01:28:53 pm
content: Thanks tld. I just need to do some more reading on inter-process communication under linux. i.e. How one process would pass messages to the daemon.I'll take everyone's advice and learn some python for the web interface. It seems like the done thing these days.Cheers

author: tld
author_rank: Regular Contributor
timestamp: March 13, 2013, 01:57:17 pm
content: Quote from: Harvs on March 13, 2013, 01:28:53 pmDon't!Err, that came out wrong...Inter-process communication under linux/unix can be many different things. You have systems for passing messages around, shared locks and what have you not, but for something like this, there's a very good chance simply using TCP or UDP would be the best choice. That'd also give the advantage of being able to talk to it from other hosts for free.Another easy to use choice would be named pipes, similar to TCP, only they're tied to a file instead of a network socket.The daemon could open /dev/ttyU0 to talk to a serial-port, and multiple other programs could open /var/tmp/serialDaemon.sock to talk to your daemon.(just thought I'd mention, before you dive head-first into all the more obscure and often more complicated IPC-stuff).tld

author: Harvs
author_rank: Super Contributor
timestamp: March 13, 2013, 11:53:28 pm
content: That sounds like a good flexible option. Still something I have little idea how to actually do, but I'm sure my friend Google will help me out. Cheers.

author: Harvs
author_rank: Super Contributor
timestamp: March 14, 2013, 01:42:50 pm
content: Just a big thankyou for pointing me in the right direction. With just a couple of hours having never had anything to do with python before, I'm passing messages around via sockets to all my PCs and RPis (windows and linux) on my LAN/WLAN.I never realised it was going to be this quick and easy! It opens up a whole new realm of things that can be done with comms this easy to implement via TCP/IP.