uC remote Data Storage - Page 1
Forum section: electronics / projects

author: Gitch
author_rank: Newbie
timestamp: June 18, 2017, 12:01:17 pm
content: Hey All,So I'm looking into starting my next project... This will be my first web connected one and I am trying to decide on how best to store variables and how often to be checking them.I'll break it down to two projects initially, so my first question is in regards how often you would check a network connected device for it's current state. I have an a/c controller that provides a JSON output when requested and I am looking to add a better interface (part of my whole home integration) I'm think 500ms should be ok? Or should it be more or less often - remembering I would also be able to write back to the a/c from several devices.Second one, is how people best store variable for their connected devices. Initially I will be using a Pi as my central unit (allow for html control and logging of certain parameters) and I am thinking of using a MySQL (or other db) and either an update statement or add statement depending on if it is or isn't a logged variable i.e. Temperature logging vs live stateAny feedback appreciatedCheers

author: jbb
author_rank: Super Contributor
timestamp: June 19, 2017, 01:37:32 am
content: In industrial settings, you would tend to choose a sample rate with reference to the time constants of the process.In this case the process is room temperature control? That won't change fast, so maybe every minute?Additionally, be aware that long term log systems get really large. So even if you have plenty of bandwidth, storage is an issue.Alternatively you can do some mixes sampling. For example, you could use a local node to poll the air con unit every 500ms, do some summary statistics (e.g. Min, max, average) and then send aggregate results off for logging every 2 minutes. Events like aircon start and stop could be logged as time stamped events to preserve time accuracy.Also FYI, I hear that Rapberry Pis are a little prone to having their file systems wrecked if they're writing and the power fails, so make sure you back up often. (Or maybe try out one of those UPS shields with backup battery.)

author: viperidae
author_rank: Frequent Contributor
timestamp: June 19, 2017, 09:38:35 am
content: Ideally your devices should push their state over the network. That's what protocols like mqtt are for. Did you build the a/c controller or can you reprogram it?

author: 3db
author_rank: Frequent Contributor
timestamp: June 19, 2017, 11:29:33 am
content: As @viperidae mentioned there's MQTT. This is often used together with NodeRed.You can find lots of information on their deployment and use on a Raspberry PI on yYutube.Peter Oakes channel might be a good starting point.3db.

author: xani
author_rank: Frequent Contributor
timestamp: June 19, 2017, 09:50:49 pm
content: Try SQLite for storage. It will save a bit of RAM on rPi and the "data" is just single file so easy to backup/copy/manageYou can also fire the "big guns" and go InfluxDB + Grafana for pretty graphs

author: Gitch
author_rank: Newbie
timestamp: June 20, 2017, 01:18:30 am
content: Quote from: viperidae on June 19, 2017, 09:38:35 amNo, at this stage it is a pre-existing controller that has given me access to it's api. So at this stage unfortunately it is a request only...Quote from: xani on June 19, 2017, 09:50:49 pmI Hadn't looked too deep into the DB yet, but yeah SQLite is looking to be the pick of the bunch - as you said better suited for embedded options.Cheers