opened
bemcdonnell
username_0
I think it's time to dream up an Input API that will find its way into the formal distribution of SWMM. :)
Input API
issue
created
bemcdonnell
username_0
EPANET has a toolkit API containing some good starting point ideas.

https://github.com/OpenWaterAnalytics/EPANET/blob/master/src/toolkit.h Lines 243 and 244 reference the functions to "set" values for a node and a link.  This could be expanded in SWMM to adjust hydrology and Ground water parameters.
comment
created
samhatchett
username_1
You and I must be drinking from the same well. I've been thinking about this recently too. Would be nice to brainstorm current/future needs for both softwares, maybe nail down a common set of conventions?
comment
created
bemcdonnell
username_0
@username_1 Good timing!  

So the main SWMM InputAPI use case I foresee is for calibration - mostly oriented toward planning.  However, there has been talk up here regarding parameter manipulation during simulation time (@username_2 has some thoughts on this).
comment
created
samhatchett
username_1
the ultimate goal on the EPANET side is to have its code split up into three different modules:
- simulation
- project definition I/O
- simulated values I/O
of course there's a fair amount of work to do. I'm working on scraping together some thoughts related to structuring of the API for creating/altering network topology - I'd be interested in your thoughts once I have a draft.
comment
created
rkertesz
username_2
I'll let @username_3 chime in but, while we currently can manipulate anything that is controlled in the control section (like orifices), what would be nice is intra-simulation manipulation of things that aren't exposed right now, such as a property for a pipe. This can help not only with calibration and tuning as well as controlling for variables like sediment during a long-term simulation.
comment
created
samhatchett
username_1
sure, what I mean is pretty simple. for maintainability and extensibility, we need to start pulling the code apart into logical units. the core of that outcome is going to be a streamlined hydraulic engine and associated structs (what i referred to as the "project definition"). you could also call it the "business logic / data model" unit. we will need to provide an API into that data model, which will be capable of programmatically constructing/ altering a hydraulic model and its associated parameters. Everything that is now done through the specification in the `inp` file format should eventually be handled by the Project API, and we can remove the dependence on the text file. It should not matter where or how the network is defined - text, json, sql, binary, protobuf...  Similarly for storage/retrieval of simulation output data - there's really no reason a hydraulic analysis engine needs to ever use `fopen` - this is just a basic level of the separation of concerns.
comment
created
lmontest
username_3
As it relates to the work we engage regularly, we find the need to be able to manipulate an asset based on the status of the system. Ideally designing RTC algorithms that can “measure” the state of the system (flows, depths, capacities, even water quality) and adjust setpoints for pumps, orifices, weirs, etc. This means that the control rules must be able to interact during simulation time with the model. The ideal situation would allow an external program to receive hydraulic status of select assets periodically and adjust setpoints at such points in time. To that end, and in our own effort to enable such applications, we have modified SWMM to allow simulations to halt periodically, populate an output database table with hydraulic states, allow time for a program to consume such data to generate appropriate setpoints, put them in an input database table, let SWMM to consume the new setpoints, and resume the simulation. The choice of using a database for the interaction with SWMM was driven by the ease of modifying the code and enable a variety of programs to interact with the data through ODBC connectors. We have demonstrated this by building a controller in Matlab. An even nicer application would be to interface SWMM directly with a PLC. I think that porting similar functionality with a DLL API should be straightforward. The main idea remains to break SWMM out of being a modeling tool and make it available to applications such a operators training modules, real time scenario analysis, and real time control applications. Cheers!
comment
created
bemcdonnell
username_0
My ideas and contributions for the InputAPI live here for now:

https://github.com/username_0/Stormwater-Management-Model/tree/InputAPI
comment
created
lrntct
username_4
The link is dead. Is there a page or wiki where are compiled the ideas and status?
comment
created
samhatchett
username_1
Private repo?
comment
created
bemcdonnell
username_0
@username_4, i must have removed that at some point. But truth be told; I doubt any of the ideas I had then were worthwhile :-)

Here’s some stuff on the api.  https://github.com/OpenWaterAnalytics/Stormwater-Management-Model/wiki/Enhanced-API

I think we should have a roadmapping meeting to talk about the future of the data model and API
comment
created
lrntct
username_4
One great addition would be to be able to add or remove objects from the simulation.
As of now, the objects are stored in arrays of structs. My basic C knowledge tells me that resizing an array is not a very efficient operation, so it might not be ideal to add objects one by one.
One possibility would be to change the data structure and maybe use one from C++ (std::vector?), but it will be an important change.
Any thought on what could be possible in pure C?
comment
created
bemcdonnell
username_0
I also agree that this would be an important change.  

The catch to this whole thing is going to be managing the connections.  Links current index the upstream and downstream node indexes.  So we are going to need to come up with a strategy to make sure as things change, the indeces still work out.

@michaeltryby has been recommending the project incorporate a transactional data model.  He can better explain his ideas
comment
created
samhatchett
username_1
we've done some similar work on EPANET -> https://github.com/OpenWaterAnalytics/EPANET/issues/43  and  https://github.com/OpenWaterAnalytics/EPANET/pull/88 

In basic form, we do use a realloc - which may end up being inefficient, but the primary purpose of the addition was to expand what was possible to do with the toolkit. And once we build and profile that capability, we can then optimize. And since it was done outside the "customary" procedure of opening an `inp` file, it doesn't affect established use-cases performance-wise.

A transactional model may make plenty of sense, but smells more like an extension than something to build into the engine API.
comment