Repository: pymodbus-dev/pymodbus

Issue:
Registers in Database DataStore cant be read from remote client
I try to implement a database backed TCP server so that my registers will be kept in database. For that task I modulated "Updating Server" example . When I run the code I can query regısters from inside. But I cant read same registers from a client located on another computer. With normal ModbusSlaveContext I was able to do that. Results are attached. and here is my code:

```
'''
Pymodbus Server With Updating Thread
--------------------------------------------------------------------------
This is an example of having a background thread updating the
context while the server is operating. This can also be done with
a python thread::
    from threading import Thread
    thread = Thread(target=updating_writer, args=(context,))
    thread.start()
'''
#---------------------------------------------------------------------------# 
# import the modbus libraries we need
#---------------------------------------------------------------------------# 
from pymodbus.server.async import StartTcpServer
from pymodbus.device import ModbusDeviceIdentification
from pymodbus.datastore import ModbusSequentialDataBlock
from pymodbus.datastore import ModbusServerContext
from database_store import DatabaseSlaveContext
from pymodbus.transaction import ModbusRtuFramer, ModbusAsciiFramer

#---------------------------------------------------------------------------# 
# import the twisted libraries we need
#---------------------------------------------------------------------------# 
from twisted.internet.task import LoopingCall

#---------------------------------------------------------------------------# 
# configure the service logging
#---------------------------------------------------------------------------# 
import logging
logging.basicConfig()
log = logging.getLogger()
log.setLevel(logging.DEBUG)

#---------------------------------------------------------------------------# 
# define your callback process
#---------------------------------------------------------------------------# 
def updating_writer(a):
    ''' A worker process that runs every so often and
    updates live values of the context. It should be noted
    that there is a race condition for the update.
    :param arguments: The input arguments to the call
    '''
    log.debug("updating the context")
    context  = a[0]
    readfunction = 0x03 # read holding registers
    writefunction = 0x10 # wrıte holding registers
    slave_id = 0x00 # slave address
    address  = 16 # adress : 400017


    values = context[slave_id].getValues(readfunction, address, count=3)

    log.debug("new values: " + str(values))


#---------------------------------------------------------------------------# 
# initialize your data store
#---------------------------------------------------------------------------#
block = ModbusSequentialDataBlock(0x00, [0]*0xff)
store = DatabaseSlaveContext(block)

context = ModbusServerContext(slaves=store, single=True)


#---------------------------------------------------------------------------# 
# initialize the server information
#---------------------------------------------------------------------------# 
identity = ModbusDeviceIdentification()
identity.VendorName  = 'pymodbus'
identity.ProductCode = 'PM'
identity.VendorUrl   = 'http://github.com/bashwork/pymodbus/'
identity.ProductName = 'pymodbus Server'
identity.ModelName   = 'pymodbus Server'
identity.MajorMinorRevision = '1.0'

#---------------------------------------------------------------------------# 
# run the server you want
#---------------------------------------------------------------------------# 
time = 5 # 5 seconds delay
loop = LoopingCall(f=updating_writer, a=(context,))
loop.start(time, now=False) # initially delay by time
StartTcpServer(context, identity=identity, address=("", 5007))

```
![image 4](https://cloud.githubusercontent.com/assets/7521396/20281509/cea1165c-aac1-11e6-8897-f914fafca04d.png)
![image 5](https://cloud.githubusercontent.com/assets/7521396/20281510/cea3a250-aac1-11e6-8337-5150de5c0a25.png)





  


Discussion:
I am sorry if this is late, but are you still having this problem ?

Yes I am still having this problem, @dhoomakethu 
@sansal54 , check out the `dev` branch for a fix. Please get back to us if you still face problems
Hi All, Could you please someone help me how to map custom based registers to kept in the database for Modbus?  I am very new to python and working on Modbus. Please help me in this regards.
@sansal54, In the above code "from database_store import DatabaseSlaveContext" -- could you please help me how to create own database according to Modbus registers and add to the server example?





@deverakondabalakrishna You need to be more specific on what exactly you would like us to help with. The examples have the explanantion on various datastores and datablocks (refer  [synchronous_server](https://github.com/riptideio/pymodbus/blob/master/examples/common/synchronous_server.py) ). I am pasting the content here for quick reference. 
```
# ----------------------------------------------------------------------- # 
    # initialize your data store
    # ----------------------------------------------------------------------- # 
    # The datastores only respond to the addresses that they are initialized to
    # Therefore, if you initialize a DataBlock to addresses of 0x00 to 0xFF, a
    # request to 0x100 will respond with an invalid address exception. This is
    # because many devices exhibit this kind of behavior (but not all)::
    #
    #     block = ModbusSequentialDataBlock(0x00, [0]*0xff)
    #
    # Continuing, you can choose to use a sequential or a sparse DataBlock in
    # your data context.  The difference is that the sequential has no gaps in
    # the data while the sparse can. Once again, there are devices that exhibit
    # both forms of behavior::
    #
    #     block = ModbusSparseDataBlock({0x00: 0, 0x05: 1})
    #     block = ModbusSequentialDataBlock(0x00, [0]*5)
    #
    # Alternately, you can use the factory methods to initialize the DataBlocks
    # or simply do not pass them to have them initialized to 0x00 on the full
    # address range::
    #
    #     store = ModbusSlaveContext(di = ModbusSequentialDataBlock.create())
    #     store = ModbusSlaveContext()
    #
    # Finally, you are allowed to use the same DataBlock reference for every
    # table or you you may use a seperate DataBlock for each table.
    # This depends if you would like functions to be able to access and modify
    # the same data or not::
    #
    #     block = ModbusSequentialDataBlock(0x00, [0]*0xff)
    #     store = ModbusSlaveContext(di=block, co=block, hr=block, ir=block)
    #
    # The server then makes use of a server context that allows the server to
    # respond with different slave contexts for different unit ids. By default
    # it will return the same context for every unit id supplied (broadcast
    # mode).
    # However, this can be overloaded by setting the single flag to False and
    # then supplying a dictionary of unit id to context mapping::
    #
    #     slaves  = {
    #         0x01: ModbusSlaveContext(...),
    #         0x02: ModbusSlaveContext(...),
    #         0x03: ModbusSlaveContext(...),
    #     }
    #     context = ModbusServerContext(slaves=slaves, single=False)
    #
    # The slave context can also be initialized in zero_mode which means that a
    # request to address(0-7) will map to the address (0-7). The default is
    # False which is based on section 4.4 of the specification, so address(0-7)
    # will map to (1-8)::
    #
    #     store = ModbusSlaveContext(..., zero_mode=True)
    # ----------------------------------------------------------------------- # 
```

For SQLSlave context refer the documentation https://pymodbus.readthedocs.io/en/latest/source/library/pymodbus.datastore.database.html#module-pymodbus.datastore.database.sql_datastore
@dhoomakethu! Thank you for the above suggestions! I am able to read and write from the database in the slave side and able print on console but when I am trying from client side I am not able to read any data from registers. When I am trying to read hold register address from the client I am getting "Illegal Address". The below are the codes and console output. 
my objective is to get database datastore register values ( server machine ) from client machine.
I am bit of confused with modbusslavecontest and sqlslavecontext. Please help me in this regards.

slave:
```
def updating_writer(a):

    log.debug("Updating the database context")
    context  = a[0]
    readfunction = 0x03 # read holding registers
    writefunction = 0x10
    slave_id = 0x01 # slave address
    #address = 40010
    #count = 50

    Context[slave_id].setValues(writefunction, 40000, [111]*0x08)
    values1 = context[slave_id].getValues(readfunction, 40000, count=7)
   #log.debug("Values from datastore: " + str(values1))
    print(values1)
    
def run_dbstore_update_server():
    # ----------------------------------------------------------------------- #
    # initialize your data store
    # ----------------------------------------------------------------------- #
    block = ModbusSequentialDataBlock(0x01, [888]*0xFF)
    #store = ModbusSlaveContext(hr=block) 
    store = SqlSlaveContext(block)
    #store = DatabaseSlaveContext(block)
    slaves  = {
                       0x01: store,
                    }
    context = ModbusServerContext(slaves={1: store}, single=False)
    #context = ModbusServerContext(slaves=slaves, single=False)

    # initialize the server information
    # ----------------------------------------------------------------------- #
    identity = ModbusDeviceIdentification()
    identity.VendorName = 'PyModbusServer'
    identity.ProductCode = 'PM'
    #identity.VendorUrl = 'http://github.com/riptideio/pymodbus/'
    identity.ProductName = 'Energy Hub'
    identity.ModelName = 'MPP'
    identity.MajorMinorRevision = '1.0'
    # ----------------------------------------------------------------------- #
    # run the server you want
    # ----------------------------------------------------------------------- #
    time = 5  # 5 seconds delay
    loop = LoopingCall(f=updating_writer, a=(context,))
    loop.start(time, now=False)  # initially delay by time
    StartTcpServer(context, identity=identity, address=("localhost", 5058))
if __name__ == "__main__":
    run_dbstore_update_server()
```

```
console output:
-----------------
 [111, 111, 111, 111, 111, 111, 111, 111]
========================================================
```
Client:
```
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
from pymodbus.mei_message import ReadDeviceInformationRequest
from pymodbus.device import ModbusDeviceIdentification
UNIT = 0x01
def run_sync_client():
    client = ModbusClient('localhost', port=5058)
    client.connect()
    log.debug("Reading Device Information")
    information = {}
    rr = None

    while not rr or rr.more_follows:
        next_object_id = rr.next_object_id if rr else 0
        rq = ReadDeviceInformationRequest(read_code=0x03, unit=UNIT,
                                          object_id=next_object_id)
        rr = client.execute(rq)
        information.update(rr.information)
        log.debug(rr)

    print("Device Information : ")
    for key in information.keys():
        print(key, information[key])

    print client.read_holding_registers(40000, 2, unit=UNIT)
    rh1 = client.read_holding_registers(0x16,1, unit=UNIT)
    print(rh1)
    #print(rh1.registers)
 client.close()
if __name__ == "__main__":
    run_sync_client()
```

console output:
------------------
```
Device Information : 
(0, 'PyModbusServer')
(1, 'PM')
(2, '1.0')
(4, 'Energy Hub')
(5, 'MPP')
Exception Response(131, 3, IllegalAddress)
Exception Response(131, 3, IllegalAddress)
```

Resolving patch:
diff --git a/pymodbus/datastore/database/sql_datastore.py b/pymodbus/datastore/database/sql_datastore.py
--- a/pymodbus/datastore/database/sql_datastore.py
+++ b/pymodbus/datastore/database/sql_datastore.py
@@ -108,7 +108,7 @@ def _get(self, type, offset, count):
         query = self._table.select(and_(
             self._table.c.type == type,
             self._table.c.index >= offset,
-            self._table.c.index <= offset + count)
+            self._table.c.index <= offset + count - 1)
         )
         query = query.order_by(self._table.c.index.asc())
         result = self._connection.execute(query).fetchall()
@@ -175,6 +175,6 @@ def _validate(self, type, offset, count):
         query = self._table.select(and_(
             self._table.c.type == type,
             self._table.c.index >= offset,
-            self._table.c.index <= offset + count))
-        result = self._connection.execute(query)
-        return result.rowcount == count
+            self._table.c.index <= offset + count - 1))
+        result = self._connection.execute(query).fetchall()
+        return len(result) == count


Test patch:
diff --git a/test/test_datastore.py b/test/test_datastore.py
--- a/test/test_datastore.py
+++ b/test/test_datastore.py
@@ -278,19 +278,16 @@ def testReset(self):
         self.slave._db_create.assert_called_once_with(
             self.slave.table, self.slave.database
         )
+
     def testValidateSuccess(self):
-        mock_result = MockSqlResult(
-            rowcount=len(self.mock_values)
-        )
-        self.slave._connection.execute = MagicMock(return_value=mock_result)
+        self.slave._connection.execute.return_value.fetchall.return_value = self.mock_values
         self.assertTrue(self.slave.validate(
             self.mock_function, self.mock_addr, len(self.mock_values))
         )
 
     def testValidateFailure(self):
         wrong_count = 9
-        mock_result = MockSqlResult(rowcount=len(self.mock_values))
-        self.slave._connection.execute = MagicMock(return_value=mock_result)
+        self.slave._connection.execute.return_value.fetchall.return_value = self.mock_values
         self.assertFalse(self.slave.validate(
             self.mock_function, self.mock_addr, wrong_count)
         )