backlog/230

Implements the issue https://github.com/telefonicaid/fiware-connectors/issues/230

Manually tested against https://data.lab.fi-ware.org. Example:

    time=2014-10-22T15:58:23.881CEST | lvl=INFO | trans=1413986302-170-0000000000 | function=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler[130] : Starting transaction (1413986302-170-0000000000)
    time=2014-10-22T15:58:23.884CEST | lvl=INFO | trans=1413986302-170-0000000000 | function=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler[206] : Received data (<notifyContextRequest><subscriptionId>51c0ac9ed714fb3b37d7d5a8</subscriptionId><originator>localhost</originator><contextResponseList><contextElementResponse><contextElement><entityId type="Room" isPattern="false"><id>Room3</id></entityId><contextAttributeList><contextAttribute><name>temperature</name><type>centigrade</type><contextValue>57.89</contextValue></contextAttribute></contextAttributeList></contextElement><statusCode><code>200</code><reasonPhrase>OK</reasonPhrase></statusCode></contextElementResponse></contextResponseList></notifyContextRequest>)
    time=2014-10-22T15:58:23.884CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler[211] : Adding flume event header (name=content-type, value=application/xml)
    time=2014-10-22T15:58:23.884CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler[213] : Adding flume event header (name=fiware-service, value=santander)
    time=2014-10-22T15:58:23.884CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler[216] : Adding flume event header (name=transactionId, value=1413986302-170-0000000000)
    time=2014-10-22T15:58:23.884CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler[218] : Adding flume event header (name=ttl, value=5)
    time=2014-10-22T15:58:23.886CEST | lvl=INFO | trans=1413986302-170-0000000000 | function=getEvents | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.handlers.OrionRestHandler[224] : Event put in the channel (id=429231097, ttl=5)
    time=2014-10-22T15:58:23.888CEST | lvl=INFO | trans=1413986302-170-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[128] : Event got from the channel (id=429231097, headers={timestamp=1413986303887, content-type=application/xml, transactionId=1413986302-170-0000000000, fiware-service=santander, ttl=5}, bodyLength=561)
    time=2014-10-22T15:58:23.939CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=initOrg | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[109] : Organization not found in the cache, querying CKAN for it (orgName=santander)
    time=2014-10-22T15:58:23.947CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[769] : CKAN operation: GET https://data.lab.fi-ware.org:443/api/3/action/organization_show?id=santander HTTP/1.1
    time=2014-10-22T15:58:24.657CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[791] : CKAN response (7882 bytes): HTTP/1.1 200 OK
    time=2014-10-22T15:58:24.657CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[794] : response payload: {"help": "Return the details of a organization...
    time=2014-10-22T15:58:24.708CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=initOrg | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[130] : Organization found (orgName=santander, orgId=1907b4f7-2f88-4edf-9d5b-5efe1bae8bba)
    time=2014-10-22T15:58:24.708CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=initOrg | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[147] : Going to iterate on the packages looking for the default one (orgName=santander, defaultPkgName=santander_fiware-test)
    time=2014-10-22T15:58:24.709CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=initOrg | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[197] : Default package not found, going to create it (orgName=santander, defaultPkgName=santander_fiware-test)
    time=2014-10-22T15:58:24.709CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[756] : request payload: { "name": "santander_fiware-test", "owner_org": "1907b4f7-2f88-4edf-9d5b-5efe1bae8bba" }
    time=2014-10-22T15:58:24.712CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[769] : CKAN operation: POST https://data.lab.fi-ware.org:443/api/3/action/package_create HTTP/1.1
    time=2014-10-22T15:58:29.055CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[791] : CKAN response (5402 bytes): HTTP/1.1 200 OK
    time=2014-10-22T15:58:29.055CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[794] : response payload: {"help": "Create a new dataset...
    time=2014-10-22T15:58:29.058CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=createPackage | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[567] : Successful package creation (pkgId=d7deae3b-3269-4fed-82ce-b78a9afe01ad)
    time=2014-10-22T15:58:29.059CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=initOrg | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[201] : Default package added to pckages map (orgName=santander -> defaultPkgId=d7deae3b-3269-4fed-82ce-b78a9afe01ad)
    time=2014-10-22T15:58:29.060CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=persist | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionCKANSink[177] : Processing context element (id=Room3, type=Room)
    time=2014-10-22T15:58:29.060CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=persist | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionCKANSink[215] : Processing context attribute (name=temperature, type=centigrade)
    time=2014-10-22T15:58:29.061CEST | lvl=INFO | trans=1413986302-170-0000000000 | function=persist | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionCKANSink[218] : Persisting data at OrionCKANSink. <1413986303887, 2014-10-22T15:58:23.887, santander, Room3-Room, temperature, centigrade, "57.89", []>
    time=2014-10-22T15:58:29.063CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=resourceLookupAndCreate | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[314] : Resource id not found in the map, going to create it (orgName=santander, resourceName=Room3-Room)
    time=2014-10-22T15:58:29.063CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[756] : request payload: { "name": "Room3-Room", "url": "http://orion.lab.fi-ware.org:1026/ngsi10/contextEntitites/Room3", "package_id": "d7deae3b-3269-4fed-82ce-b78a9afe01ad" }
    time=2014-10-22T15:58:29.063CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[769] : CKAN operation: POST https://data.lab.fi-ware.org:443/api/3/action/resource_create HTTP/1.1
    time=2014-10-22T15:58:30.256CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[791] : CKAN response (2158 bytes): HTTP/1.1 200 OK
    time=2014-10-22T15:58:30.258CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[794] : response payload: {"help": "Appends a new resource to a datasets list of resources...
    time=2014-10-22T15:58:30.260CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=createResource | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[628] : Successful resource creation (resource id=7f283e9b-2517-4feb-9e99-a421f4d6f95b)
    time=2014-10-22T15:58:30.260CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[756] : request payload: { "resource_id": "7f283e9b-2517-4feb-9e99-a421f4d6f95b", "fields": [ { "id": "recvTimeTs", "type": "int"},{ "id": "recvTime", "type": "timestamp"},{ "id": "attrName", "type": "text"},{ "id": "attrType", "type": "text"},{ "id": "attrValue", "type": "json"},{ "id": "attrMd", "type": "json"}], "force": "true" }
    time=2014-10-22T15:58:30.267CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[769] : CKAN operation: POST https://data.lab.fi-ware.org:443/api/3/action/datastore_create HTTP/1.1
    time=2014-10-22T15:58:30.936CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[791] : CKAN response (2536 bytes): HTTP/1.1 200 OK
    time=2014-10-22T15:58:30.936CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[794] : response payload: {"help": "Adds a new table to the DataStore...
    time=2014-10-22T15:58:30.937CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=createDataStore | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[674] : Successful datastore creation (resourceId=7f283e9b-2517-4feb-9e99-a421f4d6f95b)
    time=2014-10-22T15:58:30.937CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=resourceLookupAndCreate | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[327] : Resource id added to resources map (<orgName,resourceName>=<santander,Room3-Room> -> resourceId=7f283e9b-2517-4feb-9e99-a421f4d6f95b)
    time=2014-10-22T15:58:30.938CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[756] : request payload: { "resource_id": "7f283e9b-2517-4feb-9e99-a421f4d6f95b", "records": [ { "recvTimeTs": "1413986303", "recvTime": "2014-10-22T15:58:23.887", "attrName": "temperature", "attrType": "centigrade", "attrValue": "57.89" } ], "method": "insert", "force": "true" }
    time=2014-10-22T15:58:30.938CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[769] : CKAN operation: POST https://data.lab.fi-ware.org:443/api/3/action/datastore_upsert HTTP/1.1
    time=2014-10-22T15:58:31.335CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[791] : CKAN response (1703 bytes): HTTP/1.1 200 OK
    time=2014-10-22T15:58:31.335CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=doCKANRequest | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[794] : response payload: {"help": "Updates or inserts into a table in the DataStore...
    time=2014-10-22T15:58:31.335CEST | lvl=DEBUG | trans=1413986302-170-0000000000 | function=insert | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.backends.ckan.CKANBackendImpl[415] : Successful insert (resource/datastore id=7f283e9b-2517-4feb-9e99-a421f4d6f95b)
    time=2014-10-22T15:58:31.339CEST | lvl=INFO | trans=1413986302-170-0000000000 | function=process | comp=Cygnus | msg=es.tid.fiware.fiwareconnectors.cygnus.sinks.OrionSink[176] : Finishing transaction (1413986302-170-0000000000)
Are we doing some "string conformance" in ckanHost and ckanPort string? E.g., to avoid the user set "http://10.1.1.1" instead of "10.1.1.1" in ckanHost.

Probably not for this PR, but for a tech debt should issue.