Personal tools
You are here: Home Oceans IE Topics of Interest Logistics for Data Turbine Demo Data Turbine and OOSTethys toolkits integration
Log in


Forgot your password?
New user?
 

Data Turbine and OOSTethys toolkits integration

DataTurbine, as other real time distributed system is asynchronous, while SOS is synchronous. In asynchronous systems data lives on the wire, and other systems can publish or subscribe data to the wire. In the case of DataTurbine the wire is called buffered ring, publishers are called sources and subscribers are called sinks. SOS is synchronous since there are operations that follow request/response pattern, such as HTTP GET. For example describeSensor returns an XML with a SensorML document.

There are advantages of asynchronous real time distributed systems, such as speed of transaction and scalability on peer-to-peer distributed systems. However complex queries that required much more complex data models are hard to achieve in asynchronous systems. For example, querying data streams giving a geospatial extend and a specified procedure ( e.g. sensor ID).

The following are the approaches to integrate asynchronous systems, such as Data Turbine with synchronous systems, such as OOSTethys SOS JAVA toolkit (hereafter SOS Server):

- Implementation of messaging client in the back end of the SOS Server. In this case the SOS Server instantiates a Data Turbine Sink for channels of interest. A harvester stores the data in a data base so the SOS Server can respond to queries that are not supported by Data Turbine, such as spatial extent or filtering by property.
- Implementation of an observation offering in an SOS server whose result can be access in asynchronous mechanism. In this case the result in the observation doesn’t contains data, but contains metadata of how to access  DataTurbine ring buffer, such as host string and channel name.

The ideal case would be to construct an SOS server automatically from DataTurbine channels. Each Channel has name, userInfo and Data. The userInfo can be used to set metadata that can provide  information to construct a getCapabilities, sensorML or observationOffering metadata in the SOS Server. 

 

Data Turbine model:

  • ChannelMap has Channels
  • Each Channel has 
    • name
    • userInfo ( metadata ), Via the method:  PutUserInfo method we could add SensorML
  • The data to the channel could be added as follows:
    • cmap.PutTime(secondsInDoubleUTC,0);
    • cmap.PutDataAsString(chanIndex,"2009-06-10T18:28:41Z,36.69623,-122.39965,12.929,3.88978,33.100174"); This is the same as the data block in SweCommon

But the problem with this approach is that current implementations of DataTurbine commonly puts data as individual values and not blocks of data.

Ideally we would like to have a channel being dedicated for metadata,  this way we can also keep provenance of metadata records. This can be done with the userInfo for that channel, where we can point to the channel metadata via the name. But again, there is already information that communities use that is being coded in userInfo.

Maybe a higher level abstraction can be coded where a channel - has Metadata - another channel, and where the relation can be explicitly created. This will require augmenting DataTurbine model and not reusing the existing artifacts (e.g. userInfo).




Document Actions