DataTurbine Source

Introduction

A DataTurbine Source (also refereed to as an ‘on-ramp’) is a program that takes data from a target (for example a sensor or file) and puts it into a DataTurbine server.

A source runs independently from the server as a separate application and uses the network to communicate. It can run on the same machine as the server or across the world.

Each source can contain multiple channels each with its own data type. It controls its own server-side memory and hard drive space allocation

Anatomy of a Source

RBNB_Channels

  • Name: Identifies the source
  • Target Server: The server the source sends data to
  • Cache Size: Each source specifies how many frames of data to buffer for itself in the server’s memory (RAM).
  • Archive Size: Each source specifies how many frames of data to store on the server’s hard drive.
  • Multiple Channels: Data stream containing one type of data (for example numeric or video).
    In turn each channel consists of a :

    • Name: Identifies the specific channel
    • MIME Type: Media type the applications can use to make decisions about the data they are receiving. Each channel can only store one type of data.
    • Data: Series of data points consisting of a time and value

Practical Example:

For example let us imagine a simple meteorological tower that measures temperature and humidity on top of a hill. Nearby is a field station that is also measuring temperature. We want to get this data into DataTurbine on a laptop at the field station. Lets go over what we would do.

Assuming we have custom sources for our instrumentation.

RBNB_Channels_EX

  1. Start a DataTurbine Server on the laptop (rbnb.jar)
  2. Start a source on the laptop targeting our server that reads data from the meteorological tower and puts it into DataTurbine. This source would contain two channels (temperature & humidity)
  3. Start another source on the laptop that reads from the local field station and writes puts the data into the server. This source would contain a single channel (temperature)

Our laptop would now have three independent lightweight programs running. And now that we have the data in the server we now need a way to access it. This is discussed in the next section.

PlugIns

PlugIns are a specialized on-request type of data source. Whereas regular sources proactively push data to the DT server, plugins reply with data in response to sink requests forwarded to them via their plugin server connection.

Things to Keep in Mind

Each channel can only have one data type associated with it. Also remember that data cannot be back-loaded into the server. For each channel data has to be entered sequentially and so for a given channel each data point has to have a timestamp that is greater than the previous timestamp on record.

Open Source DataTurbine Initiative © 2017 Frontier Theme