Campbell LoggernetSrc

About this document

This document provides information about how to obtain and use the LoggerNetSrc developed in UCSD. It describes the deployment steps, and focuses on how to configure the input parameters for the particular user’s settings.

LoggerNetSrc is a software component that connects the LoggerNet software developed by Campbell Scientific Inc. to the open source streaming data milddleware, DataTurbine (http://www.dataturbine.org) which was developed by Creare.

Overview of LoggerNetSrc

LoggerNetSrc is a JAVA program that inserts data recorded by LoggerNet into DataTurbine(DT).

LoggerNet typically writes the sampled data into a file when it downloads the data from the datalogger. This file generated by the LoggerNet program serves s the interface between LoggerNetSrc, and LoggerNet. Specifically, LoggerNetSrc parses the data file, generated by LoggerNet. LoggerNet has two common configuration options for generating (writing to) the output file after downloading sensor data: an overwrite mode and the append mode. LoggerNetSrc handles both cases, by letting user specify the option in the xml configuration file.

Once configured appropriately, LoggerNetSrc then inserts the parsed data into DataTurbine server. One of the design goals of LoggetNetSrc programs is to handle network and server failures. To that end, if either the network or the server is down, LoggerNetSrc caches the data in a text file that it maintains internally. Next time the program is called, LoggerNetSrc checks the conditions. Data is logged to this internal file in the append mode if the network connectivity is not available or the remote DataTurbine server is still unavailable. Only when the network connectivity is reestablished and server becomes available, it continues streaming the data from where it stopped. This ensures that no piece of data is lost during this time.

LoggetNetSrc therefore simplifies the development and deployment of applications in challenged network environments by freeing the application developer from handling complex issue of network disconnections and server failures in distributed environments. Applications written on using this framework can leverage a common system for data synchronization that handles network outages as well as resolving data conflicts arising from network partitions.

How to install

LoggerNetSrc can be obtained here|

Requirements:
–    Java 1.4+
–    DataTurbine: LoggerNetSrc imports packages from rbnb.jar.  When the user installs the DataTurbine from this website: http://code.google.com/p/dataturbine/

The source code contains ANT build file.  The only variable that the user must set is the RBNB_HOME. RBNB_HOME should point to the directory where DataTurbine is installed.

The ant builder creates oss-apps.jar file.  LoggerNetSrc is a DataTurbine source program.  DataTurbine Source programs need several parameters.  For further details on the concept of DataTurbine Source and information about its parameters, please refer to www.dataturbine.org.

How to configure

The following example shows contents of the configuration file:

<?xml version=”1.0″ encoding=”UTF-8″?>
<!DOCTYPE properties SYSTEM “http://java.sun.com/dtd/properties.dtd”>
<properties>
<comment>Application Configuration</comment>
<entry key=”ConfigFilePath”>C:\Campbellsci\LoggerNet\LoggerNetParam.xml</entry>
<entry key=”AppendMode”>Yes</entry>
<entry key=”tempFilePath”>C:\Campbellsci\LoggerNet\LoggerNetSource.tmp</entry>
<entry key=”lineNumFilePath”>C:\Campbellsci\LoggerNet\U_lineNum.txt</entry>
<entry key=”Delimiter”>comma</entry>
<entry key=”TimeZoneOffsetSpecified”>no</entry>
<entry key=”SensorInfoLineNumber”>0</entry>
<entry key=”ChannelNameLineNumber”>1</entry>
<entry key=”UnitLineNumber”>2</entry>
<entry key=”ExtraInfoLineNumbers”>3</entry>
<entry key=”DataLineNumber”>4</entry>
<entry key=”LoggerNetDataFilePath”>C:\Campbellsci\LoggerNet\CR3000_LS_30Min.dat</entry>
</properties>

The user must supply the values for the entries in the xml document.

  • ConfigFilPath – This variable contains a path to the configuration file
  • AppendMode – The values need to be Yes or No.  It describes whether or not the LoggerNet software writes the data file in the append mode.
  • tempFilePath – This temp file is used when when the network or DT server fails.  This variable contains the path to the temp file that will be created when such failures happen.
  • lineNumFilePath – In the append mode, LoggerNetSrc processes the file partially (only on the new data).  The last processed line number is saved in this file.  This file is usually created by the program.  Please delete this file if the user desires to reinsert all the values (for example, when the user wishes to restart the process).
  • Delimiter – It can be either a comma or a tab.
  • TimeZoneOffsetSpecified – This variable indicates whether LoggerNet software writes the time with UTC time zone offset.  It can be either yes or no.
  • SensorInfoLineNumber – Typically, the first line of the LoggerNet data file has sensor information.  This variable contains the line number for such a line.  Please be aware that the line number starts from 0. This key usually has the value 0.
  • ChannelNameLineNumber – Typically, the second line of the LoggerNet data file contains the names of the channels separated by the delimiter.  This variable contains the line number for such a line.  Please be aware that the line number starts from 0. This key usually has the value of 1.
  • UnitLineNumber – Following the channel names, the user may supply corresponding units. This variable contains the line number for such a line.  Please be aware that the line number starts from 0.
  • ExtraInfoLineNumbers – Sometimes, the data file from LoggerNet contains many lines with extra information.  The user can specify such lines in this variable, separated by a comma.  Please be aware that the line number starts from 0.
  • DataLineNumber – This variable contains the line number that the data line starts. Please be aware that the line number starts from 0.
  • LoggerNetDataFilePath – LoggerNetSrc program parses the data file that LoggerNet has produced.  This variable contains a path to the data file.

Here is an example of the data file from LoggerNet:

“TOACI1″,”CR510TD-PB”,”10minute”
“TMSTAMP”,”RECNBR”,”BattVolt”,”AirTemp”
“2008-09-19 11:50:00”,7332,14.67,24.22

How to run it

First, the user should specify the four variables.

DATATURBINE_SERVER=someServer.ucsd.edu
DATATURBINE_SOURCE_NAME=”LoggerNetSource”
DATATURBINE_CACHE=2048
DATATURBINE_ARCHIVE=10240

Then, the user can execute the LoggerNetSrc in such a way:

java -cp %CLASSPATH%;%RBNB_HOME%\bin\rbnb.jar;C:\dataturbine\oss-apps.jar edu.ucsd.osdt.source.numeric.LoggerNetSrc -S %DATATURBINE_SOURCE_NAME% -s %DATATURBINE_SERVER% -z %DATATURBINE_CACHE% -Z %DATATURBINE_ARCHIVE% -c “C:\dataturbine\LogParamNTL.xml”

Each option is described here:

Classpath (-cp):
–    existing ones:  %CLASSPATH%
–    rbnb.jar must be referenced: %RBNB_HOME%\bin\rbnb.jar
–    executable jar the user compiled: C:\dataturbine\oss-apps.jar (Note that the user must supply the path to the compiled jar)

Name of the LoggerNetSrc program:
edu.ucsd.osdt.source.numeric.LoggerNetSrc

Name of the DataTurbine Source program (-S):
DATATURBINE_SOURCE_NAME

Name of the DataTurbine Server (-s):
DATATURBINE_SERVER – this can be an IP address as well.

Number of data points that are cached in the memory at the DataTurbine server (-z):
DATATURBINE_CACHE

Number of data points that are cached in the hard drive at the DataTurbine server (-Z):
DATATURBINE_ARCHIVE

Name of the XML configuration file (-c):
–    The user must specify this. C:\dataturbine\LogParamNTL.xml

Open Source DataTurbine Initiative © 2016 Frontier Theme