Revisions
Version |
Date |
Comment |
---|---|---|
V1.07 |
2019-02-27 |
* VariableAdditionalDataStructID = 3: added |
V1.07 |
2018-10-09 |
Added clear note that additional data has to be ignored in case of unknown (new) structure ID to keep compatibility |
V1.07 |
2018-09-07 |
Removed invalid XmlRpcValueLen |
V1.07 |
2017-07-01 |
* AdditionalDataStructID = 3: added |
V1.07 |
2016-10-03 |
* Some naming changed; * AdditionalData part restructured; * AdditionalDataStructID = 2: added |
V1.07 |
2015-06-03 |
Some small document layout changes @ section “ModuleAdditionalData”; |
V1.07 |
2014-03-03 |
Some changes in ModuleAdditionalData structure without changing version (until now not used ⇒ ModuleAdditionalDataLen = 0); |
Introduction
UDBF is Gantner Instrumnets'proprietary file format for storing measurement data in the binary structure. It creates a minimum of file size overhead and ensures high processing performance because it simply reflects the data structure on the hardware:
The general structure of binary data:
Binary data stream |
|
---|---|
Binary Header Data |
Describes the data stream: * general metadata * start time\time format definitions * channel information (name, unit, data type,…) |
Separation |
* separation chars if header and data are merged together * or header and data in separate files * or header and data as separate interface queries |
Binary Data |
* raw binary data without any separation * data structure according to header information * every measurement cycle of the system is stored as a single “row/frame/line of data/..” * always starting with the time stamp followed by all channels of this stream * timestamp according to time decoding rules in the header |
Please take note of the compatibility issues when working with a Q.station controller.
File format
Header
Bytes |
Data Type |
Description |
|
---|---|---|---|
1 byte |
Unsigned Integer |
IsBigEndian |
|
0 |
IsLittleEndian (like Intel CPUs) |
||
<> 0 |
IsBigEndian (like Motorola CPUs) |
||
Attention: If the decoding CPU has a different Endian as marked in this file, it needs to swap all data bytes, to get the right access. Only 1-byte data and for this also 1-byte data array like strings do not need to be swapped. Swapping must be done like: e.g. 4-byte integer which is B0 B1 B2 B3 ⇒ B3 B2 B1 B0. |
|||
2 byte |
Unsigned Integer |
Version The version of file structure times 100. E.g. dec 110 means version 1.10 (actual version: v1.07). |
|
2 byte |
Unsigned Integer |
TypeVendorLen If version >= v1.06, then delivers a number of 1-byte integers, which are used for TypeVendor. |
|
1 byte |
Unsigned Integer Array |
TypeVendor If version >= v1.06, then TypeVendor including terminating \0. This is now: “UniversalDataBinFile – GANTNER Instruments”. |
|
1 byte |
Unsigned Integer |
WithCheckSum If Version >= v1.01 |
|
0 |
NoCheckSum |
||
<> 0 |
WithCheckSum |
||
2 byte |
Unsigned Integer |
AdditionalDataLen Shows how many AdditionalData bytes are appended. If an AdditionalDataLen is set, it needs to be at least 18 bytes in length to have elements. AdditionalDataStructID is declared to determine how AdditionalData is coded. In case of an unknown AdditionalDataStructID, ignore AdditionalData by skipping AdditionalDataLen bytes. |
|
4 byte |
Unsigned Integer |
Optional: FullMIDStruct part MAIN If AdditionalData is appended, this item needs to be set and is the main type of MID in standard coding. |
|
4 byte |
Unsigned Integer |
Optional: FullMIDStruct part SUB If AdditionalData is appended, this item needs to be set and is the sub-type of MID in standard coding. |
|
4 byte |
Unsigned Integer |
Optional: FullMIDStruct part FUNCTION If AdditionalData is appended, this item needs to be set and is the function type of MID in standard coding. |
|
4 byte |
Unsigned Integer |
Optional: FullMIDStruct part CASING If AdditionalData is appended, this item needs to be set and is the casing type of MID in standard coding. |
|
2 byte |
Unsigned Integer |
Optional: AdditionalDataStructID If AdditionalData is appended, this item needs to be set and is an ID to describe the used structure. In case of an unknown AdditionalDataStructID, ignore AdditionalData by skipping AdditionalDataLen bytes. |
|
8 byte |
Double (IEEE754) |
Optional: AdditionalData If AdditionalDataLen is set to >= 18, all following data depend on a user-definable structure. How this structure is coded depends on AdditionalDataStructID. |
|
AdditionalDataStructID = 0 |
|||
— |
|||
AdditionalDataStructID = 1 |
|||
AdditionalData Item 1 = CenterMethod |
4-byte Unsigned Integer |
||
Arithmetic = 0 |
|||
Geometric = 1 |
|||
Manual = 2 |
|||
AdditionalData Item 2 = CenterX |
4-byte Float (IEEE754) |
||
AdditionalData Item 3 = CenterY |
4-byte Float (IEEE754) |
||
AdditionalDataStructID = 2 |
|||
LocationLen Delivers a number of 1-byte integers, which are used for Location |
2-byte Unsigned Integer |
||
Location Location (name) including terminating \0. |
1 byte unsigned Integer Array |
||
SNRLen Delivers a number of 1-byte integers, which are used for SerialNumber |
2-byte Unsigned Integer |
||
SNR Serial number including terminating \0. |
1 byte unsigned Integer Array |
||
AppVersionLen Delivers a number of 1-byte integers, which are used for AppVersion |
2-byte Unsigned Integer |
||
AppVersion Firmware version including terminating \0. |
1 byte unsigned Integer Array |
||
UIDLen Delivers a number of 1-byte integers, which are used for container device UID |
2-byte Unsigned Integer |
||
UID Container device UID including terminating \0. |
1 byte unsigned Integer Array |
||
AdditionalDataStructID = 3 |
|||
XmlRpcValue XmlRpc-Struct based info value (can be XML or JSON). Use class CGIns_UDBFHeaderInfo from GInsCommon/GInsXmlRpcStdAPI to decode content, including terminating \0. Size of XmlRpcValue is AdditionalDataLen - 18. |
1 byte unsigned Integer Array |
||
8 byte |
Double (IEEE754) |
StartTimeToDayFactor This is the factor of the following StartTime with respect to a day. StartTime*StartTimeToDayFactor = Time in [days] |
|
2 byte |
Unsigned Integer |
dActTimeDataType If version >= 1.07. Since version 1.07, the data type of the controller timestamp is configurable. (See “Variable settings” for data type coding). If version < 1.07 timestamp data type = UnSignedInt32 |
|
8 byte |
Double (IEEE754) |
dActTimeToSecondFactor This is the factor of the following dActTime (in the time/data section) with respect to a second. dActTime*dActTimeToSecondFactor = Time in [s] If this is set to a value < =0.0 than no ActTime is included. |
|
8 byte |
Double (IEEE754) |
StartTime This is the time at which the start event occurred. This time is based on all the following data items (dActTime). |
|
8 byte |
Double (IEEE754) |
SampleRate This is the measurement frequency |
|
2 byte |
Unsigned Integer |
VariableCount The number of configured variables (The variable for time stamp is NOT in the VariableCount, but is still in the data stream and has to be considered in the decoding with the format in → dActTimeDataType.) |
|
Variable Settings Will be looped for as long as indicated by VariableCount |
|||
2 byte |
Unsigned Integer |
NameLen Delivers a number of 1-byte integers, which are used for VariableName. |
|
1 byte |
Unsigned Integer Array |
Name Name of variable including terminating \0. |
|
2 byte |
Unsigned Integer |
DataDirection Delivers data direction of variable in standard coding. Attention: if a variable data direction has no Input set, it also is not set in the time/data section. |
|
Input |
0 |
||
Output |
1 |
||
InputOutput |
2 |
||
Empty |
3 |
||
2 byte |
Unsigned Integer |
DataType Delivers data type of variable in standard coding. Changed from V1.01 to V1.02 |
|
No |
0 |
||
Boolean |
1 |
||
SignedInt8 |
2 |
||
UnSignedInt8 |
3 |
||
SignedInt16 |
4 |
||
UnSignedInt16 |
5 |
||
SignedInt32 |
6 |
||
UnSignedInt32 |
7 |
||
Float |
8 |
||
BitSet8 |
9 |
||
BitSet16 |
10 |
||
BitSet32 |
11 |
||
Double |
12 |
||
SignedInt64 |
13 |
||
UnSignedInt64 |
14 |
||
BitSet64 |
15 |
||
Old: No – 0, Boolean – 1, Integer (SInt16) – 2, Real (Float) – 3, BitSet8 – 4, BitSet16 – 5, LongInt (SInt32) – 6. |
|||
2 byte |
Unsigned Integer |
FieldLen Delivers the set field length. |
|
2 byte |
Unsigned Integer |
Precision Delivers the set precision (see also DataType) |
|
2 byte |
Unsigned Integer |
UnitLen Changed from V1.02 to V1.06. Delivers a number of 1-byte integers, which are used for Unit. |
|
1 byte |
Unsigned Integer Array |
Unit Changed from v1.02 to v1.06 The unit, including terminating \0. |
|
2 byte |
Unsigned Integer |
VariableAdditionalDataLen Shows how many VariableAdditionalData bytes are appended. If a VariableAdditionalDataLen is set, it needs to be at least 4 bytes in length to have elements VariableType and VariableAdditionalDataStructID declared to determine how VariableAdditionalData is coded. In case of unknown VariableAdditionalDataStructID, ignore VariableAdditionalData by scipping VariableAdditionalDataLen bytes. |
|
2 byte |
Unsigned Integer |
Optional: VariableType If AdditionalData is appended, this item needs to be set and shows the type of variable in standard coding. |
|
Empty |
0 |
||
AnalogInput |
1 |
||
Arithmetic |
2 |
||
DigitalOutput |
3 |
||
DigitalInput |
4 |
||
SetPoint |
5 |
||
Alarm |
6 |
||
BitsetOutput |
7 |
||
BitsetInput |
8 |
||
PIDController |
9 |
||
AnalogOutput |
10 |
||
SignalConditioning |
11 |
||
RemoteInput |
12 |
||
Reference |
13 |
||
2 byte |
Unsigned Integer |
Optional: VariableAdditionalDataStructID If VariableAdditionalData is appended, this item needs to be set and is an ID to describe the used structure. In case of unknown VariableAdditionalDataStructID, ignore VariableAdditionalData by skipping VariableAdditionalDataLen bytes!! |
|
Optional: VariableAdditionalData If VariableAdditionalDataLen is set to >= 4, all following data depend on a user-definable structure. How this structure is coded depends on VariableAdditionalDataStructID. |
|||
VariableAdditionalDataStructID = 0 |
|||
— |
|||
VariableAdditionalDataStructID = 1 |
|||
Data1 Deprecated |
2-byte Unsigned Integer |
||
Data2 Deprecated |
2-byte Unsigned Integer |
||
Data3 Deprecated |
2-byte Unsigned Integer |
||
VariableAdditionalDataStructID = 2 |
|||
UIDLen Delivers a number of 1-byte integers, which are used for variable UID. |
2-byte Unsigned Integer |
||
UID Variable UID including terminating \0. |
1 byte unsigned Integer Array |
||
VariableAdditionalDataStructID = 3 |
|||
XmlRpcValue XmlRpc-Struct based info value (can be XML or JSON). Use class CGIns_UDBFHeaderVariableInfo from GInsCommon/GInsXmlRpcStdAPI to decode content, including terminating \0. Size of XmlRpcValue is AdditionalDataLen - 4 |
1 byte unsigned Integer Array |
Separation Chars
There are separation characters (*) inserted. At least 8 pieces and maximal as many as needed so that the next valid data byte is written to a 16-bytes aligned address.
Time/Data
For each data item, a dActTime (configurable data type) is placed at the beginning. The appended data depend on variable settings listed in the header and are repeated for each defined variable.
Checksum
Changed from v1.00 to v1.01. When “WithCheckSum” is set a Checksum (4-byte unsigned integer) is placed at the end of the file. The Checksum is calculated from each byte of the file except the checksum itself. It is the 4-byte unsigned integer summation of mentioned bytes.
Decoding binary data
This section describes how to use UDBF header information to decode the related binary data stream.
The UDBF header information above has to be read, decoded, and stored first to have it available while decoding binary data.
Timestamp |
CH1 |
CH2 |
CH3 |
Timestamp |
CH1 |
CH2 |
CH3 |
Timestamp |
… |
---|---|---|---|---|---|---|---|---|---|
1 |
0,12 |
50,2 |
12 |
2 |
0,13 |
50,1 |
12 |
3 |
… |
Decoding the timestamp
The UDBF format always ensures that every frame has absolute time information independent of the timestamp data type. Therefore, the following formula has to be used for every timestamp type:
-
Timestamp * dActTimeToSecondFactor/86400 + StartTime * StartTimeToDayFactor = OLE automation time (double IEEE754)
OLE automation time = time in days since 30.12.1899 0:00:00.000000,
e.g. 2.5 = 1.1.1900 12:00:00.000000
Excel has an error inside if calculating with dates before March 1st, 1900:
The discrepancy between Excel and Visual Basic only exists for dates before March 1st, 1900. If you do the same experiment above and use “3/1/1900” as your test date, both Excel and VB will return 61.25. The discrepancy for earlier dates is due to the fact that Excel incorrectly considers 1900 to be a leap year, which was fixed in other implementations (in Excel, 2/29/1900 has an OADate of 60, while VB considers 60 to be 2/28). The different origin date for VB was chosen so that the Leap Year bug in Excel could be fixed in other implementations whiles still keeping the same value for current dates (for more info: from (http://social.msdn.microsoft.com/forums/en-US/isvvba/thread/747d3fbf-85b5-43f6-b4a5-1de88b0dad2d).
Decoding channel data
Depending on the data type of every channel, length and offset in the binary stream can be determined. A byte swap is necessary for all values with more than 1 byte if the decoding CPU has a different Endian as marked in the header. Integer values have to be divided by 10^ precision.