Universal Data Bin File (UDBF) format

Revisions

Version	Date	Comment
V1.07	2019-02-27	* VariableAdditionalDataStructID = 3: added
V1.07	2018-10-09	Added clear note that additional data has to be ignored in case of unknown (new) structure ID to keep compatibility
V1.07	2018-09-07	Removed invalid XmlRpcValueLen
V1.07	2017-07-01	* AdditionalDataStructID = 3: added
V1.07	2016-10-03	* Some naming changed; * AdditionalData part restructured; * AdditionalDataStructID = 2: added
V1.07	2015-06-03	Some small document layout changes @ section “ModuleAdditionalData”;
V1.07	2014-03-03	Some changes in ModuleAdditionalData structure without changing version (until now not used ⇒ ModuleAdditionalDataLen = 0);

Introduction

UDBF is Gantner Instrumnets'proprietary file format for storing measurement data in the binary structure. It creates a minimum of file size overhead and ensures high processing performance because it simply reflects the data structure on the hardware:

The general structure of binary data:

Binary data stream
Binary Header Data	Describes the data stream: * general metadata * start time\time format definitions * channel information (name, unit, data type,…)
Separation	* separation chars if header and data are merged together * or header and data in separate files * or header and data as separate interface queries
Binary Data	* raw binary data without any separation * data structure according to header information * every measurement cycle of the system is stored as a single “row/frame/line of data/..” * always starting with the time stamp followed by all channels of this stream * timestamp according to time decoding rules in the header

Please take note of the compatibility issues when working with a Q.station controller.

File format

Header

Bytes	Data Type	Description
1 byte	Unsigned Integer	IsBigEndian
		0	IsLittleEndian (like Intel CPUs)
		<> 0	IsBigEndian (like Motorola CPUs)
		Attention: If the decoding CPU has a different Endian as marked in this file, it needs to swap all data bytes, to get the right access. Only 1-byte data and for this also 1-byte data array like strings do not need to be swapped. Swapping must be done like: e.g. 4-byte integer which is B0 B1 B2 B3 ⇒ B3 B2 B1 B0.
2 byte	Unsigned Integer	Version The version of file structure times 100. E.g. dec 110 means version 1.10 (actual version: v1.07).
2 byte	Unsigned Integer	TypeVendorLen If version >= v1.06, then delivers a number of 1-byte integers, which are used for TypeVendor.
1 byte	Unsigned Integer Array	TypeVendor If version >= v1.06, then TypeVendor including terminating \0. This is now: “UniversalDataBinFile – GANTNER Instruments”.
1 byte	Unsigned Integer	WithCheckSum If Version >= v1.01
		0	NoCheckSum
		<> 0	WithCheckSum
2 byte	Unsigned Integer	AdditionalDataLen Shows how many AdditionalData bytes are appended. If an AdditionalDataLen is set, it needs to be at least 18 bytes in length to have elements. AdditionalDataStructID is declared to determine how AdditionalData is coded. In case of an unknown AdditionalDataStructID, ignore AdditionalData by skipping AdditionalDataLen bytes.
4 byte	Unsigned Integer	Optional: FullMIDStruct part MAIN If AdditionalData is appended, this item needs to be set and is the main type of MID in standard coding.
4 byte	Unsigned Integer	Optional: FullMIDStruct part SUB If AdditionalData is appended, this item needs to be set and is the sub-type of MID in standard coding.
4 byte	Unsigned Integer	Optional: FullMIDStruct part FUNCTION If AdditionalData is appended, this item needs to be set and is the function type of MID in standard coding.
4 byte	Unsigned Integer	Optional: FullMIDStruct part CASING If AdditionalData is appended, this item needs to be set and is the casing type of MID in standard coding.
2 byte	Unsigned Integer	Optional: AdditionalDataStructID If AdditionalData is appended, this item needs to be set and is an ID to describe the used structure. In case of an unknown AdditionalDataStructID, ignore AdditionalData by skipping AdditionalDataLen bytes.
8 byte	Double (IEEE754)	Optional: AdditionalData If AdditionalDataLen is set to >= 18, all following data depend on a user-definable structure. How this structure is coded depends on AdditionalDataStructID.
		AdditionalDataStructID = 0 AND AdditionalDataLen = 18
		—
		AdditionalDataStructID = 1 AND AdditionalDataLen = 30
		AdditionalData Item 1 = CenterMethod	4-byte Unsigned Integer
			Arithmetic = 0
			Geometric = 1
			Manual = 2
		AdditionalData Item 2 = CenterX	4-byte Float (IEEE754)
		AdditionalData Item 3 = CenterY	4-byte Float (IEEE754)
		AdditionalDataStructID = 2 AND AdditionalDataLen > 18
		LocationLen Delivers a number of 1-byte integers, which are used for Location	2-byte Unsigned Integer
		Location Location (name) including terminating \0.	1 byte unsigned Integer Array
		SNRLen Delivers a number of 1-byte integers, which are used for SerialNumber	2-byte Unsigned Integer
		SNR Serial number including terminating \0.	1 byte unsigned Integer Array
		AppVersionLen Delivers a number of 1-byte integers, which are used for AppVersion	2-byte Unsigned Integer
		AppVersion Firmware version including terminating \0.	1 byte unsigned Integer Array
		UIDLen Delivers a number of 1-byte integers, which are used for container device UID	2-byte Unsigned Integer
		UID Container device UID including terminating \0.	1 byte unsigned Integer Array
		AdditionalDataStructID = 3 AND AdditionalDataLen > 18
		XmlRpcValue XmlRpc-Struct based info value (can be XML or JSON). Use class CGIns_UDBFHeaderInfo from GInsCommon/GInsXmlRpcStdAPI to decode content, including terminating \0. Size of XmlRpcValue is AdditionalDataLen - 18.	1 byte unsigned Integer Array
8 byte	Double (IEEE754)	StartTimeToDayFactor This is the factor of the following StartTime with respect to a day. StartTime*StartTimeToDayFactor = Time in [days]
2 byte	Unsigned Integer	dActTimeDataType If version >= 1.07. Since version 1.07, the data type of the controller timestamp is configurable. (See “Variable settings” for data type coding). If version < 1.07 timestamp data type = UnSignedInt32
8 byte	Double (IEEE754)	dActTimeToSecondFactor This is the factor of the following dActTime (in the time/data section) with respect to a second. dActTime*dActTimeToSecondFactor = Time in [s] If this is set to a value < =0.0 than no ActTime is included.
8 byte	Double (IEEE754)	StartTime This is the time at which the start event occurred. This time is based on all the following data items (dActTime).
8 byte	Double (IEEE754)	SampleRate This is the measurement frequency
2 byte	Unsigned Integer	VariableCount The number of configured variables (The variable for time stamp is NOT in the VariableCount, but is still in the data stream and has to be considered in the decoding with the format in → dActTimeDataType.)
Variable Settings Will be looped for as long as indicated by VariableCount
2 byte	Unsigned Integer	NameLen Delivers a number of 1-byte integers, which are used for VariableName.
1 byte	Unsigned Integer Array	Name Name of variable including terminating \0.
2 byte	Unsigned Integer	DataDirection Delivers data direction of variable in standard coding. Attention: if a variable data direction has no Input set, it also is not set in the time/data section.
		Input	0
		Output	1
		InputOutput	2
		Empty	3
2 byte	Unsigned Integer	DataType Delivers data type of variable in standard coding. Changed from V1.01 to V1.02
		No	0
		Boolean	1
		SignedInt8	2
		UnSignedInt8	3
		SignedInt16	4
		UnSignedInt16	5
		SignedInt32	6
		UnSignedInt32	7
		Float	8
		BitSet8	9
		BitSet16	10
		BitSet32	11
		Double	12
		SignedInt64	13
		UnSignedInt64	14
		BitSet64	15
		Old: No – 0, Boolean – 1, Integer (SInt16) – 2, Real (Float) – 3, BitSet8 – 4, BitSet16 – 5, LongInt (SInt32) – 6.
2 byte	Unsigned Integer	FieldLen Delivers the set field length.
2 byte	Unsigned Integer	Precision Delivers the set precision (see also DataType)
2 byte	Unsigned Integer	UnitLen Changed from V1.02 to V1.06. Delivers a number of 1-byte integers, which are used for Unit.
1 byte	Unsigned Integer Array	Unit Changed from v1.02 to v1.06 The unit, including terminating \0.
2 byte	Unsigned Integer	VariableAdditionalDataLen Shows how many VariableAdditionalData bytes are appended. If a VariableAdditionalDataLen is set, it needs to be at least 4 bytes in length to have elements VariableType and VariableAdditionalDataStructID declared to determine how VariableAdditionalData is coded. In case of unknown VariableAdditionalDataStructID, ignore VariableAdditionalData by scipping VariableAdditionalDataLen bytes.
2 byte	Unsigned Integer	Optional: VariableType If AdditionalData is appended, this item needs to be set and shows the type of variable in standard coding.
		Empty	0
		AnalogInput	1
		Arithmetic	2
		DigitalOutput	3
		DigitalInput	4
		SetPoint	5
		Alarm	6
		BitsetOutput	7
		BitsetInput	8
		PIDController	9
		AnalogOutput	10
		SignalConditioning	11
		RemoteInput	12
		Reference	13
2 byte	Unsigned Integer	Optional: VariableAdditionalDataStructID If VariableAdditionalData is appended, this item needs to be set and is an ID to describe the used structure. In case of unknown VariableAdditionalDataStructID, ignore VariableAdditionalData by skipping VariableAdditionalDataLen bytes!!
		Optional: VariableAdditionalData If VariableAdditionalDataLen is set to >= 4, all following data depend on a user-definable structure. How this structure is coded depends on VariableAdditionalDataStructID.
		VariableAdditionalDataStructID = 0 AND VariableAdditionalDataLen = 4
		—
		VariableAdditionalDataStructID = 1 AND VariableAdditionalDataLen = 10
		Data1 Deprecated	2-byte Unsigned Integer
		Data2 Deprecated	2-byte Unsigned Integer
		Data3 Deprecated	2-byte Unsigned Integer
		VariableAdditionalDataStructID = 2 AND VariableAdditionalDataLen > 4
		UIDLen Delivers a number of 1-byte integers, which are used for variable UID.	2-byte Unsigned Integer
		UID Variable UID including terminating \0.	1 byte unsigned Integer Array
		VariableAdditionalDataStructID = 3 AND VariableAdditionalDataLen > 4
		XmlRpcValue XmlRpc-Struct based info value (can be XML or JSON). Use class CGIns_UDBFHeaderVariableInfo from GInsCommon/GInsXmlRpcStdAPI to decode content, including terminating \0. Size of XmlRpcValue is AdditionalDataLen - 4	1 byte unsigned Integer Array

Separation Chars

There are separation characters (*) inserted. At least 8 pieces and maximal as many as needed so that the next valid data byte is written to a 16-bytes aligned address.

Time/Data

For each data item, a dActTime (configurable data type) is placed at the beginning. The appended data depend on variable settings listed in the header and are repeated for each defined variable.

Checksum

Changed from v1.00 to v1.01. When “WithCheckSum” is set a Checksum (4-byte unsigned integer) is placed at the end of the file. The Checksum is calculated from each byte of the file except the checksum itself. It is the 4-byte unsigned integer summation of mentioned bytes.

Decoding binary data

This section describes how to use UDBF header information to decode the related binary data stream.

The UDBF header information above has to be read, decoded, and stored first to have it available while decoding binary data.

Timestamp	CH1	CH2	CH3	Timestamp	CH1	CH2	CH3	Timestamp	…
1	0,12	50,2	12	2	0,13	50,1	12	3	…

Decoding the timestamp

The UDBF format always ensures that every frame has absolute time information independent of the timestamp data type. Therefore, the following formula has to be used for every timestamp type:

Timestamp * dActTimeToSecondFactor/86400 + StartTime * StartTimeToDayFactor = OLE automation time (double IEEE754)

OLE automation time = time in days since 30.12.1899 0:00:00.000000,
e.g. 2.5 = 1.1.1900 12:00:00.000000

Excel has an error inside if calculating with dates before March 1st, 1900:
The discrepancy between Excel and Visual Basic only exists for dates before March 1st, 1900. If you do the same experiment above and use “3/1/1900” as your test date, both Excel and VB will return 61.25. The discrepancy for earlier dates is due to the fact that Excel incorrectly considers 1900 to be a leap year, which was fixed in other implementations (in Excel, 2/29/1900 has an OADate of 60, while VB considers 60 to be 2/28). The different origin date for VB was chosen so that the Leap Year bug in Excel could be fixed in other implementations whiles still keeping the same value for current dates (for more info: from (http://social.msdn.microsoft.com/forums/en-US/isvvba/thread/747d3fbf-85b5-43f6-b4a5-1de88b0dad2d).

Decoding channel data

Depending on the data type of every channel, length and offset in the binary stream can be determined. A byte swap is necessary for all values with more than 1 byte if the decoding CPU has a different Endian as marked in the header. Integer values have to be divided by 10^ precision.