Why can a buffer overrun occur when streaming DAQ data to MATLAB?

A buffer overrun can occur when MATLAB cannot process incoming samples as quickly as the measurement buffer supplies them. Reduce the buffer sample rate or analyze high-speed recorded data offline when necessary.

How to Stream Gantner DAQ Measurement Data to MATLAB

Q: How can I stream Gantner DAQ measurement data to MATLAB?

Gantner DAQ measurement data can be streamed into MATLAB over Ethernet from a Q.station, Q.monixx or GI.bench PostProcess Buffer using the GInsData C-API or the GInsMex MEX interface.

Q: Do I need a compiler to use GInsData with MATLAB?

The legacy MATLAB loadlibrary and calllib workflow requires a supported C compiler. A compatible precompiled GInsMex file can avoid compiling the interface yourself.

Q: Can MATLAB import Gantner .DAT measurement files?

Yes. GInsMex can open Gantner .DAT measurement files and return channel measurement data and relative time values for analysis and visualization in MATLAB.

Gantner measurement data can be streamed into MATLAB over Ethernet from a Q.station, Q.monixx or GI.bench PostProcess Buffer. Engineers can access buffered data through the GInsData C-API using MATLAB shared-library functions or use the precompiled GInsMex MEX interface to connect, read channels, acquire samples and import recorded .DAT files.

To stream data to MATLAB, you need the GInsData Library. For more information about this library, please refer to this knowledge base article.

Methods for Streaming Gantner Measurement Data to MATLAB

Here are two methods for reading buffered data from a Q.series controller (Q.station or Q.monixx) or a PostProcess Buffer (GI.bench) into MATLAB:

Call DLL functions (C-API) directly in MATLAB: Refer to the section on using shared libraries.
Use a pre-compiled MEX file: You can use either the mexw64 file for 64-bit versions or the mexw32 file for 32-bit versions. Refer to the section on using MEX files.

To directly read measurement data into MATLAB, you can call the DLL within your MATLAB code. For example:

eGateUtility.dll (available only in 32-bit)
giutility.dll (GInsData → C-API: eGateHighSpeedPort API): available in both 32-bit and 64-bit versions.

Notes:

These methods are compatible with MATLAB on both Windows and Linux platforms.
The functions loadlibrary() and calllib() have been tested with MATLAB versions R2013a through R2021b (64-bit), using the "Microsoft Visual C++ 2010" compiler.
Please note that a supported compiler must be installed in MATLAB to run these functions. If a suitable compiler is not available, you will need to use the compiled MEX file (available in both 32-bit and 64-bit versions).
You can check or install available compilers in MATLAB by using the command: "mex -setup"
For more information, please visit www.mathworks.com.

Stream Data to MATLAB Using the GInsData Shared Library

Load the GInsData C/C++ Shared Library in MATLAB

To load a library in MATLAB, use the following syntax:

loadlibrary('giutility','eGateHighSpeedPort.h','alias','HighSpeedPortDLL');

For detailed information about the loadlibrary syntax, refer to the MATLAB documentation: loadlibrary.

Call GInsData Library Functions from MATLAB

A DLL function can be called from a loaded library for example

calllib('HighSpeedPortDLL','CD_eGateHighSpeedPort_Init',host,timeout,conType,10,hclient,hconnection);

See for detailled info for calllib: https://www.mathworks.com/help/matlab/ref/calllib.html

Use MATLAB Pointer Objects with the Shared Library

Pointer objects are essential when working with shared libraries. Below are examples of how to create and use pointer objects for various data types:

Creating Pointer Objects:

Int32:
Ptr = libpointer('int32Ptr', -1);
Double:
Ptr = libpointer('doublePtr', 0);
CString:
Ptr = libpointer('cstring', '000000000000000000000000000000000000000000000000000000');
CString (Null Pointer):
Ptr = libpointer('cstring');

Retrieving Values from Pointers:

To obtain the value from a pointer, use the following syntax:

Ptr.Value

For detailed information on pointer objects, refer to the MathWorks documentation.

Stream Data to MATLAB Using GInsMex MEX Files

GInsMex MEX File Requirements and Compatibility

You can utilize MEX files to directly access GInsData functionality through the C-API within MATLAB. By writing simple C code, you can create an interface between MATLAB and DLL functions (written in C/C++), allowing you to call these functions directly from MATLAB.

Introduction to MEX Files:

For more information on MEX files, refer to the Introduction to MEX files.

Gantner Instruments provides the following MEX files for accessing DLL functionality:

For 64-bit MATLAB:
GInsMex.mexw64 (generated in MATLAB R2013a 64-bit)
For 32-bit MATLAB:
GInsMex.mexw32

Ensure that the corresponding giutility.dll file is present in your working directory alongside the GInsMex.mexw32 or GInsMex.mexw64 file. Alternatively, it can be placed in the Windows System32 or SysWOW64 directory.

The MEX files are generated using the Microsoft Visual C++ 2010 compiler. To ensure proper functionality, make sure that the Microsoft Visual Studio 2010 runtime libraries are installed on the computer where the MEX files will be used.

GInsMex Work Modes for Data Acquisition and File Import

The MEX file function is called GInsMex and supports various work modes. It wraps key functions of the DLL interface (giutility.dll), making them easily accessible within MATLAB. The work modes include operations such as reading buffer data from a device or post-process buffers (GI.bench) and importing files in the UDBF format.

Usage:

 [ret(, Additional Ret Values) ] = GInsMex(workmode[, Additional Parameters])

To access the online help for GInsMex, simply type GInsMex in your MATLAB Command Window. Many functions are designed to be used in combination with multiple work modes. An example demonstrating usage can be found in HSP_ReadBuffer.m.

The available work modes are:

1 - connect to hardware with specified ip address
2 - set the buffer index to specify which buffer should be used (0-3)
3 - get number of channels of the selected buffer
4 - get channel name
5 - read data (handle this function with care! on high buffer data rates, your Matlab application needs to be able
         to process the received amount of data fast enough. Otherwise a buffer-overrun will occur and connection will be closed! 
         In this case, think about reducing the buffer sample rate or only "post"-proccess the high speed data in Matlab 
         using .DAT files (avoid "online" highspeed data processing in Matlab).
6 - close connection
7 - get samplerate
8 - sleep (ms)
10 - Identify first - Use this Fuction (+ workmode 11) to scan the network for devices...
11 - Identify next - Use this in addition to workmode 10 to scan the network for devices... 
12 - Get Number of available PostProcess buffers. Use this function in combination with workmode 13 
13 - Read PostProcess Buffer information by index 
14 - Initialize connection to a PostProcess Buffer (GI.bench) 
15 - Set Backtime - use this function only in combination with workmode 14. 
101 - Open .DAT file
102 - Read channel data from selected file

MATLAB Data Streaming and Import Examples

Stream Buffered DAQ Data with HSP_ReadBuffer.m

This example illustrates how to use the GInsMex functions to connect to various buffer streams via a simple user interface for parameter configuration (HSP stands for HighSpeedPort Protocol). To execute the example, enter the following command in your MATLAB Command Window:

 >> HSP_ReadBuffer

This example allows you to connect to different data sources:

Network Device: For example, a Q.station, using a specified IP address.
PostProcess Buffers: Local system data buffers, such as those provided by GI.bench, identified by their ID.

To locate devices or PostProcess buffers, you can perform a network scan or a local PostProcess buffer scan, both of which are accessible via buttons in the user interface. The example code is available in the HSP_ReadBuffer.m file.

This example demonstrates how to use the functions to import data into MATLAB. Any subsequent data visualization or processing is the responsibility of the user.

Import Gantner .DAT Measurement Files into MATLAB

With the MEX function, you can import binary Gantner .DAT file data into MATLAB. This function loads single-channel data from a file by channel index into a 1×1 structure. The structure contains two fields, each with Nx1 vectors, where N represents the number of samples in the file:

XData: Relative time values in seconds ([s])
YData: Channel measurement values

To use the MEX function, specify the following work modes:

example_filedata_import.m

%% load file data -> workmode = 101
GInsMex(101, 'backup___0_2018-10-02_05-11-00_000000.dat')
 
%% read channel data of channelindex 5 to "data" -> workmode = 102
[ret, data] = GInsMex(102,5);
 
%% close file -> workmode = 6
GInsMex(6)
 
%% plot channel data
plot(data.XData, data.YData)

To load data from an entire file containing all channels, you can use a MATLAB struct.

example_filedata_import2.m

%% load file data -> workmode = 101
GInsMex(101, 'backup___0_2018-10-02_05-11-00_000000.dat')
 
%% read number of channels -> workmode = 3
[ret, numChannels] = GInsMex(3);
 
%% create struct for all channels
channelstruct = struct();
 
for i=1:numChannels
   %read channel name -> workmode = 4
   [ret, strChannel] = GInsMex(4, i-1);
 
   %read channel data -> workmode = 102
   [ret, data] = GInsMex(102,i-1);
 
   %write to struct -> remove invalid characters from string!
   channelstruct.(regexprep(strChannel, {'#' '/' ' '}, '_')) = data;
end
 
%% close file -> workmode = 6
GInsMex(6)
 
%% plot channel data of a single channel
plot(channelstruct.sine3.XData,channelstruct.sine3.YData)

Download GInsMex Files and MATLAB Examples

64-bit: Download the file + examples here: mexw64.zip
32-bit: Download the file + examples here: mexw32.zip

FAQs

How can I stream Gantner DAQ measurement data to MATLAB?

Gantner measurement data can be read into MATLAB from a Q.series controller such as Q.station or Q.monixx, or from a GI.bench PostProcess Buffer. Use either the GInsData C-API directly or Gantner's GInsMex MEX interface.

Can MATLAB read data directly from a Q.station?

Yes. MATLAB can connect to a Q.station over Ethernet and read buffered measurement data using the GInsData interface or the GInsMex MEX file. The supplied HSP_ReadBuffer.m example demonstrates connecting to a network device or GI.bench PostProcess Buffer.

Do I need a compiler to use GInsData with MATLAB?

For the legacy loadlibrary/calllib approach, MATLAB requires a supported C compiler. The article's tested workflow applies through MATLAB R2021b. MathWorks recommends its newer C/C++ library interface beginning with R2022a. A compatible precompiled GInsMex file can avoid compiling the interface yourself.

Can MATLAB import Gantner .DAT measurement files?

Yes. GInsMex work mode 101 opens a .DAT file and work mode 102 reads channel data. Returned data includes XData for relative time and YData for measurement values, allowing engineers to analyze or plot recorded measurements directly in MATLAB.

Why can a buffer overrun occur when streaming data to MATLAB?

A buffer overrun can occur when MATLAB cannot process received samples as quickly as the measurement buffer supplies them. Gantner recommends reducing the buffer sample rate or performing high-speed analysis offline from recorded .DAT files when MATLAB cannot sustain the incoming data rate.