Skip to content
STIRUP

TECHNOLOGY INTELLIGENCE

Understand what's next.

High-Data-Rate Acquisition and Real-Time AI in Scientific Instruments

A high-dynamic-range electron microscopy camera generates over 500 Gbps of data, requiring advanced processing techniques to enable real-time AI during acquisition.

Share
AI-generated conceptual illustration accompanying: High-Data-Rate Acquisition and Real-Time AI in Scientific Instruments
AI-generated editorial illustration of the article topic; not a photograph, product screenshot or measured result.

High-Data-Rate Acquisition and Processing

High-data-rate acquisition and processing are critical for enabling real-time AI in scientific instruments. A high-dynamic-range electron microscopy camera developed at Lawrence Berkeley National Laboratory generates over 500 gigabits of data per second (Gbps), necessitating advanced processing techniques to make the measurements useful during acquisition. This data stream must be received and processed in real time, requiring a robust pipeline that allows for inspection of the data during the measurement. Through this pipeline, the team enables real-time adjustments to acquisition parameters and regions of interest, all while handling the high bandwidth data incoming at more than 500 Gbps. This advancement simplifies the experimental setup and microscope optics, but the challenge of managing such a high data rate remains significant. The ability to process and analyze data as it is acquired is a key enabler for real-time AI applications in scientific instrumentation.

GPU-Native Data Acquisition with DAQIRI

NVIDIA DAQIRI enables real-time AI processing for scientific instruments by receiving detector packets directly into GPU memory, allowing for real-time frame assembly, correction, and energy-loss spectra reduction during experiments, as noted in NVIDIA Developer. This capability is particularly valuable for existing Ethernet-capable instruments, as it opens opportunities to add processing and feedback capabilities, as described in prior discussions on real-time AI for high-speed data acquisition, according to NVIDIA Developer. Furthermore, the DAQIRI ibverbs engine nearly doubles throughput for 64-byte and 256-byte packets compared with DPDK, while maintaining similar throughput for 8 KB packets, as stated in NVIDIA Developer.

Optimizing Network Performance for Real-Time AI

Optimizing network performance is critical for enabling real-time AI in scientific instruments, where low-latency data processing is essential. By reordering data as it arrives with NIC hardware, systems can avoid the latency and memory overhead associated with protocol processing or receive-side buffering. The ibverbs engine further reduces per-packet CPU overhead, delivering approximately twice the per-core throughput of the previous default DPDK backend at small packet sizes (up to 1024 bytes) and extreme data rates (greater than 200 Gbps), based on benchmark tests performed on the NVIDIA DGX Spark, NVIDIA IGX Thor, and NVIDIA RTX PRO server platforms. These results are from single-queue benchmarks on the NVIDIA IGX Thor. For latency-sensitive applications, direct polling can reduce feedback latency, allowing intelligent instruments to adjust acquisition or control settings in real time based on GPU analysis.

Sources and reporting notes

Based on published source material. No independent hands-on testing is claimed.

Read more