Sui Blockchain Launches gRPC Streaming for Real-Time Low-Latency Data Indexing

TLDR:

• gRPC streaming delivers finalized checkpoints instantly, eliminating polling delays and artificial latency intervals. 
• Hybrid architecture pairs streaming with polling fallbacks to ensure historical data access and failure recovery. 
• Custom Indexing Framework enables incremental streaming adoption without modifying existing checkpoint processing logic. 
• General-Purpose Indexer uses streaming as primary path while maintaining polling sources as safety mechanisms.

Sui has implemented gRPC streaming as a primary data source for its indexing infrastructure, enabling real-time checkpoint ingestion with minimal latency.

The blockchain platform combines streaming capabilities with traditional polling methods to ensure data accuracy and system resilience.

This hybrid approach allows developers to access finalized checkpoints immediately while maintaining backward compatibility with existing custom indexers.

The Custom Indexing Framework supports this streaming-first architecture without requiring modifications to checkpoint processing logic.

Streaming Eliminates Polling Delays in Data Access

The new gRPC streaming capability fundamentally changes how indexers receive blockchain data on Sui. Full nodes now push checkpoint data directly to indexers as soon as finalization occurs.

This eliminates the repeated polling cycles that traditionally introduce delays between checkpoint creation and downstream processing.

According to the announcement, the system delivers “real-time checkpoints as soon as they’re finalized” with “faster data, resilient pipelines, less infra work on Sui.”

The streaming mechanism operates through a straightforward configuration where developers add a streaming-url argument pointing to a full node endpoint. The system then receives checkpoints as events rather than fetching them at predetermined intervals.

This event-driven model proves particularly valuable for latency-sensitive applications including monitoring systems and real-time analytics platforms.

Sui pairs streaming with mandatory polling-based fallback sources to address inherent limitations. Streaming connections only deliver data from the moment of establishment forward.

The General-Purpose Indexer demonstrates this hybrid model in production environments. It uses streaming as the primary ingestion path while maintaining polling sources as safety mechanisms.

This configuration keeps indexed data current while ensuring clean restarts and seamless recovery from failures.

Framework Design Supports Incremental Adoption

The Custom Indexing Framework separates checkpoint processing from data ingestion entirely. Indexers consume and transform checkpoints without coupling to specific data sources.

This abstraction allows teams to modify ingestion strategies as requirements evolve without rewriting processing logic.

The documentation notes that with gRPC streaming, “there is no need to poll, no guesswork around timing, and no artificial delay introduced by fetch intervals.”

Developers can implement streaming gradually based on workload characteristics. Applications prioritizing data freshness benefit from immediate streaming adoption.

Systems handling offline processing or simpler workflows can continue using polling-only configurations. The framework accommodates both approaches within the same processing model. Existing custom indexers built on the official framework require minimal changes to adopt streaming.

Adding gRPC capability involves including a streaming-url parameter alongside the existing remote-store-url configuration.

The checkpoint processing logic remains unchanged throughout this transition. The framework automatically manages source switching during operation, preventing common failure modes of systems that either lose data or lag behind network state.