BRIDGE

SIGKDD 2026

BRIDGE: Block-Wise Speculative Coordination for Cloud-Edge Retrieval-Augmented Generation

BRIDGE jointly uses public cloud knowledge and private edge knowledge for retrieval-augmented generation, while avoiding sensitive data transfer and reducing coordination stalls with block-wise speculative verification.

Yuting Li* · Shaoyuan Huang* · Xiangqi Liu · Yunfeng Zhao · Xiaofei Wang

Tianjin University • * Equal contribution • Corresponding author: Xiaofei Wang

Paper Code Cite
79.1% Max end-to-end latency reduction
46.0% Average P99 latency reduction
65.7% Rollback reduction on Qwen
34.8% Best personalization gain over cloud-only RAG
A grounded cloud-edge recommendation scene that needs both public cloud knowledge and private edge context

Overview

Why cloud-edge RAG needs a new coordination strategy

Real deployments often split knowledge across two tiers: cloud-side corpora carry broad and fresh public information, while edge-side storage holds private user context. Existing approaches either sacrifice personalization, expose sensitive data, or pay heavy synchronization and rollback cost during collaborative decoding.

BRIDGE addresses this by letting both sides retrieve and draft in parallel, then coordinating them through block-wise transmission and verification. The result is stronger relevance and personalization, lower latency, and better service stability under varying networks.

Challenge 1

Public knowledge and private user context live in different places, but practical RAG tasks need both.

Challenge 2

Token-level aggregation causes frequent cross-side synchronization and RTT-sensitive stalls.

Challenge 3

Speculative rejection wastes KV states and compute through rollback-heavy recomputation.

01

Block-wise speculative RAG

Jointly uses cloud and edge retrieval without uploading private edge data or rebuilding KV states from raw text.

02

Block-wise transmission and verification

Amortizes communication overhead and commits the longest acceptable prefix in one shot.

03

Adaptive block length

Balances communication savings and verification cost under changing runtime and network conditions.

Actual Use Case

Public facts and private profile signals have to meet in one answer

In our real serving scenario, the cloud side contributes fresh public facts about the queried movie, while the edge side contributes the user's background, profession, hobbies, and writing style. BRIDGE fuses both sides into one explanation that is more personal than cloud-only RAG and more informed than edge-only retrieval.

Actual BRIDGE use case combining cloud-side public corpora and on-edge private corpora
A concrete movie recommendation example showing how cloud-side public corpora and on-edge private corpora jointly shape the final response.

Method

From token-wise waiting to block-wise coordination

BRIDGE framework

Cloud and edge decoders retrieve from their own corpora, draft tokens independently, buffer them into blocks, and send the blocks to an event-driven coordinator. The coordinator verifies both drafts jointly against the aggregated target distribution and broadcasts the committed prefix back to both sides.

  • Parallel cloud-edge drafting keeps both sides productive.
  • Block-wise verification improves acceptance efficiency and cuts rollback waste.
  • Adaptive block length keeps throughput high under latency fluctuation.
Overview of the BRIDGE framework
Token-wise versus block-wise speculative coordination pipeline
Block-wise transmission reduces synchronization frequency and reaches longer committed outputs under the same wall-clock budget.

Transmission

BRIDGE batches draft tokens and auxiliary statistics into short blocks, reducing RTT-dominated communication overhead.

Verification

Instead of stopping at the first rejected token, BRIDGE verifies the whole draft block and commits the longest acceptable prefix.

Adaptation

Runtime profiling chooses a safe and efficient block length, preserving low latency while improving throughput.

Results

Higher quality, lower latency, and fewer wasted steps

Best practical generation quality

Across Co-Wiki and Movie-Wiki, BRIDGE achieves the strongest Q-A relevance and personalization among practical baselines in both Qwen and LLaMA series.

Stable under network variance

On Movie-Wiki, BRIDGE reduces average latency by 50.3% versus VCR and 37.0% versus DRAGON on Qwen, while lowering P99 latency by up to 79.1%.

Reliable tail behavior

Beyond mean latency gains, BRIDGE keeps tail latency under control, making collaborative RAG more practical under fluctuating network conditions.

Quality comparison table for BRIDGE against baselines
BRIDGE delivers the strongest practical quality across both datasets and model families.
Latency comparison under different network conditions
Average end-to-end latency remains lower than distributed baselines as injected network latency increases.
Rollback and communication comparison between BRIDGE and the token-wise ablation
Block-wise speculation reduces rollback frequency and communication cost compared with the token-wise ablation.

Paper

Abstract and materials

Retrieval-augmented generation improves factuality by conditioning LLMs on retrieved evidence, but real deployments often split knowledge across tiers: cloud-based RAG can exploit large public corpora, whereas edge-based RAG is the natural place to access private, user-specific stores. BRIDGE asks how to jointly use both without transferring sensitive edge data or paying prohibitive coordination cost.

The framework introduces block-wise speculative coordination for inter-database distributed generation, including block-wise transmission, block-wise verification, and an adaptive block-length strategy. Across public-private RAG benchmarks, model families, and network regimes, BRIDGE improves relevance and personalization while reducing end-to-end latency by up to 79.1%.

PDF Full Paper Read the PDF version used for this page. CODE Code Official implementation repository for BRIDGE. MAIL Contact xiaofeiwang@tju.edu.cn

Citation

BibTeX

@inproceedings{li2026bridge,
  title     = {BRIDGE: Block-Wise Speculative Coordination for Cloud-Edge Retrieval-Augmented Generation},
  author    = {Li, Yuting and Huang, Shaoyuan and Liu, Xiangqi and Zhao, Yunfeng and Wang, Xiaofei},
  booktitle = {Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year      = {2026}
}