Another paper accepted to ICDE 2024

ZeroTune: Learned Zero-Shot Cost Models for Parallelism Tuning in Stream Processing

2024/03/14

Authors: Pratyush Agnihotri, Boris Koldehofe, Paul Stiegele, Roman Heinrich, Carsten Binnig, Manisha Luthra

We are happy to announce that our paper ZeroTune was accepted to the International Conference of Data Engineering (ICDE) 2024, Utrecht, Netherlands.

We propose ZeroTune, a cost model for setting initial parallelism degrees in parallel and distributed stream processing. The cost model uses the proposed data-efficient techniques for zero-shot learning providing very accurate cost predictions for unseen parallel query plans, while avoiding the extensive training required by traditional models. When applied to systems like Apache Flink, ZeroTune significantly improves performance, yielding an average speed-up of 5x over current methods while reducing the training effort by 4x for generalization.

It can be downloaded here.