Stateful processing with Apache Beam

Beam lets you process unbounded, out-of-order, global-scale data with portable high-level pipelines. Stateful processing is a new feature of the Beam model that expands the capabilities of Beam, unlocking new use cases and new efficiencies. In this post, I will guide you through stateful processing in Beam: how it works, how it fits in with the other features of the Beam model, what you might use

[i101330]

これのanomaly detectionで透過的にstatefulなDoFnを記述できるのはかなりワクワクする。

[kimutansk]

Beamを使うかどうかはさておき、こういう視点や区切りによって管理されるという考え方は読んでいて面白いですね。