Yes

TBH I don’t know how much time I would invest in learning it. I think it’s an amazing piece of engineering, but Google is pretty much losing ground everywhere. I noticed people used to speak of the “three major cloud providers AWS, Azure and GCP”, these days it’s only AWS and Azure. And TensorFlow is losing to PyTorch. I don’t know anyone using TFX, people use MLflow, Metaflow, Great Expectations/pandera, … Kubernetes has the reputation of being too complex, people would rather use ECS. Same goes for Kubeflow, people use Airflow or Step Functions. And people don’t care about having unified batch/stream processing, they would rather exchange data on S3 or use Kinesis. Spark is dead anyway, people would rather use dask…

Okay, that I would agree with.

Why would you say it’s outdated? I don’t see any other technology that has a similar scope as Beam.

Fluent Python + Effective Python First is more complete and advanced, second is great if you are beginner/intermediate and want to level up really fast.

I am not aware of a particular book that covers this, but I can say that “Building Machine Learning Pipelines” by Hapke/Nelson has some sections on Apache Beam (chapters 2 and 11).

It’s basically a book on the Google Machine Learning ecosystem (TFX, Kubeflow), which is based on Beam.

I started learning Rust a year ago, but then stopped thinking: This looks amazing, but I will never have any use case for it…

I guess this just changed. :)