GCP for Apache Kafka Users: Stream Ingestion and Processing (Cloud Next '19)

In private and public clouds, stream analytics commonly means stateless processing systems organized around Apache Kafka or a similar distributed log service. GCP took a somewhat different tack, with Cloud Pub/Sub, Dataflow, and BigQuery, distributing the responsibility for processing among ingestion, processing and database technologies. We compare the two approaches to data integration and show how Dataflow allows you to join and transform and deliver data streams among on-prem and cloud Kafka clusters, Cloud Pub/Sub topics and a variety of databases. The session will have a mix of architectural discussions and practical code reviews of Dataflow-based pipelines. Trusted Cloud Access Through Chrome Browser → https://bit.ly/2K9lfQr Get Chrome Browser for Enterprise → https://bit.ly/2TWkABa Watch more: Next '19 Data Analytics Sessions here → https://bit.ly/Next19DataAnalytics Next ‘19 All Sessions playlist → https://bit.ly/Next19AllSessions Subscribe to the G Suite Channel → https://bit.ly/G-Suite1 Speaker(s): Ricardo Ferreira, Karthi Thyagarajan Session ID: DA305 product:BigQuery; fullname:David Pessis; http://bit.ly/2P8aTPn G Suite April 12, 2019 at 06:02PM