Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
A New Approach to Logging as a Stack: Fluent Bit + PostgreSQL (FPS) - Jonathan Gonzalez, EDB
Logging at scale is a very interesting challenge, and having the right open source stack is mandatory. There are many ways to solve the log collection, processing and aggregation problem, but when it comes to perform data analysis once the data has been centralized, it's really easy to face latencies and get general problems when indexing our data that don't comes with a fixed structure. In this session, we will share our experience of implementing a new stack that we called FPS: Fluent Bit + PostgreSQL. The open source database PostgreSQL offers a very interesting storage engine for non-columnar structured documents: JSONB (binary JSON) which have shown to be a very efficient and scalable implementation to store trillions of record logs and provide fast results when performing queries.