blog image

Emitting Apache Druid Metrics To Kafka

Apache Druid captures detailed metrics about how it is operating and performing, including query performance, memory usage, CPU utilisation, ingestion speed and more. We may need to capture and monitor these metrics in order to investigate problems or identify proactive optimisations which can improve the performance of the cluster. In this article we explain how

Read More
blog image

What Is Kafka Streams?

Kafka Streams is the component of the Kafka ecosystem which can be used for stream processing. This involves taking messages from Kafka, usually as they are produced in real time, and processing or responding to them in some way. Common examples of this include: Calculating analytics over event streams e.g. tracking the average order value

Read More
blog image

Streaming Technology – Sourcing Events

In order to move towards Event Streaming and Event Based Architecture, the first thing we need to do is source data in real time and publish it onto an event stream. Even this first step can be difficult, because many applications do not “publish” events as they are entered by users. Instead, these applications interact

Read More
blog image

Free Guide – An Overview Of Streaming Data Technology

Many companies are looking to use their data more effectively in order to improve their customer experience and the efficiency of their business. Speed is an important part of this. The earlier you can respond to incoming data, the more opportunities you have to improve the customer experience. Recognising this, businesses are implementing Streaming Data

Read More
blog image

Databricks Structured Streaming Example

Spark 2 introduced the concept of structured streaming, giving users the ability to process streams of unbounded data using higher level abstractions. This is an extremely powerful capability which allows data engineers to do streaming transformations and analytics over data as it is ingested, and potentially join and integrate this with batch data at rest.

Read More