The Topic, the Message and the Partition
Traditional messaging patterns: message queue and publish – subscribe, have some limitations as a result of their design.
In the previous post – Apache Kafka Ideas – Part 1, a couple of messaging use cases were introduced. In order to define those cases with Kafka, it is important to understand its ideas. At the very heart of Kafka are topics and partitions. This post explains basic concepts behind them.
Continue reading “Apache Kafka Ideas – Part 2”
What Apache Kafka is?
Apache Kafka can be thought of as a message broker. It has the following characteristics:
- allows sending messages between two parties
- allows one-to-one (peer to peer, queue) or one-to-many (broadcast, topic) message delivery
- persists messages
What ideas are behind Kafka and how does it differ from a classical broker? In this series of posts you’ll find out how does Apache Kafka work and be able to run and use Kafka cluster.
Continue reading “Apache Kafka Ideas – Part 1”
This is the last post in the Datadog series. Previous two: PostgreSQL Metrics in Datadog and Sending Metrics to Datadog with Java showed a need to report data and how to do it. This one will focus on how those metrics can be observed and used.
Continue reading “Viewing Metrics in Datadog”
In my previous post I drew an idea of sending PostgreSQL metrics to Datadog using Java code. This post will reveal implementation details of the Send action described previously.
How does Datadog collect metrics?
There are two basic ways of collecting and sending data:
- Use Datadog agent
- Collect and send manually
Continue reading “Sending Metrics to Datadog with Java”
I currently work on a project were AWS RDS PostgreSQL is used as a data storage. Since Postgres is using a mechanism called Multiversion Concurrency Control – MVCC, an UPDATE or DELETE command does not remove old versions of a row immediately. These are left on a disk, waiting to be collected and cleaned by a vacuum process. Vacuum can be automated and autovacuum serves that purpose.
Once autovacuum is configured, how do I know it works as expected? Is it triggered when I expect it to be?
Continue reading “PostgreSQL Metrics in Datadog”