The following blog will make you understand about the second approach of sending data on Kafka i.e. Synchronus Approach. You will also see the how this approach is implemented through java programming Language.
Synchronous Approach
In this approach we send a message to kafka broker and wait till we don’t get response from kafka broker. When your message gets successfully delivered to kafka then in response you get an RecordMetaData object and but if any failure occurs then you will get an Exception. The more focus is put on the exception rather than success. This approach can be handy when you don’t afford to lose any messages. It is interesting to note that most of time you are doing nothing on getting success but in case of exception you might be checking logs for error analysis. While using this approach one should also have the knowledge about the limitation of this approach. The main limitation associated with the approach is it slow down the whole process because for every message you have to wait for the response send by the kafka broker.
Implementation using Java
1. Import Necessary java Packages;
2. Create variables which will define the topicName, key and value; It is already known to you that kafka accepts messages in the form of key and value pair. The key and value pair is nothing but it is your message which you want to send to kafka.
3. Create Properties class object and put some configurations into it.
a) bootstrap.servers is the list of kafka brokers addresses.
b) StringSerializer is the class which is used to serialize your key and value. It is known to you that kafka accepts the data in the form of array of bytes. So the process of converting your message into bytes is known as serialization.
4) Create KafkaProducer object and constructor of KafkaProducer takes the properties class object as an argument.
5) Create ProducerRecord Object and constructor of ProducerRecord takes three values as an argument. Your topicName, key and your value.
6) Now use send method of Producer and pass the ProducerRecord object as an argument. It is important to note that the send method will return you the future object and once you apply get method on that object then you will get the RecordMetaData Object. With the help of this object you can get the information like the offset number or partition number in which your message gets stored.
7) In case of failure you will get an exception and that’s the reason try-catch-finally block is used and after catching the exception you can easily trace down the failure.
8) Lastly, run your java program and in order to test your program just run the kafka console consumer on your terminal and you will see that the message you have written in your program will be on that kafka console consumer.
If you have any confusion then pls refer the whole java program as shown below.
Hope Now you get an idea to send data to kafka through Synchronous approach. There is one more approach which is used to send data to kafka. I will be cover it along with its differences in upcoming parts of blog. Till then keep reading Kafka.
