A Kafka topic is a category or a common name used to store and publish a particular stream of data in Apache Kafka. It is similar to a folder in a filesystem, or a table in a relational database. A topic is comprised of a log of events, which are easy to understand because they are simple data structures with well-known semantics. The events are organized and durably stored in topics, and each topic has a name that is unique across the entire Kafka cluster.
Producers write data to topics, and consumers read data from topics. Kafka topics are multi-subscriber, meaning that a topic can have zero, one, or multiple consumers subscribing to that topic and the data written to it. Offsets are assigned to each message in a partition to keep track of the messages in the different partitions of a topic.
Kafka topics can be created either automatically or manually. It is best practice to manually create all input/output topics before starting an application, rather than using auto topic. For auto topic creation, it’s good practice to check num.partitions for the default number of partitions and default.replication.factor for the default number of replicas of the created topic. To create topics manually, run kafka-topics.sh and insert topic name, replication factor, and any other relevant attributes.
In summary, a Kafka topic is a virtual group or log that holds messages and events in a logical order, allowing users to send and receive data in a scalable and fault-tolerant way.
