Kafka interviews test whether you understand how data moves through a log: how topics are split into partitions, what ordering you can rely on, how consumers share work, and what happens when something fails. Scenario questions about lag, duplicates, and data loss are common, because those are the problems people hit in production.
Below are 22 questions with answers. Details such as defaults vary between Kafka versions, so mention that you'd check the version in use.
Core concepts
1. What is Kafka, and what is it used for?
Kafka is a distributed, replicated commit log. Producers append records to topics, and consumers read them at their own pace. It's used for event streaming, log and metrics pipelines, decoupling services, feeding data into analytics systems, and stream processing.
2. What are topics, partitions, and offsets?
A topic is a named stream of records. Each topic is split into partitions, and each partition is an ordered, append-only log. Every record in a partition has an offset, a sequential number that identifies its position. Offsets are unique only within a partition.
3. How do partitions affect ordering?
Kafka guarantees order only within a partition. Records with the same key go to the same partition by default, so they stay in order relative to each other. Across partitions there's no ordering. If you need all events for one customer in order, use the customer ID as the key.
4. What is a broker, and what does the controller do?
A broker is a Kafka server that stores partitions and serves clients. A cluster has several brokers. One acts as the controller, which manages cluster metadata and partition leadership. Newer Kafka versions use KRaft, a built-in consensus mechanism, instead of ZooKeeper for this metadata.
5. How does replication work?
Each partition has a replication factor. One replica is the leader, which handles reads and writes, and the others are followers that copy it. The in-sync replicas (ISR) are those caught up with the leader. If the leader fails, a new leader is chosen from the ISR.
Producers
6. What do the acks settings mean?
| acks | Meaning | Tradeoff |
|---|---|---|
| 0 | Producer doesn't wait for any acknowledgment | Fastest, can lose data |
| 1 | Leader acknowledges after writing | Can lose data if the leader fails before replicating |
| all | Leader waits for all in-sync replicas | Strongest durability, higher latency |
For durable writes, use acks=all with min.insync.replicas set to at least 2 and a replication factor of 3.
7. How do you avoid duplicate messages from a producer?
Enable the idempotent producer (enable.idempotence=true), which lets the broker discard duplicates caused by retries within a partition. For writes that must span several partitions or topics atomically, use transactions.
8. How does a producer choose a partition?
If you give a key, the default partitioner hashes it to pick the partition. With no key, the producer spreads records across partitions (recent versions use a sticky approach to fill batches efficiently). You can also write a custom partitioner.
9. What affects producer throughput?
Batching (batch.size, linger.ms), compression (such as lz4 or zstd), the number of partitions, and acks. Larger batches and compression usually raise throughput at the cost of a little latency.
Consumers
10. What is a consumer group?
A set of consumers that share the work of reading a topic. Each partition is assigned to one consumer in the group at a time. So the number of useful consumers in a group is limited by the number of partitions, and extra consumers sit idle.
11. What happens during a rebalance?
When a consumer joins or leaves, or partitions change, the group reassigns partitions. During a classic rebalance, consumption pauses while this happens. Cooperative rebalancing and static membership reduce the disruption. Frequent rebalances often come from consumers that take too long to process (max.poll.interval.ms) or miss heartbeats.
12. How are offsets committed?
Consumers commit the offset of the next record to read, stored in an internal topic. Auto-commit is simple but can commit before processing finishes. Manual commits after processing give you control over what "done" means.
13. What do at-most-once, at-least-once, and exactly-once mean?
- At-most-once: commit the offset before processing. A crash can lose records.
- At-least-once: process, then commit. A crash can cause reprocessing, so duplicates are possible.
- Exactly-once: the effect happens once. In Kafka this is possible inside Kafka using idempotent producers, transactions, and read_committed consumers. When writing to an external system, you usually need idempotent writes or deduplication by a key.
A good answer says that "exactly once" depends on where the side effects are.
14. How would you handle a poison message that always fails?
Catch the error, retry a limited number of times, then send the record to a dead-letter topic with details, commit the offset, and keep going. Alert on the dead-letter topic so someone investigates.
Operations and scenarios
15. Consumer lag is growing. How do you diagnose it?
- Check which partitions have lag. One hot partition points to a skewed key. Lag everywhere points to slow processing or too few consumers.
- Check processing time per record and downstream calls such as a database.
- Check for frequent rebalances.
- Check whether the producer rate rose.
- Fix with more consumers (up to the partition count), faster processing, batch handling, or more partitions.
16. When should you increase the partition count?
When you need more parallelism than current consumers can provide, or throughput per partition is the limit. Be careful: adding partitions changes the key-to-partition mapping for new records, so ordering for a key may break across the change. Plan the partition count up front when ordering matters.
17. How is data retained, and what is log compaction?
By default, Kafka deletes old segments by time or size (retention.ms, retention.bytes). With log compaction, Kafka keeps at least the latest value for each key, which suits a changelog or a table-like topic. A null value (a tombstone) marks a key for deletion.
18. What happens if a broker fails?
Partitions led by that broker elect new leaders from the ISR. Clients refresh metadata and continue. If there's no in-sync replica available, the partition is unavailable unless you allow unclean leader election, which risks data loss.
19. How do you secure a Kafka cluster?
TLS for encryption in transit, SASL (such as SCRAM or OAuth) or mutual TLS for authentication, ACLs for authorization on topics and groups, and network controls. Encrypt sensitive fields at the application level if needed.
20. How do you monitor Kafka?
Key signals are consumer lag, under-replicated partitions, offline partitions, request latency, disk usage, and ISR shrink and expand rates, collected through JMX metrics and tools such as Prometheus and Grafana.
Design questions
21. When would you choose Kafka over a message queue such as RabbitMQ or SQS?
Choose Kafka when you need high throughput, replay of past events, several independent consumers of the same stream, or stream processing. Choose a traditional queue for per-message routing, simple task queues, or when you don't want to manage partitions.
22. Design an order events pipeline.
"Orders go to an orders topic keyed by order ID so events for an order stay in order. I'd use three or more brokers, a replication factor of 3, acks=all, and idempotent producers. Consumers for billing, inventory, and analytics each use their own group. Billing writes are idempotent by order ID, failed records go to a dead-letter topic, and I'd monitor lag per group. I'd pick the partition count from peak throughput and expected consumer count, with room to grow."
How to prepare
If you can, run a local Kafka, write a producer and a consumer group, stop a consumer, and watch the rebalance. Break something on purpose, such as a slow consumer, and see the lag. Prepare one real story about a data pipeline problem you solved. For related topics, see our guides to Databricks and system design interviews.
Practice explaining the delivery guarantees out loud, because that's where answers get vague. A mock interview and a transcript will show you where. Follow the employer's rules on AI tools in any live interview.
Practice Kafka interview answers with YesToTheOffer
Run a mock technical interview, explain delivery guarantees out loud, and review your transcript afterward. 30 free minutes, no subscription.
