2.2.4. Kafka Connector¶
Currently, only the TO direction is supported.
2.2.4.1. Usage¶
To use the Kafka Connector, create a link for the connector and a job that uses the link.
2.2.4.1.1. Link Configuration¶
Inputs associated with the link configuration include:
| Input | Type | Description | Example |
|---|---|---|---|
| Broker list | String | Comma separated list of kafka brokers. Required. | example.com:10000,example.com:11000 |
| Zookeeper connection | String | Comma separated list of zookeeper servers in your quorum. Required. | /etc/conf/hadoop |
2.2.4.1.2. TO Job Configuration¶
Inputs associated with the Job configuration for the FROM direction include:
| Input | Type | Description | Example |
|---|---|---|---|
| topic | String | The Kafka topic to transfer to. Required. | my topic |
2.2.4.2. Loader¶
During the loading phase, Kafka is written to directly from each loader. The order in which data is loaded into Kafka is not guaranteed.