Setup Apache Kafka Environment on Windows
Wrote a guide on setting up Kafka in a Windows environment
In this article, I will be walking you through the process of installing Apache Kafka successfully on Windows. Let's dive into it.
Why use Apache Kafka?
Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. You can use Kafka for real-time data streaming, big data collection, and real-time data analysis. It can be used with in-memory microservices ensuring durability. It will allow you to feed events to complex event streaming systems. It is one of the most commonly used fault-tolerant messaging systems.
Prerequisites
- Download and Install the Java Runtime Environment (JRE8). This can be done in two different ways which are by:
- Download from the Oracle Java Official Downloads Page here
- Install using Chocolatey by pasting this script on your windows command line.
After installing, confirm your Java installation
- Download and Install Apache Zookeeper
Apache Zookeeper helps keeps track of the status of the Kafka cluster nodes and it also keeps track of Kafka topics, partitions.
Get Started with Apache Kafka
For the installation process, please follow the steps given below:
Step 1: Visit Apache Kafka Official Download Page
- Download the latest stable release of Apache Kafka Binaries - https://kafka.apache.org/downloads
Step 2: Extract Apache Folder
- Visit your downloads folder
- Extract the downloaded file and move it to any folder of your choice. For this tutorial, I recommend moving to the
C:/
folder.
Step 3: Create directories for storing Kafka/zookeeper snapshots and logs
After moving to the C:/
folder ;
- Create a
data
folder in the Kafka directory - Create
Kafka
andzookeeper
directories inside thedata
folder
Step 4: Update configuration file for Zookeeper and Kafka Logs
- In
C:/kafka
, open the config folder in a text editor and update thezookeeper.properties
file. Change thedataDir
variable to the path where your zookeeper snapshots will be stored as shown in the image below.
- In the same config folder, update the
server.properties
file. Update thelog.dirs
variable to the path where your Kafka logs will be stored.
Step 5: Start Apache Zookeeper and Kafka
- Start Apache zookeeper,
Open windows command prompt, change your directory to
c:/
and run the following scripts
cd kafka/bin/windows
zookeeper-server-start.bat ../../config/zookeeper.properties
- Start Apache Kafka, open up a new command prompt
Run the following command from C:\kafka\bin\windows
directory
Apache Kafka should start running successfully on your machine 🎉🎉🎉🎉
Conclusion
Apache Kafka is now successfully installed on your machine, you can start making use of Kafka for event and data streaming. Learn more about Kafka Use cases and applications.
If you have any questions, don't hesitate to reach out me on Twitter: @adeileri