Introduction
Setting up Apache Kafka on your local machine is an essential step for unlocking its powerful real-time data streaming capabilities. In this blog, we’ll walk you through the entire process of installing Kafka on Ubuntu, configuring it correctly, and running it in a Jupyter Notebook environment. From installing Java (a Kafka prerequisite) to starting and stopping Kafka and Zookeeper servers, this guide will ensure that your Kafka installation is up and running smoothly. By following this step-by-step tutorial, you’ll be ready to test your Kafka setup and start leveraging its potential for real-time data processing.
Before installing Kafka, ensure you have the required Java Development Kit (JDK) installed on your system. Kafka runs on the Java platform, so it requires Java 8 or later.
Here's a step-by-step guide to installing Kafka and Jupyter Notebook in Ubuntu, and running Kafka in a Jupyter Notebook:
Step 1
First update packages with following commands
Keeping Your System Up-to-Date
To ensure your system has the latest package information, use the command:
sudo apt update 
Step 2
Keep Upgrading Your System Packages
To upgrade all installed packages to their latest versions, you can use the following command:
sudo apt upgrade -y
Step 3
Install Java (Kafka Dependency)
Kafka requires Java, so install OpenJDK:
sudo apt install openjdk-11-jdk -yTo verify the installation:
java -version
Step 4
Install Kafka
a. Download Kafka
Visit Apache Kafka Downloads to get the latest version or run:
b. Extract Kafka
tar -xvzf kafka_2.12-3.8.0.tgz
c. Move Kafka to a desired directory
sudo mv kafka_2.12-3.8.0 /usr/local/kafka
d. Set up Kafka environment variables
Open your .bashrc file and Add these lines at the end:
nano ~/.bashrc
Now We have Successfully Setup kafka
Step 5
Starting and Stopping Kafka Server
Kafka depends on Zookeeper to manage its brokers. Therefore, you need to start Zookeeper first, followed by Kafka.
Start Zookeeper
Run the following command in the terminal:
zookeeper-server-start.sh $KAFKA_HOME/config//zookeeper.properties
After starting zookeeper looks like this.
You should see logs indicating that Zookeeper is running. Leave this terminal window open.
Start Kafka Broker
Open a new terminal window and run this command to start kafka broker.
kafka-server-start.sh $KAFKA_HOME/config/server.properties
After starting kafka looks like it
Stopping Zookeeper and Kafka
To stop Kafka, press ctrl + c in the terminal running Kafka or use
/usr/local/kafka/bin/kafka-server-stop.sh
Similarly, to stop Zookeeper, press Ctrl+C in the terminal running Zookeeper or use:
/usr/local/kafka/bin/zookeeper-server-stop.sh
Comentários