Allow External Connections on Your VPS

First, we need to enable external connections on our VPS. I’ve already covered that in a separate guide, which you can refer to here.

Setting Up Your Server

1. Login to your Linux Server using SSH

2. Install Java

    sudo apt install default-jdk
   

3. Download Kafka

    wget https://archive.apache.org/dist/kafka/3.2.0/kafka_2.12-3.2.0.tgz
    tar -xzf kafka_2.12-3.2.0.tgz
    sudo mv kafka_2.12-3.4.0 /usr/local/kafka
   

4. Create Sytem file units

    nano /etc/systemd/system/zookeeper.service
   

    [Unit]
    Description=Apache Zookeeper server
    Documentation=http://zookeeper.apache.org
    Requires=network.target remote-fs.target
    After=network.target remote-fs.target
   
    [Service]
    Type=simple
    ExecStart=/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties
    ExecStop=/usr/local/kafka/bin/zookeeper-server-stop.sh
    Restart=on-abnormal
   
    [Install]
    WantedBy=multi-user.target
   

    nano /etc/systemd/system/kafka.service
   

    [Unit]
    Description=Apache Kafka Server
    Documentation=http://kafka.apache.org/documentation.html
    Requires=zookeeper.service
   
    [Service]
    Type=simple
    Environment="JAVA_HOME=/usr/lib/jvm/java-11-openjdk-arm64"
    ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
    ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh
   
    [Install]
    WantedBy=multi-user.target
   

   Modify your java path in Environemt="JAVA_HOME=<Your_Path>“ as required. You can find it using

    readlink -f $(which java)
   

5. Modify server config

    nano /usr/local/kafka/config/server.properties
   

   Paste the following under Socket Server Settings.

    listeners=PLAINTEXT://0.0.0.0:9092,PLAINTEXT_EXTERNAL://0.0.0.0:9093
    advertised.listeners=PLAINTEXT://localhost:9092,PLAINTEXT_EXTERNAL://<YOUR_PUBLIC_IP>:9093
    listener.security.protocol.map=PLAINTEXT:PLAINTEXT,PLAINTEXT_EXTERNAL:PLAINTEXT
   

   replace <YOUR_PUBLIC_IP> with the public IP above.

6. Reload systemd daemon

    sudo systemctl daemon-reload
   

7. Start zookeeper & kafka server

    sudo systemctl start zookeeper
    sudo systemctl start kafka
    sudo systemctl status kafka
   

8. Create a Topic

    usr/local/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic myTopic
   

On Databricks

Create a new notebook after initializing a cluster, create the following cells

%sh
java -version

%sh
wget https://archive.apache.org/dist/kafka/3.2.0/kafka_2.12-3.2.0.tgz
ls -ltr ./

%sh
tar -xzf kafka_2.12-3.2.0.tgz

%sh
cd kafka_2.12-3.2.0
bin/kafka-console-consumer.sh --topic myTopic --bootstrap-server <YOUR_PUBLIC_IP>:9093

replace <YOUR_PUBLIC_IP> with the public IP above.

Naturally, I forgot how I had done this a year ago—because, of course, who remembers such things after that long? So I did what anyone would do: started Googling all over again.

I figured this would be the simplest way to keep track of documentation I might need later—so here we are. This is a quick, no-nonsense guide for future-me (and maybe even future-you) on how to allow external traffic to a specific port on an Oracle Cloud VPS.

1. Open Oracle Cloud Console

   

2. Go to VCN

   

3. Select the VCN in which the current server is configured.

   

4. Select the subnet in which the current server is configured.

   

5. Select the Default Security List for vcn-name , where vcn-name would be the name of your VCN.

   

6. Click on Add Ingress Rules and enter the following:

   Source CIDR : 0.0.0.0/0 (allow all)

   Destination Port Range : 9093

   IP Protocol : TCP

   Description : Kafka ( or anything you would like to write so that you remember which service you have reserved the port for )

   

   

7. Now Log on to your Server using SSH and Allow port 9093 to accept TCP Connections on port 9093 on the Firewall. (Oracle Cloud Infrastructure Ubuntu Installations use iptables by default )

sudo iptables -A INPUT -p tcp --dport 9093 -j ACCEPT