Tag: KAFKA

Near Real Time Data Driven SaaS Integration with Streaming | Part 2: Extracting/Publishing Data Changes to Streaming Topics


As we mentioned in a previous post, the idea here is to develop a construct that seeks for changes in the SaaS data system and publishes those changes to a stream for later consumption from other systems over there. Indeed, as we see in the diagram, there is a block in which we are executing a program … Continue reading Near Real Time Data Driven SaaS Integration with Streaming | Part 2: Extracting/Publishing Data Changes to Streaming Topics →

Near Real Time Data Driven SaaS Integration with Streaming | Part 1: Overview


File based approaches File-based data exchange comfort a large percentage of the integrations between SaaS and other ERP solutions in the past and today. The approach is robust and allows a large number of transactions to be executed in batch without affecting the online systems, but it has an inconvenience: the information is not updated … Continue reading Near Real Time Data Driven SaaS Integration with Streaming | Part 1: Overview →

Building Producer and Consumer Clients in go Language for Oracle Event Hub Cloud Service


CREATE AN OEH CLUSTER Documentation here or watch following video: When the cluster is created, go to details page and grab the connection url, which has the  format <broker1_ip>:6667,...,<brokern_ip>:6667 CREATE A TOPIC INSTALL THE LIB I've done the following (Mac): git clone https://github.com/edenhill/librdkafka.git cd librdkafka ./configure --prefix /usr/local make sudo make install (sudo make uninstall … Continue reading Building Producer and Consumer Clients in go Language for Oracle Event Hub Cloud Service →

jMeter Test Script for Kafka (tested against Oracle Event Hub Cloud Service)


In this post we are testing a jMeter script that sends messages to Kafka Cluster running in Oracle Event Hub (OEH). jMETER CONFIG Install plugins manager in JMeter if you don't have it, restart jMeter and then install Kafka Support, then you will be told to restart jMeter. jMETER SCRIPT Download the jMeter script from … Continue reading jMeter Test Script for Kafka (tested against Oracle Event Hub Cloud Service) →

Creating a Fast&Simple Container for Sending Messages to a Topic in Oracle Event Hub Cloud Service (aka OEHCS, which is a Kafka cluster) and Deploying it to Kubernetes Cluster


The container uses 4 environment variables, you can find a container already built for you here SOURCE CODE OF THE PRODUCER var sleep = require('system-sleep'); const oehcs_connect_url = process.env.OEHCS_CONNECTURL const topic_name = process.env.TOPIC_NAME const num_partitions = process.env.NUM_PARTITIONS const message = process.env.MESSAGE var kafka = require('kafka-node'), HighLevelProducer = kafka.HighLevelProducer, client = new kafka.KafkaClient({kafkaHost: oehcs_connect_url}), producer = new … Continue reading Creating a Fast&Simple Container for Sending Messages to a Topic in Oracle Event Hub Cloud Service (aka OEHCS, which is a Kafka cluster) and Deploying it to Kubernetes Cluster →

Mirroring a Topic Between 2 Oracle Event Hub (Kafka) Clusters in 20 Minutes


Oracle Event Hub Cloud Service (OEHCS) is a Kafka managed PaaS cloud service. In a few minutes you can provision a full Kafka cluster ready for creating topics for sending and consuming messages. In this post we will configure a feature called 'mirroring' that allows you to replicate the messages that are sent to a … Continue reading Mirroring a Topic Between 2 Oracle Event Hub (Kafka) Clusters in 20 Minutes →