Set Up an Ingress Controller in OKE Cluster

Step 1: setting up the nginx ingress controller

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml

Step 2: exposing the ingress as a service of type LoadBalancer (as a public IP)

kubectl apply -f https://raw.githubusercontent.com/javiermugueta/rawcontent/master/cloud-generic.yaml

Step 3: Execute this command several times until the external-ip appears as non <pending>, grab the IP for later testing

kubectl get svc -n ingress-nginx

Step 4: a certificate for ssl termination

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"

kubectl create secret tls tls-secret --key tls.key --cert tls.crtopenssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"

Step 5

kubectl create -f https://raw.githubusercontent.com/javiermugueta/rawcontent/master/hello-world-ingress.yaml

Step 6: A typical hello world deployment with a pod in it and 3 replicas

kubectl create -f https://raw.githubusercontent.com/javiermugueta/rawcontent/master/ingress.yaml

Step 7: Test the url and see what happens…

That’s all folks! Hope it helps 🙂

Deploying a Coherence Cluster in Kubernetes

Coherence-Operator is a Kubernetes opeartor for deploying Oracle Coherence in k8s. Let’s see how to do it.

1 Clean previous setup intents:

helm del --purge sample-coherence
helm del --purge sample-coherence-operator
kubectl delete namespace sample-coherence-ns

2 Execute the following:

kubectl config set-context $(kubectl config current-context) --namespace=sample-coherence-ns

helm repo add coherence https://oracle.github.io/coherence-operator/charts

helm repo update

helm --debug install coherence/coherence-operator --name sample-coherence-operator --set "targetNamespaces={}" --set imagePullSecrets=sample-coherence-secret

helm ls

helm status sample-coherence-operator

3 Create a secret with your credentials to Oracle Container Registry:

kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='youruser' --docker-password='yourpasswd' --docker-email='youremail'

4 Install:

helm --debug install coherence/coherence --name sample-coherence --set imagePullSecrets=oracle-container-registry-secret

5 Proxy to a pod:

export POD_NAME=$(kubectl get pods --namespace sample-coherence-ns -l "app=coherence,release=sample-coherence,component=coherencePod" -o jsonpath="{.items[0].metadata.name}")

kubectl --namespace sample-coherence-ns port-forward $POD_NAME 20000:20000

6 Download and install Coherence Stand Alone

7 Download this and this

8 Build the client:

export COHERENCE_HOME=~/Oracle/Middleware/Oracle_Home/coherence
javac -cp .:${COHERENCE_HOME}/lib/coherence.jar HelloCoherence.javaexport COHERENCE_HOME=~/Oracle/Middleware/Oracle_Home/coherence

9 Test it:

java -cp .:${COHERENCE_HOME}/lib/coherence.jar -Dcoherence.cacheconfig=$PWD/example-client-config.xml HelloCoherence

2019-07-11 01:21:33.575/0.538 Oracle Coh...


Oracle Coherence Version Build 68243

2019-07-11 01:21:34.430/1.392 Oracle 

The value of the key is 7

That’s all, hope it helps 🙂

WebLogic Kubernetes Operator: Deploying a Java App in a WebLogic Domain on Oracle Kubernetes Engine (OKE) in 30 Minutes

WebLogic Kubernetes Operator provides a way of running WLS domains in a k8s cluster.

For this post we are depicting the steps of the tutorial you can find in the documentation here. So let’s get started!

What you need:

  • a k8s cluster
  • kubectl
  • maven
  • git
  • docker
  • 60 minutes
git clone https://github.com/oracle/weblogic-kubernetes-operator

docker login

docker pull oracle/weblogic-kubernetes-operator:2.2.0

docker pull traefik:1.7.6

For the next step, if you don’t have a user, goto https://container-registry.oracle.com and register yourself

docker login container-registry.oracle.com 

docker pull container-registry.oracle.com/middleware/weblogic:

K8s uses role based access control (RBAC):

cat <<EOF | kubectl apply -f -
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
   name: helm-user-cluster-admin-role
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: cluster-admin
 kind: ServiceAccount
 name: default
 namespace: kube-system

Traefik is a router:

helm install stable/traefik \
   --name traefik-operator \
   --namespace traefik \
   --values kubernetes/samples/charts/traefik/values.yaml  \
   --set "kubernetes.namespaces={traefik}" \
cat <<EOF < values.yaml
 serviceType: NodePort
     http: "30305"
     https: "30443"
   enabled: true
   domain: traefik.example.com
   enabled: true
   enabled: true
   #enforced: true 
   #upstream: true
   #insecureSkipVerify: false
   tlsMinVersion: VersionTLS12
helm install stable/traefik --name traefik-operator --namespace traefik --values values.yaml  --set "kubernetes.namespaces={traefik}" --wait

Namespace for the operator:

kubectl create namespace sample-weblogic-operator-ns

kubectl create serviceaccount -n sample-weblogic-operator-ns sample-weblogic-operator-sa
cd weblogic-kubernetes-operator/

helm install kubernetes/charts/weblogic-operator \
   --name sample-weblogic-operator \
   --namespace sample-weblogic-operator-ns \
   --set image=oracle/weblogic-kubernetes-operator:2.2.0 \
   --set serviceAccount=sample-weblogic-operator-sa \
   --set "domainNamespaces={}" \
kubectl create namespace sample-domain1-ns

helm upgrade \
   --reuse-values \
   --set "domainNamespaces={sample-domain1-ns}" \
   --wait \
   sample-weblogic-operator \
helm upgrade \
   --reuse-values \
   --set "kubernetes.namespaces={traefik,sample-domain1-ns}" \
   --wait \
   traefik-operator \

Creating the WLS domain image:

kubernetes/samples/scripts/create-weblogic-domain-credentials/create-weblogic-credentials.sh \
   -u weblogic -p welcome1 -n sample-domain1-ns -d sample-domain1

Tag the docker image created and push to a registry:

docker images

docker tag container-registry.oracle.com/middleware/weblogic: javiermugueta/weblogic:

docker push javiermugueta/weblogic:

NOTE: Remember to make private this image in the registry!!! As a recommended option, please follow the steps here to push to the private registry offered by Oracle.

Now let’s make a copy of the yaml file with properties to change and put the appropriate values:

cp kubernetes/samples/scripts/create-weblogic-domain/domain-home-in-image/create-domain-inputs.yaml .

mv create-domain-inputs.yaml mycreate-domain-inputs.yaml

vi mycreate-domain-inputs.yaml

(change values in lines #16, #57, #65, #70, #104, #107 appropriately) Here the one I utilised just in case it helps

And now let’s create the domain with the image:

cd kubernetes/samples/scripts/create-weblogic-domain/domain-home-in-image

./create-domain.sh -i ~/Downloads/weblogic-kubernetes-operator/mycreate-domain-inputs.yaml -o ~/Downloads/weblogic-kubernetes-operator/output -u weblogic -p welcome1 -e

Verify that everything ig working!

kubectl get po -ns sample-domain1-ns

kubectl get svc -ns sample-domain1-ns

Change the type of the cluster and adminserver services to LoadBalancer:

kubectl edit svc/sample-domain1-cluster-cluster-1 -n sample-domain1-ns

kubectl edit svc/sample-domain1-admin-server-external -n sample-domain1-ns
Use vi commands

Verify and write down the public IP’s of the AdminServer external service and the cluster:

kubectl get svc -ns sample-domain1-ns

Create a simple java app and package it:

mvn archetype:generate -DgroupId=javiermugueta.blog -DartifactId=java-web-project -DarchetypeArtifactId=maven-archetype-webapp -DinteractiveMode=false

mnv package

Open a browser, log in WLS AdminServer console and deploy your app (use the public IP of the AdminsServer service):

Open a new browser tab and test the app (use the public IP of the WLS cluster service):

That’s all folks, hope it helps!! 🙂

Load Balancig, High Availability and Fail-Over of a Micro-Service Deployed in two Separated Kubernetes Clusters: one running in Oracle Kubernetes Engine (OKE) and the other in Google Kubernetes Engine (GKE)

Oracle Cloud Edge Services

Oracle Cloud Infrastructure provides Edge Services, is a group of services related with DNS, Health Checks, Traffic Management and WAF (Web Application Firewall).

In this episode we are utilising DNS Zone Management, Traffic Management Steering Policies and Health Checks for load balancing and fail-over of a micro-service running in two different Kubernetes clusters, in two different regions and distinct cloud providers, giving a robust solution that accomplishes a very powerful load balanced, active-active and disaster recovery topology.

Deploying the micro-service

Deploy the following to two different k8s clusters, such as OKE in two distinct regions or OKE and GKE. As OKE and GKE are petty much identical, we can use kubectl and Kubernetes Dashboard in both of them as we prefer:

k8s deployment in OKE visualised with Kubernetes kubectl
k8s deployment in OKE visualised with Kubernetes dashboard

It is a very simple service that greets you and says where is it running.

Greetings from OKE
Greetings from GKE

Configuring DNS

For this part of the setup we need a FQDN registered, we are using bigdatasport.org, a name registered by myself.

Let’s create domain entries in OCI. Create a DNS zone in OCI as follows:

Now, let’s grab the DNS servers and go to our Registrar and change the DNS’s configuration so that they point to Oracle DNS’s:

Verify the change:

Configuring Health Checks

Let’s create a Health Check that we’ll use later in the traffic management. Health checks are performed external to OCI from a list of vantage points executed in Azure, Google or AWS, select your preferred choice.

Configuring Traffic Management Steering Policies

Let’s create a traffic management policy as follows:

Testing it all

Ok, we have all the tasks already done, let’s test it!

Delete the deployment in OKE:

Go to the Traffic policy and verify that the OKE endpoint is unhealthy:

Go to your browser and request http://bigdatasport.org/greet, as you can see the service is retrieved from GKE:

Redeploy in OKE again:

As you can see, the OKE service is running well again:

Now let’s delete the deployment in GKE:

Now the greeting is retrieved again form OKE:

And that’s all folks, hope it helps! 🙂

Oracle Kubernetes (OKE): Deploying a Custom Node.js Web Application Integrated with Identity Cloud Service for Unique Single Sign On (SSO) User Experience

In this post we are deploying a custom Node.js web application in Oracle Kubernetes Engine (OKE).

What we want to show is how to configure the custom web application in order to have a unique Single Sing On experience.

First part

Follow this tutorial here explaining how to enable SSO to the web app running locally

Second part

Now we are making small changes to deploy on kubernetes

Create a Dockerfile in the nodejs folder of the cloned project with the following:
FROM oraclelinux:7-slim
ADD . /app
RUN curl --silent --location https://rpm.nodesource.com/setup_11.x | bash -
RUN yum -y install nodejs npm --skip-broken
CMD ["npm","start"]
Create K8s deployment file as follows:
apiVersion: v1
kind: Service
name: idcsnodeapp
type: LoadBalancer
app: idcsnodeapp
- name: client
protocol: TCP
port: 3000
Deploy to k8s:
kubectl apply -f service.yaml
Grab the url of the new external load-balancer service created in k8s and modify the file auth.js with the appropriate values in your cloud environment
var ids = {
oracle: {
"ClientId": "client id of the IdCS app",
"ClientSecret": "client secret of the IdCS app",
"ClientTenant": "tenant id (idcs-xxxxxxxxxxxx)",
"IDCSHost": "https://tenantid.identity.oraclecloud.com",
"AudienceServiceUrl" : "https://tenantid.identity.oraclecloud.com",
"TokenIssuer": "https://identity.oraclecloud.com/",
"scope": "urn:opc:idm:t.user.me openid",
"logoutSufix": "/oauth2/v1/userlogout",
"redirectURL": "http://k8sloadbalancerip:3000/callback",
Build the container and push to a repo you have write access to, such as:
docker build -t javiermugueta/idcsnodeapp .
docker push javiermugueta/idcsnodeapp
Modify the IdCS application with the public IP of the k8s load-balancer service
Create k8s deployment file as follows:
apiVersion: apps/v1
kind: Deployment
name: idcsnodeapp
app: idcsnodeapp
replicas: 1
app: idcsnodeapp
type: Recreate
app: idcsnodeapp
- image: javiermugueta/idcsnodeapp
name: idcsnodeapp
- containerPort: 3000
name: idcsnodeapp

Deploy to k8s
kubectl apply -f  deployment.yaml
Test the app and verify SSO is working:

This slideshow requires JavaScript.

Hope it helps! 🙂


How to ssh to OKE (k8s) Private Node (worker compute node) via Jump Box (Bastion Server)

In OKE typically you create, for redundancy and high availability reasons, a k8s cluster in 5 or more subnets:

  • 2 are public and, in there, is where the public load balancer is deployed, for example one in AD1 and the other in AD3
  • 3 or more are private, and, in there, is where the worker compute nodes are deployed, for example one subnet in AD1, other in AD2, other in AD3 and looping…

If you need to reach one or more compute worker nodes for some reason, you can create a bastion server (jump box) with a public IP and then do the following:

ssh -i privateKey -N -L localhost:2222:k8scomputenode:22 opc@jumpboxpublicip

ssh -i privateKey -p 2222 opc@localhost

Hope it helps! 🙂



Shared Disk for your Pods: PersistentVolumes for Oracle Kubernetes Engine (OKE) Implemented as NFS File Storage in Oracle Cloud Infrastructure (OCI)

When you deploy in k8s a pod depending in persistent volume attached to block storage (for example this post), the volume created is mounted on a specific node. If that node fails or is stopped, the pods running on it fail when trying to be created in other node according to the replication policies they have, because other nodes do not have the disk mounted.

Oracle Cloud Infrastructure (OCI) File Systems are shared storage that you can easily expose/attach to your pods for those use cases where shared persistent data is needed.

Of course, you can still mount the disk in each node, but this is not a good approach because we have a better way to achieve it.

So, let’s get started

Go to OCI dashboard, create a new File System and a new Export called /myexport. Click on the mount target link and take note of the File System IP address.


This slideshow requires JavaScript.

Download and deploy the following yaml, it creates:

  • a persistentVolume
  • a persistentVolumeClaim
  • a deployment with 3 replicas of a container image with nginx in it
  • a service with public IP and LoadBalancer with round-robin policy
kubectl apply -f k8spod2nfs.yaml

Get a list of the pods and “ssh” to one of them

kubectl get pods
k8spod2nfs-xxx 1/1 Running 0 99s
k8spod2nfs-yyy 1/1 Running 0 99s
k8spod2nfs-zzz 1/1 Running 0 99s
kubectl exec -it k8spod2nfs-xxx bash

Go to /usr/share/nginx/html directory and create or edit a file called index.html

cd /usr/share/nginx/html/

echo hola > index.html

You can also “ssh” to other pod and verify that you see the same file

Now get the list of services and grab the public IP of the k8spod2nfs service

kubectl get services
k8spod2nfs LoadBalancer x.y.z.t 80:31014/TCP 10m
kubernetes ClusterIP <none> 443/TCP 7d4h

Go to http://yourserviceip in your browser:


Change the content of the index.html file:

echo adios >> index.html

Go to http://yourserviceip in your browser again, the data changes


Delete all pods of the k8spod2nfs deployment and wait until at lest one of them are recreated again:

kubectl delete pod -l app=k8spod2nfs

Go to http://yourserviceip in your browser again, the data is still in there!

Delete the deployment and create it again:

kubectl delete -f k8spod2nfs.yaml

kubectl apply -f k8spod2nfs.yaml

Wait until al least one of the ñods are ready and go to http://yourserviceip in your browser again, the data is still in there!

As you can see, the data is shared across all pods and is persistent! (unless you delete it or destroy the OCI FIleSystem)

Hope it helps! 🙂