From August 2018

Oracle Management Cloud: Log Analytics Advanced Features:Extended Fields & Enrichment


In this use case, we will extract geographic information from log records of communication devices that protect us from attacks and we will represent it in a map.

abstract barbed wire black white black and white
Photo by Pixabay on Pexels.com

This is a somewhat obfuscated real record:

Aug 14 12:02:45 XXXX_F5_DMZXX err dcc[11457]: 9999999999:9: [SECEV] Request blocked, violations: Web scraping detected. HTTP protocol compliance sub violations: N/A. Evasion techniques sub violations: N/A. Web services security sub violations: N/A. Virus name: N/A. Support id: 99999999999999999, source ip: 999.999.999.999, xff ip: N/A, source port: 99999, destination ip: 999.999.999.999, destination port: 999, route_domain: 200, HTTP classifier: /Common/www.xxxxxxxxx.yy.http, scheme HTTPS, geographic location: <RU>, request: <GET /ns/xxxxxx.yyy?id_seccion=9999 HTTP/1.1\r\nContent-Length: 0\r\nCookie: XXXXXXXX_XXXXXXXXX=d8b78f937f6f9d569cda500fd5cae49>, username: <8117.1533970714@MAILCATCH.COM>, session_id: <d9ba5ea0f4e98df0>

To do this we must configure these two characteristics in the log source:

Extended Fields

It is a way to extract values from a field in the log record by means of a regular expression. We are going to extract the source IP and put it in the “Source IP” field:

extended field

Base Field: Message
Example Content: Whatever
Extended Field Extraction Expression: source ip:?\s{Source IP:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}}
Enabled: true

 

Field Enrichment

It is about obtaining geographic information from the origin ip that we have obtained before.

enrinchment

enrinchment1

Wait 5 minutes or so and you will start to see records on the map:

mapa1.png

If you select a continent and zoom (+) you can see by countries:

mapa2

 

Enjoy 😉

Oracle Management Cloud: Log Analytics Recipes


Here a bunch of queries I’ve been working recently. Hope it helps!

close up code coding computer
Photo by Lorenzo Cafaro on Pexels.com

SEARCH FOR NOT FOUND WEB PAGES

Perform the following query:

404 and 'Log Source' like '%access%' 

SEARCH REQUEST THAT RETRIEVE A PAYLOAD BIGER THAN A SIZE AND ORDER THEM BY SIZE DESCENDING

'Content Size' > 0 | eval MB = 'Content Size' / (1024 * 1024) | where MB > X | sort -MB

Where X is the size in MB, for example:

'Content Size' > 0 | eval MB = 'Content Size' / (1024 * 1024) | where MB > 5 | sort -MB

DETECT IF MORE THAN ONE LOG SOURCE IS LOADING THE SAME LOG RECORDS

My colleague created a new log source for OSB access logs with a pattern that was already configured in the WLS Server log source. In the osa access log source he included a data filter but all the records appeared in searches. My guess was that the records were processed by the 2 log sources… With this query I found that record were coming from the 2 sources:

<unique expression you have checked is only in one log file>  | distinct 'Log Source'
'/pedi2/img/u173.png' | distinct 'Log Source'

REJECT ALL RECORDS EXCEPT THOSE CONTAINING “or /AAA/ or /BBB/ or /CCC/ or /DDD/”

SOLUTION: Create a data filter with an expression like:

^(?!.*(\/AAA\/|\/BBB\/|\/CCC\/|\/DDD\/)).*

Example:

^(?!.*(\/pedis\/|\/regus\/|\/pedi2\/|\/regis\/)).*

 

SEARCH FOR A PHRASE

Enclose it in ” or “”. Example:

'MANAGERS PedidoManagerMov' and 'Log Source' = 'MDONA WEBAPP Logs' | cluster 
analytics text
Photo by Timur Saglambilek on Pexels.com

Enjoy 😉

Helm: Error: incompatible versions client[v2.9.1] server[v2.8.2]


antique armor black and white chrome
Photo by Mikes Photos on Pexels.com

Working with helm on my mac against a Kubernetes cluster in Oracle Cloud as get an error:

MacBook-Pro-de-javi:wishlist javiermugueta$ helm install -n wishlist-chart -f values.yaml .
Error: incompatible versions client[v2.9.1] server[v2.8.2]
MacBook-Pro-de-javi:wishlist javiermugueta$ brew info kubernetes-helm
kubernetes-helm: stable 2.9.1 (bottled), HEAD
The Kubernetes package manager
https://helm.sh/
/usr/local/Cellar/kubernetes-helm/2.9.1 (50 files, 66.2MB) *
Poured from bottle on 2018-08-04 at 18:55:56
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/kubernetes-helm.rb
==> Dependencies
Build: mercurial ✘, go ✘, glide ✘
==> Options
--HEAD
Install HEAD version
==> Caveats
Bash completion has been installed to:
/usr/local/etc/bash_completion.d

The following worked for me:

MacBook-Pro-de-javi:wishlist javiermugueta$ helm init --upgrade
Creating /Users/javiermugueta/.helm
Creating /Users/javiermugueta/.helm/repository
Creating /Users/javiermugueta/.helm/repository/cache
Creating /Users/javiermugueta/.helm/repository/local
Creating /Users/javiermugueta/.helm/plugins
Creating /Users/javiermugueta/.helm/starters
Creating /Users/javiermugueta/.helm/cache/archive
Creating /Users/javiermugueta/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /Users/javiermugueta/.helm.

Tiller (the Helm server-side component) has been upgraded to the current version.
Happy Helming!

 

Now it works!

MacBook-Pro-de-javi:wishlist javiermugueta$ helm install -n wishlist-chart -f values.yaml .
NAME: wishlist-chart
LAST DEPLOYED: Sat Aug 4 21:25:46 2018
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
wishlist-chart NodePort 10.96.84.197 <none> 8080:31124/TCP,8082:32204/TCP,8081:31772/TCP 1s

==> v1/Deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
wishlist-chart 3 3 3 0 1s

==> v1/Pod(related)
NAME READY STATUS RESTARTS AGE
wishlist-chart-69796586bb-5frnz 0/3 ContainerCreating 0 1s
wishlist-chart-69796586bb-d9gn8 0/3 ContainerCreating 0 1s
wishlist-chart-69796586bb-z8l67 0/3 ContainerCreating 0 1s


NOTES:
1. Get the application URL by running these commands:
export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services wishlist-chart)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT

Enjoy 😉

Deploy Oracle Database to Kubernetes Cluster (Oracle Cloud Container Clusters)


red lights in line on black surface
Photo by Pixabay on Pexels.com

Disclaimer: This is for knowledge sharing purposes, review licensing matters here

I have pushed the image mentioned in my public repo for your convenience.

Follow this simple steps:

kubectl create deployment orcl --image=docker.io/javiermugueta/orcl
...
kubectl expose deployment orcl --type=LoadBalancer --port=1521
...
kubectl get services

Take a look to the public IP created (with kubectl get services)  and connect with SqlDeveloper as follows:

orclk8ssqldevconn

Enjoy 😉

Deploy Sample Application to Kubernetes Cluster (Oracle Cloud Container Clusters)


bandwidth close up computer connection
Photo by panumas nikhomkhai on Pexels.com

Gonna deploy in K8S a simple-sample HTML5/JS app made with JET Toolkit

  1. Grab a K8S cluster from here
  2. Follow setup instructions from here and here and here and/or check this post
  3. And finally execute this:
kubectl create deployment jetapp --image=docker.io/javiermugueta/myjetapp
kubectl expose deployment jetapp --type=LoadBalancer --port=8000

Enjoy 😉

Oracle Self Service Integration Cloud: Integration for Citizen Developers


I mean Citizen, or in other words, “Do It Yourself” (DIT) as an approach in which business users create applications or automate integration task without the need of IT.

people on a meeting
Photo by rawpixel.com on Pexels.com

Concepts

  • Recipes: Perform actions when a condition occurs
  • Editor: Configure recipes in simple mode, step by step and with drag and drog
  • Business User oriented: Recipes catalog, connectors, business data an more
  • Dashboard: Statistics, run now, job history and more

Learn more about Self Service Integration here

This slideshow requires JavaScript.

Related tools: Learn more about Autonomous Visual Builder here

Enjoy 😉