Tagged ORACLE MANAGEMENT CLOUD

Oracle Management Cloud: Log Analytics Advanced Features:Extended Fields & Enrichment


In this use case, we will extract geographic information from log records of communication devices that protect us from attacks and we will represent it in a map.

abstract barbed wire black white black and white
Photo by Pixabay on Pexels.com

This is a somewhat obfuscated real record:

Aug 14 12:02:45 XXXX_F5_DMZXX err dcc[11457]: 9999999999:9: [SECEV] Request blocked, violations: Web scraping detected. HTTP protocol compliance sub violations: N/A. Evasion techniques sub violations: N/A. Web services security sub violations: N/A. Virus name: N/A. Support id: 99999999999999999, source ip: 999.999.999.999, xff ip: N/A, source port: 99999, destination ip: 999.999.999.999, destination port: 999, route_domain: 200, HTTP classifier: /Common/www.xxxxxxxxx.yy.http, scheme HTTPS, geographic location: <RU>, request: <GET /ns/xxxxxx.yyy?id_seccion=9999 HTTP/1.1\r\nContent-Length: 0\r\nCookie: XXXXXXXX_XXXXXXXXX=d8b78f937f6f9d569cda500fd5cae49>, username: <8117.1533970714@MAILCATCH.COM>, session_id: <d9ba5ea0f4e98df0>

To do this we must configure these two characteristics in the log source:

Extended Fields

It is a way to extract values from a field in the log record by means of a regular expression. We are going to extract the source IP and put it in the “Source IP” field:

extended field

Base Field: Message
Example Content: Whatever
Extended Field Extraction Expression: source ip:?\s{Source IP:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}}
Enabled: true

 

Field Enrichment

It is about obtaining geographic information from the origin ip that we have obtained before.

enrinchment

enrinchment1

Wait 5 minutes or so and you will start to see records on the map:

mapa1.png

If you select a continent and zoom (+) you can see by countries:

mapa2

 

Enjoy 😉

Oracle Management Cloud: Log Analytics Recipes


Here a bunch of queries I’ve been working recently. Hope it helps!

close up code coding computer
Photo by Lorenzo Cafaro on Pexels.com

SEARCH FOR NOT FOUND WEB PAGES

Perform the following query:

404 and 'Log Source' like '%access%' 

SEARCH REQUEST THAT RETRIEVE A PAYLOAD BIGER THAN A SIZE AND ORDER THEM BY SIZE DESCENDING

'Content Size' > 0 | eval MB = 'Content Size' / (1024 * 1024) | where MB > X | sort -MB

Where X is the size in MB, for example:

'Content Size' > 0 | eval MB = 'Content Size' / (1024 * 1024) | where MB > 5 | sort -MB

DETECT IF MORE THAN ONE LOG SOURCE IS LOADING THE SAME LOG RECORDS

My colleague created a new log source for OSB access logs with a pattern that was already configured in the WLS Server log source. In the osa access log source he included a data filter but all the records appeared in searches. My guess was that the records were processed by the 2 log sources… With this query I found that record were coming from the 2 sources:

<unique expression you have checked is only in one log file>  | distinct 'Log Source'
'/pedi2/img/u173.png' | distinct 'Log Source'

REJECT ALL RECORDS EXCEPT THOSE CONTAINING “or /AAA/ or /BBB/ or /CCC/ or /DDD/”

SOLUTION: Create a data filter with an expression like:

^(?!.*(\/AAA\/|\/BBB\/|\/CCC\/|\/DDD\/)).*

Example:

^(?!.*(\/pedis\/|\/regus\/|\/pedi2\/|\/regis\/)).*

 

SEARCH FOR A PHRASE

Enclose it in ” or “”. Example:

'MANAGERS PedidoManagerMov' and 'Log Source' = 'MDONA WEBAPP Logs' | cluster 
analytics text
Photo by Timur Saglambilek on Pexels.com

Enjoy 😉