What is SPLUNK

Splunk (the product) captures, indexes and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards and visualizations.
SPLUNK Web Site

Sample SPLUNK Dashboard

SPLUNK_dashboard

Using SPLUNK

Once you have added the LAMAXU logs data to SPLUNK the MQ metrics can be queried using SPLUNK search syntax.

UsingSplunk

Graphing the Current Queue Depth

source=”statusData.log” sourcetype=”_json_lamaxuStatus” “identifier.dataSource”=DEMO “attributes.CURDEPTH”=”*” “attributes.QUEUE”!=”AMQ.*” “attributes.QUEUE”!=”SYSTEM.*”| timechart avg(“attributes.CURDEPTH”) by attributes.QUEUE

GraphCurrent QueueDepth

Graphing Message Volume Statistics

source=”statData.log” sourcetype=_json_lamaxuEvents “identifier.dataType”=STATISTICS  identifier.dataSource”=DEMO ” attributes.QUEUE”=”QM_TEST_REPLY1*” | eval pfields=split(‘attributes.MQIAMO_PUTS’, ” “)  | eval PUTSNP=mvindex(pfields,0)| eval PUTSP=mvindex(pfields,1) |eval TPUTS=PUTSNP+PUTSP| eval gfields=split(‘attributes.MQIAMO_GETS’, ” “) | eval GETSNP=mvindex(gfields,0)| eval GETSP=mvindex(gfields,1) |eval TGETS=GETSNP+GETSP | timechart avg(attributes.MQIAMO_Q_MAX_DEPTH)  as “Hwm” avg(TPUTS) as “Puts” avg(TGETS) as Gets

GraphMsgVolumeStats

Graphing Message Time on Queue

source=”statusData.log” “attributes.QTIME”=”*” “attributes.QUEUE”!=”SYSTEM.*” “attributes.QUEUE”!=”AMQ.*” sourcetype=”_json_lamaxuStatus” “identifier.dataSource”=DEMO| timechart avg(“attributes.QTIME”) by attributes.QUEUE

GraphMsgTimeOnQueue

LAMAXU/SPLUNK Source Types

The following source type need to be added to your SPLUNK props.conf file in order to correctly recognise the time-stamp fields in the log file. Alternatively you could manually create the source types using the red highlight values as a guide to defining the time-stamps.

Source type for LAMAXU Status logs

[_json_lamaxuStatus]
INDEXED_EXTRACTIONS = json
KV_MODE = none
NO_BINARY_CHECK = true
TIMESTAMP_FIELDS = createDate
TIME_FORMAT = %d/%m/%Y %OI:%M:%S %p
TZ = Australia/Sydney
category = Structured
description = JavaScript Object Notation format. For more information, visit
http://json.org/
disabled = false
pulldown_type = true

Source type for LAMAXU Event logs

[_json_lamaxuEvents]
INDEXED_EXTRACTIONS = json
KV_MODE = none
NO_BINARY_CHECK = true
TIMESTAMP_FIELDS = attributes.MQCAMO_END_DATE,attributes.MQCAMO_END_TIME
TIME_FORMAT = %Y-%m-%d %H.%M.%S
TZ = Australia/Sydney
category = Structured
description = JavaScript Object Notation format. For more information, visit
http://json.org/
disabled = false
pulldown_type = true

Log File Format for statusData.log

Log file attributes are modeled on the IBM MQ runmqsc attribute names.

QMGR_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "QMGR_STATUS",
        "sui": "DEMO"
    },
    "createDate": "10/10/2015 9:47:45 AM",
    "attributes": {
        "STARTTI": "13.44.07",
        "INSTDESC": "",
        "CONNS": "24",
        "STANDBY": "NOPERMIT",
        "INSTNAME": "Installation1",
        "CHINIT": "RUNNING",
        "STATUS": "RUNNING",
        "STARTDA": "2015-09-30",
        "CMDSERV": "RUNNING",
        "LDAPCONN": "INACTIVE",
        "QMNAME": "DEMO"
    },
    "category": "STATUS"
}
QUEUE_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "QUEUE_STATUS",
        "sui": "LOCALQNAME"
    },
    "createDate": "10/10/2015 11:47:42 AM",
    "attributes": {
        "IPPROCS": "2",
        "LGETDATE": "",
        "QTIME": "-1 -1",
        "QUEUE": "LOCALQNAME",
        "MONQ": "OFF",
        "OPPROCS": "2",
        "CURDEPTH": "0",
        "LGETTIME": "",
        "MSGAGE": "-1",
        "LPUTDATE": "",
        "LPUTTIME": "",
        "TYPE": "QSTATUS",
        "UNCOM": "NO"
    },
    "category": "STATUS"
}
SUBSCRIPTION_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "SUBSCRIPTION_STATUS",
        "sui": "TEST.SUBSCRIBER "
    },
    "createDate": "10/10/2015 9:47:45 AM",
    "attributes": {
        "MCASTREL": "-1 -1",
        "DURABLE": "NO",
        "LMSGTIME": "",
        "RESMDATE": "2015-10-10",
        "QMNAME": "DEMO",
        "CURDEPTH": "4",
        "MSGAGE": "0",
        "TOPICSTR": "TEST.TOPIC",
        "SUB": "TEST.SUBSCRIBER",
        "LMSGDATE": "",
        "SUBID": "414d512044454d4f2020202020202020885a0b56049a9c20",
        "NUMMSGS": "0"
    },
    "category": "STATUS"
} 
PUBSUB_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "PUBSUB_STATUS",
        "sui": "DEMO"
    },
    "createDate": "10/10/2015 9:47:45 AM",
    "attributes": {
        "TPCOUNT": "6",
        "STATUS": "ACTIVE",
        "TYPE": "LOCAL",
        "SUBCOUNT": "3",
        "QMNAME": "DEMO"
    },
    "category": "STATUS"
}
LISTENER_STATUS
{
    "identifier": {
        "dataSource": "DEMO",
        "dataType": "LISTENER_STATUS",
        "sui": "DEMO.1414"
    },
    "createDate": "10/10/2015 9:47:26 AM",
    "attributes": {
        "STARTTI": "00.31.37",
        "PORT": "1414",
        "PID": "38076556",
        "TRPTYPE": "TCP",
        "BACKLOG": "100",
        "IPADDR": "::ffff:10.30.40.74",
        "STATUS": "RUNNING",
        "LISTENER": "DEMO.1414",
        "STARTDA": "2015-10-10",
        "DESCR": "DEMO default Listener"
    },
    "category": "STATUS"