Troubleshooting SNMP Probe Load

Technote (FAQ)

Question

MTTrapd probe : How is trap delivery confirmed?

Cause

SNMP trap overload on probe port

Answer

IMPORTANT:
It is recommended that a firewall is installed on the SNMP(MTTrapd) probe server whenever event delivery is mission critical. This will allow the timely diagnosis of any event delivery issues.

The latest version of the probe should have the following log messages when traps are being deliberately dropped by the SNMP (MTTrapd) probe due to the trap Queue becoming full.

Warning: Dropping Trap!

During normal processing the following messages are available dependent upon message level;

Information: Number of items in the trap queue is 0
Debug: 1 trap in queue

Earlier versions of the probe may not have these messages.

To diagnose SNMP delivery the IP analysis tool for the probes platform should be used.
e.g.
Ethereal/WireShark
Capture a fixed number of traps (e.g. The TrapQueue size 20,000) then analyse the data using the Analyse functions. The most useful of which is by IP address. The Analysis Summary should indicate how much data is being sent to the probes port.

To examine load processing via the probes rulesfile use;

if ( match(%counter,"") )
{
%counter = 1
%start_time=getdate
}
else
{
%end_time=getdate
$time_elapsed = real(int(%end_time) - int(%start_time))

if (int($time_elapsed) > 59 )
{
$current_load = real(%counter) / real($time_elapsed)
log(info,"Events per second = " + $current_load + " " + (%counter) + " [ " + $time_elapsed + " ]")
%counter = 1
%start_time=getdate
}
else
{
%counter = int(%counter) + 1
}
}


The difference between the two loads, IP on the SNMP (MTTrapd) port and the events that the probe is actually parsing, should help diagnose issues.
It may be necessary to reduce the overall loading of the probe by using more SNMP (MTTrapd) probes or by reducing the events being sent.

The use of a firewall on the probe server allows timely diagnosis of issues as well as controlling access to the SNMP (MTTrapd) probes port.

http://www-01.ibm.com/support/docview.wss?uid=swg21327391