SAP Network Blog: SAP RFC que monitoring in Big Brother

Introduction

At my current work we faced a problem with internal rfc queues within sap for our sap xi solution. For monitoring and alerting we use a non sap tool called hobbitmon (big brother). I decided to share this solution since I can not imagine we where the only ones wanting to monitor queues in another monitoring tool.

We had a problem that queues sometimes stopped functioning and we did not signal this until customers started to complain that certain functions did not work properly anymore. A glance on smq1 and smq2 showed us that the queues where stopped in these situations and a manual start/unlock solved the problem in most cases.

We needed something to signal these incidents before customers started complaining. And after a little diggin we came up with a solution.

Background

This solution was developed on a suse 9 installation running db2. But if you run xi or sap on another system it might give you some leads on how to implement this kind of logging as well.

What we needed

We needed to integrate some form of alerting into our big brother monitoring system that would alert us if a que would go over a certain treshold. Big brother is a graphical monitoring application that shows the status of different systems and is able to send out sms. pager and email messages in certain conditions.

Solution

All information on messages that are currently in the que are stored in tables in the SAP database: ARFCRSTATE, ARFCSSTATE, TRFCQIN and TRFCQOUT are the once i used in this solution.

The solution consists out of 2 scripts that are scheduled using crontab, but any other schedular might work as well.

First script is a batch script that will extract the information we need out of the databse using 2 sql queries. The result is stored in a file on our native system:

db2 'select ARFCSTATE, count(*) from <database name here>.ARFCRSTATE GROUP BY ARFCSTATE union all select ARFCSTATE, count(*) from <database name here>
.ARFCSSTATE GROUP BY ARFCSTATE' > /home/exampleuser/queinfo.txt

db2 'select QNAME, QSTATE, count(*) from <database name here>
.TRFCQIN GROUP BY QNAME, QSTATE union all select QNAME, QSTATE, count(*) from <database name here>
.TRFCQOUT GROUP BY QNAME, QSTATE' >> /home/exampleuser
/queinfo.txt

Be sure to run this script under a user that has access to the database. In our case we use an admin user.

A second script is scheduled to run right after this shell script. This script is written in perl. And picks up the output from the first script, formats it and sends the information on to big brother.

#configure these parameters to the correct values.
sleep(5); # to make sure the first job is ready
my $flow = "example,system,com"; #party
my $test = "que"; #flow
my $colour = "green"; #status colour
my $text = "Trfcq and Qrfcq on Production system.\n\n"; #additional text
# Ok lets get the time so we can format the log file name

open (IN, "</home/exampleuser/queinfo.txt");
while (<IN>)
{
        $text .= $_;
}
close (IN);

if ($text=~/QNAME(.+?)(-+)\s(-+)\s(-+)(.+)/is)
{
    my @data = split(/\n/,$5);
    foreach my $line (@data)
    {
        if ($line=~/record/is)
        {
            next;
        }

        if ($line=~/(\S+?)\s(.+?)(\d+)/is)
        {
            if ($3 > 50)
            {
                $colour = "yellow";
            }

if ($3 > 100)

{

$colour = "red";

             }
        }
    }

}

system ("/home/bbclient/bb/bin/bb <big brother server ip here> \"status $flow.$test $colour $text\"");

As you can see this second script picks up the output file of the first script. It reads through it and checks if 1 que has more then 50 or 100 messages with the same status (RUNNING, WAITING, etc).

If so it will change the status colour to yellow or red. (Big brother will trigger alerts on this).

SAP RFC que monitoring in Big Brother Wietse Braam Business Card Posted on May. 13, 2008 12:50 AM in Perl, SAP Exchange Infrastructure (XI)

SAP RFC que monitoring in Big Brother
Wietse Braam
Business Card
Posted on May. 13, 2008 12:50 AM in Perl, SAP Exchange Infrastructure (XI)