Blogs

How to use BWA python standalone tool
Jens Gleichmann 
Business Card
Company: Brose Fahrzeugteile GmbH & Co. Kommanditgesellschaft
Posted on Nov. 23, 2009 02:30 PM in BI Accelerator, Enterprise Data Warehousing/Business Warehouse

URL: www.sdn.sap.com/irj/sdn/bwa

Subscribe.Subscribe
Print. Print
Permalink Permalink
Share

Hi everybody,

in the past there were many questions about the standalone tool TREXAdmin.sh on the BWA , but there is no detailed documentation or 'how to' from sap and I will answer a few questions regarding this helpful little tool and list some general hints.

At first I will answer a very rudimental question:

Q: There are a lot of tools/transactions (TREXADMIN, RSDDBIAMON2) on ABAP site, for what I need this tool?

A: You get more information and you have more possibilities as on the ABAP site. For example you get a load analyze over your BWA blades or you can use the statistic index (described in note 1163732), but this are only two advantages of many of them which this tool includes.

 

I don´t want to go into deep on every topic, the most stuff is self-explanatory:

  1. How to start the tool
  2. Tab: Queue
  3. Tab: Index
  4. Tab: Backup
  5. Tab: Landscape
  6. BWA Performance Tuning
  7. Tools and interesting BW tables
  8. Summary

Before I begin please note that I´m using Rev.49 so it´s possible if you have another revision that your tabs can differs from my ones.

All this tabs are already documented shortly by SAP, it´s more a summary of my own view ;)

You will find this short TREX related SAP doc here.

 

1. How to start the tool

every administrator should know this if he have read the Technical Operations Manual for SAP NetWeaver BW Accelerator 7.0. If not, here a short summary:

Prerequisites
On UNIX/Linux: The TREX admin tool has a graphical interface, therefore you need an X-server. You cannot use a terminal program that only supports text mode, such as telnet.


Procedure
1. Log on with the user <sapsid>adm.
2. Perform the following steps:
Operating System Procedure
Linux Enter the following:
cd <TREX_DIR>
./TREXAdmin.sh
<TREX_DIR> = /usr/sap/<SAPSID>/TRX<instance_number>

If you have done all this thing correct you get a new X-Window.

 

2. Queue

I can´t say so much about this tab because I don´t use it with the BWA, only with other TREX instances. If you are interested in it anyway, please refer to this link.

3. Index

This tab is more interesting. Here you can see detailed information about your indices for example:number of documents, on which server the index is located, the usage of the index and reporting on BWA statistics indices (note: 1163732; you need Rev.50)

One thing which is also mentionable is the split/merge option when you do a right-click on an index, but be careful if you reindex, your action has no effect any more. 

Check here your sizing regarding 1348190. Your index data shouldn´t consume more than 50% of your available memory (check note 1093719).

 

 

4. Backup

Backup is a topic which wasn´t very interesting for me because our environment (4 blades with 16GB per blade) isn´t appropriate for backup. Backup & Recovery scenarios are only reasonable, if the environment is so big that reindexing takes longer than a recovery, but this must be communicate with SAP (note: 1077439). Reindexing takes about 20-25min on my blades, so there isn´t a reason for backup ;)

All backup features are implemented you only must set some parameters to activate them, this information you will get from SAP when you are in touch with them on this topic.

 

5. Landscape

This is the topic which I will the most of the time because it´s for me the most important one.

5.1 Alert

When you click on 'Alert', you wil get the same view as in the transaction TREXADMIN. But if you click on 'Alert Server Configuration' in the left upper corner you can configure your mail reciptents for the alert which you can define in the box thereunder.

I can tell you our settings which was recommended to us by our hardware vendor together with SAP:

Is part
Check_name
Parameters
 X check_topology_index_garbage  auto_repair=yes
 X cpu_usage warning=70%,error=80%
 X global_dict_fill_level warning=80%,error=90%
 X index_status 
 X memory_usage warning=75%,error=80%
 X network 
 X rfc_check 
 X share_memory_use  
 X trace_entries 
 X trace_file_size error_alertfiles=20MB,warning=50MB,warning_alertfiles=10MB,error=2000MB
 X unassigned_master_indexes 
 X version_info 

 

You can activate more parameter if you want but to avoid to many not significant mails this are good settings.

Exact explaination for all parameter you will find in Technical Operations Manual for SAP NetWeaver BW Accelerator 7.0.

 5.2 Configuration

The person who installed your 'blackbox' BWA, which should be a support guy from your hardware vendor, asked you on which backup mode you want to run your BWA.

There are 4 possibilities:

  • shared (one backup for all masters)
  • dedicated (one backup for each master)
  • multiplexed (one backup for several masters)
  • mutual (each server is backup for another master)
image

 

 

I use mode mutual, on all other solutions you must spend one or more blades for backup which are idling (note that currently mutual isn´t recomended by sap  in cause of bugs in all versions!). Please check note 1383228). If one blade fails, the backup blade, which is also a master, will take over the parts. You will configurate this in the box below.

 

5.3 Reorg

We have a scheduled reorg every day on a time where no load is on the system (report: RSDDTREX_REORGANIZE_LANDSCAPE).

I think the interactive Reorg can only really used if you are using mutual mode, because here you can interactive switch the indexes between each blade. This is only recommended when you know what you are doing ;)

image

 

The screenshot shows us the current distributed indexes over all blades in a short time before a reorg will start. The paly parts above the 100% are backup parts for each blade when it is in mutual mode. 

 

Another interesting tab is the last one called 'Options'. For the recommended options please refer to note 1163149, because every system differs.

 

5.4 Ini

Use this if you want e.g. change your TREXIndexServer. When you do this on OS you must change this setting for every blade, in the tool you can spread it with one click to all of you blades. When you must edit this ini files is e.g. performance tuning which I explain here later in the blog or special configuration by SAP.

If you have a new config, I recommend to restart all services to ensure sychronism of the services.

 

5.5 Hosts

On tab Storage  you can check you storage on disk, so you can see your free space and the read/write rating.

On tab Network you can check the performance of your network between each blade, the result should be green in all cases (speed on the host itself can be disregarded).

 

5.6 Services

One of the most tabs I´m using is this one. 

Here (MC) you can stop the whole TREX system or each host apart.

On tab 'Load' you can monitor your BWA performance in a line graph (memory. network, CPU etc.). 

Please note that the data are only kept for 3 days, if you want a continuously history I recommend to save data like memory, network or CPU usage from OS via snmp e.g. with a tool like cacti. So you can indentify trends on your landscape.

image

 

5.7 Trace

The trace files which you found also directly on OS ( /usr/sap/<SID>/TRX<instance>/<host>/trace ) are located under 'Trace'. Here you can sort easier as on OS over all hosts.

5.8 Performance Trace

Under Perf.Trace you can make a trace for a special user, I also recommend to use this together with the transaction RSRT (option execute + debug).

You will see where most of the time is lost and can do improvements in system tuning on the right system ;)

 

6. Performance Tuning

 There are some different kinds of tuning:

●      Optimizing Data Distribution by Reorganizing BIA Landscapes 

●      Enabling 'FEMS Compression'  

●      Enabling 'Package Wise Read'  

●      Enabling Parallel Indexing 

 

7. Tools and interesting BW tables

7.1 Tools

If you want a scheduled restart of your BIA landscape from OS you can use a script to start and stop the services:

for i in host1 host2 host3 host4

       do
                ssh $i "su - <sidadm> -c stopsap"
       done

simple but effective, but be sure that the ssh keys are correct on every host ;)

For starting the services copy all lines and replace 'stopsap' with 'startsap'.

 

7.2 Cacti

You can save histories for years with this third party tool. Here you can see a load average over 3 hours, but you can also expand this over weeks or even years, also for other details like memory, network etc..

image

 

7.3 interesting BI tables

Table Description
RSDDKF
Derived key figures
RSDDSTATLOGGING                Logging Table for OLAP Statistics (new)
RSDDSTATTREX                   Statistic Data for Process for TREX Aggregates
RSDDSTATTREX_MM
TREX Statistic Table for DB Buffer
RSDDTREXENQUE                  Lock Table for HPA Aggregates
RSDDTZA_ACT_LOG
Logging of BIA Admin Actions
RSDDTZA_LOCK

Lock Table for BIA Actions

RSDDTREXEMAILE-Mail-Addresses = Recipient for Canceled HPA Queries
RSDDTREXADMIN
Additional Settings for Indexing an HPA Index
RSDDTREXHPAFAILError Recording for Queries on HPA
RSRREPDIRDirectory of all reports
RSDDSTAT_DMData Manager Part of the Statistic

 

A real important table is RSDDTREXEMAIL, if your BWA fails (-> you have an entry in  table RSDDTREXHPAFAIL) and all queries run against your DB, you want to know this immediatly or? You can enter e-mail adresses in this table, don´t seperate the entries with comma or semicolon. Add a row for each address and you will get an email every time the BWA is not reachable.

Also a very interesting table is  RSRREPDIR, please refer to note 940635:

"A user can deactivate the BI accelerator manually for an InfoCube with a single click in transaction RSDDV. If necessary, a suitably authorized user can deactivate the BI accelerator for a particular query in transaction SE16 by setting a flag in table RSRREPDIR. The approach described here is the stand as of NetWeaver 2004s Support Package 07."

Another nice table for your analyzes is RSDDSTAT_DM with field CALDAY you can select on which day a query hit the BWA.

 

8. Summary

It should only be a summary of BWA stuff on a technical view. Many people don´t find the notes or the right documentations. I hope I could help you to get a little bit more deeper in the topics above.

If you have helpful suggestions or improvements please give me feedback!

 

Also some useful links and notes:

1139280 - BIA 7.00: Optimal settings for quad core cpu's

1260249 - BWA 7.00/7.20: List of Error Codes

1383228 - BWA 7.00: Use of backup mode "mutual"

1158597 - BIA 7.00: Improving indexing performance

1157582 - BIA 7.00: New feature "package wise read"

1161395 - Parallelization of the data indexing in the BIA

1093719 - BIA capacity utilization: Preventing overloading

1095886 - Checking the data consistency in BI Accelerator

970771  -  BI accelerator monitor messages to CCMS

1028556 - BIA: incorrect data after compressing/deleting 

1018798 - Reading large data volumes from BI accelerator 

1139280 - BIA 7.00: Optimal settings for query performance 

1133742 - BIA 7.00: DimFunction Index (DFI)

1002839 - BIA 7.00: Error due to huge resultset of query  

1223239 - Indexing processes lock each other - increase wait time  

940635  - BIA&High Availibility: DB fallback for BIA  

1077782 - BIA 7.00: Exception for Query using ICM  

817145 - Composite SAP note/overview: TREX troubleshooting  

879305 - TREX 7.0: List of Error Codes  

 

important note if you need support from SAP:

1058533 - TREX/BIA service connection to customer systems  

 

other links

Cacti website:

http://www.cacti.net

BIA blog series

 

 

Edit History:

edited cacti description and added a link for it

added table RSDDSTAT_DM

 

Jens Gleichmann   SAP Basis administrator


Comment on this articleDo you use this tool often or is it completely new for you?
Comment on this weblog
Showing messages 1 through 8 of 8.

Titles Only Main Topics Oldest First

  • TREXAdmin Load Data Information
    2010-02-15 09:59:03 Dave Marshall Business Card [Reply]

    As an BIA administrator I have been asked to give view only access to the Load Graphs for BI developers ,is such a thing possible as the developers are not granted OS level access.
    If there is another way of getting the Load Data so that the BI support teams can view real time the impact of the end user reports , data loads etc .
    • TREXAdmin Load Data Information
      2010-03-01 23:21:03 Jens Gleichmann Business Card [Reply]

      In my opinion this is not a tool for developers, because there are some options in this tool which are to dangerous for example you get an OS command console File->OS Command Console.
      But this tool is based on python so if you invistigate some time you will find the function to export the current load data and can schedule this by a job scheduler and send this data via mail.


      --> This was also interesting for me so I found the function for you:
      -> go as sidadm to your python_support folder (e.g. /<gpfs>/<SID>/exe/nuc/linuxx86_64/python_support) and execute this:
      ./serviceLoad.py --export=<path>
      e.g.: ./serviceLoad.py --export=/tmp/test.service.load


      exactly usage of the commands:
      Usage: serviceLoad.py [options]


      Options:
      --history=0|1 (0=actual values, 1=historical values; default=0)
      --allHosts=0|1 (0=current host only ;default=1)


      Options for --history=1:
      --timeStamp=timestamp for the begin of the history (default=0)
      (0=all,-1=latest, <-1=last t seconds, else=since timestamp:
      seconds like time_t or YYYYmmddHHMMSS or YYY-mm-dd HH:MM:SS)
      --export=<filename|1> (export history like Admintool->Services->Load to file|stdout)
      --wily=0|1 print latest history value in wily format


      so just create a script that execute this command and trigger this by a job scheduler and send this file via mail to your developers or create the script every hour with the crontab and give samba access to this folder (or send it via email).


      how to use the output follow the steps which are provided in the standalone tool (import in Excel): service->load-> dropdown button near export -> help


      Regards,
      Jens

  • RSDDTREXENQUE
    2010-02-05 10:45:29 Vitaliy Rudnytskiy Business Card [Reply]

    Hi Jens,


    First of all: great summary! thanks for sharing this.


    The question I have is if you get a chance to find how to analyze usage of table RSDDTREXENQUE that is by design not available for display?


    Cheers,
    -Vitaliy

    • RSDDTREXENQUE
      2010-09-23 10:56:24 Marc Bernard SAP Employee Business Card [Reply]

      Vitaliy,


      as far as I can tell this table is not used. Locking for BWA is implemented by other means.


      Regards,
      Marc
      SAP Techology RIG

    • RSDDTREXENQUE
      2010-03-01 23:29:23 Jens Gleichmann Business Card [Reply]

      Hi Vitaliy,


      I never have to use this table, the only possibility I found was to select it directly on DB level.
      I donīt know another function or transaction which is using data from this table.


      I will ask around and inform you if I found usable information for you.


      Regards,
      Jens

  • using caci to assist monitoring
    2009-11-24 02:01:28 Sarah Plunkett Business Card [Reply]

    could you give more details about using caci to analyse the data collect from the TREXAdmin standalone tool ?
    • using cacti to assist monitoring
      2009-11-24 02:29:08 Jens Gleichmann Business Card [Reply]

      cacti donīt collects data from the standalone tool! cacti is a third party tool for collecting data from OS via snmp. I only use it for a continuously history for memory(+swap space), cpu, network and disk space. Please go to http://www.cacti.net/ to get more information about it. There are several tools besides cacti for collecting snmp data.


      You donīt get details like index read/write/merge time/requests or RFC-Server memory. Itīs more a general info about each host, but I think thatīs enough information for identifying a trend.
      If you think thatīs not enough you can export the data from trexadmin into excel ->services->load-> Export/Import


Showing messages 1 through 8 of 8.