I ran into an issue where every hyperlink in Solarwinds was working except license details. The issue turned out to be that in the recent past, the customer had renamed the Solarwinds server, and the old hostname never got removed from the allengines table. After removing the old entry in allengines, the license details screen was immediately accessible.

        -DaveB7114  

    Loop1 Systems: SolarWinds Training and Professional Services      

•                 LinkedIN: Loop1 Systems          
•                 Facebook: Loop1 Systems          
•                 Twitter: @Loop1Systems          

[ はじめに ]

予期せぬトラブルでも、Orionサービスを継続させる 「 フェールオーバーエンジン (FoE) 」 について記載します。

FoEは、Orionサービスの可能率を大幅に向上させるソフトウェアとなります。

Orion Failover Engine

http://www.solarwinds.com/extensions/failover.aspx

[ ステータス確認 ]

1.  SolarWinds Failover Manager を起動していただくことで確認できます。

　具体的には、SolarWinds Failover Manager 内、 Server Summary にてステータス確認が行えます。

　以下の画面キャプチャーは、セカンダリにフェイルオーバーされて継続的にOrionサービスが行われている例となります。

- Role を見ることで現在が稼働中か非稼働かの判断が可能です。　　Active になっていると稼働中。　Passive になっていると非稼働（スタンバイ）となります。

- 絵の矢印を見ていただくことでも判断可能かと存じます。

2.　各サーバー内のFoEステータスを直接見ることでも確認できます。　タスクバーにあるFoEを右クリックすることで以下の画面から把握できます。

[ FAQ ]

< Question >

- フェールオーバー後、セカンダリーからプライマリに自動で切り替わりますか？

< Answer >

- いいえ、自動で切り替わりません。　戻る場合は、手動で行う必要が御座います。

< Reference >

SolarWinds Failover Engine v6.7 Administrator's Guide

http://www.solarwinds.com/documentation/failoverengine/docs/OrionFailoverEngineAdministratorGuide.pdf

* P19-20 に記載ある Recovery from a Failoverをご一読いただければと存じます。

[ FoE 監視 ]

SAMにて、FoE監視が可能となります。

SAM: SolarWinds Failover Engine

http://www.solarwinds.com/documentation/en/flarehelp/SAM/default.htm#samagswfailoverengine.htm

[ 参考情報 ]

Failover Engine (FoE)について　（その１）

Failover Engine (FoE)について　（その2） + Q&A

【 Orion 製品 】

本ガイドは、機能評価を目的とした評価・トライアル環境の構築のためのガイドとなります。

サーバーのサイジングなど、パフォーマンス要件は考慮しておりません。予めご了承ください。

個別のご相談につきましては、営業担当者にご確認ください。

■ NPM (Network Performance Monitor / ネットワーク・パフォーマンス・モニター)

NPM 評価版のインストール

NPM: Network Sonar ウィザード (初期設定)

■ NTA (NetFlow Traffic Analyzer / ネットフロー・トラフィック・アナライザー)

NTA 評価版のダウンロード

NTA 評価版のインストール

■ NCM (Network Configuration Manager / ネットワーク・コンフィグレーション・マネージャー)

NCM 評価版のインストール

NCM 7.4 : HotFix 2 インストール方法

■ IPAM (IP Address Manager / IPアドレスマネージャー)

IPAM 評価版のインストール

IPAM 評価版のセットアップ

IPAM ver4.3のHotfix適用方法について

■ UDT (User Device Tracker / ユーザー・デバイス・トラッカー)

UDT 評価版のインストール

UDT 評価版のセットアップ

■ SAM (Server & Application Monitor / サーバー&アプリケーション・モニター)

SAM 評価版のインストール

SAM評価版のセットアップ

■ VMan (Virtualization Manager / バーチャライゼーション・マネージャー)

VMAN 評価版のインストール

VMAN評価版のセットアップ（その１）

VMAN評価版のセットアップ（その２）

VMAN 評価版【Hyper-V版】のインストール

NPM/SAM: Virtualization Manager(VMan)の統合プロセス

■ SRM (Storage Resource Monitor / ストレージ・リソース・モニター)

- 作成予定

■ WPM (Web Performance Monitor / ウェブ・パフォーマンス・モニター)

Web Performance Monitor (WPM)で、Webページを監視する手順を教えてください。

【 ソフトウェア / ハードウェア要件 】

Orion Server Software Requirements

Orion Server Hardware Requirements

Requirements for the Orion Database Server (SQL Server)

Requirements for Virtual Machines and Servers

【 全般 】

全般: OrionPlatform-v2015.1.2-HotFix5の適用方法

全般: デフォルトの電子メール送信先の設定

【 クイックビュー 】

グループ作成　クイックビュー

カスタムプロパティ　クイックビュー

Web-based アラート管理　クイックビュー

Web-based レポート　クイックビュー 1

Web-based レポート　クイックビュー 2

【 その他 】

Windows Server : SNMP 設定方法

Windows クライアント: SNMP 設定方法

[ はじめに ]

Orion 製品のトラブルシューティングを記載致します。

[ 背景 ]

トラブルが一つも発生しないソフトウェアやハードウェアは御座いません。

本文章は、Orion に何かしらの問題が発生した場合の対処方法について記載致します。　

[ 注意点 ]

原因や障害は環境によってさまざまです。　この手順通りにやって直らない場合は、

テクニカルサポートにSWIDと詳細状況をご連絡いただきます様お願い致します。

[ 本文章の見かた ]

順序どおりに行ってください。　直った場合、それ以上の手順は行わないでください。

[ 手順 ]

0.　"ノードの詳細" > "管理" にて、"今すぐポーリング" と "再検出" をクリックします。

1.　Orion製品全般： Orion サービスの停止/起動

2.　　Orion サーバーの再起動

3.　Orion製品全般： SolarWinds設定ウィザード

4. 　全般: インストールコンポーネントの修復

[ ポイント ]

- 各作業後は、2,3分待ってから確認してください。　（バックグラウンド処理にて一時的な負荷が高いため）

[ 復旧後 ]

- 最新版へアップグレードを行っていただきます様お願い致します。

- 複数製品が稼働している場合は、アップグレード順序がありますので以下KBにてご確認ください。

SolarWinds Knowledge Base :: Compatibility of SolarWinds Orion Products for Installation and Upgrade

- Orion プラットフォームのホットフィックスを適応してください。

全般: OrionPlatform-v2015.1.2-HotFix5の適用方法

[ 参考情報 ]

全般: SQLDBのバックアップ/リストア方法

SolarWinds Online Help - Troubleshooting

It took me a while to get the right combination of AIX SNMP Version 3 settings that will work with Authentication & Privacy enabled for Solar Winds.

Hopefully the steps below will help you get it working too. A list of sources which provided the clues I needed is at the end, its a long one as no single place had everything you will need. Hopefully this doco corrects that.

AIX Configuration

These steps worked ok on "AIX 6 TL08" and "AIX 7 TL 03".

• Install the snmp.crypto fileset to enable encryption (Obtained mine from AIX 6 Expansion Pack DVD 5765-G62 11/2012)

> lslpp -cl snmp.crypto

#Fileset:Level:PTF Id:State:Type:Description:EFIX Locked

/usr/lib/objrepos:snmp.crypto:6.1.2.0::COMMITTED:I:56-bit DES Encrypted SNMPV3 Support:

/etc/objrepos:snmp.crypto:6.1.2.0::COMMITTED:I:56-bit DES Encrypted SNMPV3 Support:

• Backup the /etc/rc.tcpip file as it's about to be modified
• Turn on encryption with the snmp switch command:

> /usr/sbin/snmpv3_ssw -e

This command will create symbolic links as required to enable/disable the encrypted/non-encrypted versions of snmpd & clsnmp

• Confirm and update /etc/rc.tcpip so that the following lines are no longer commented. Comment out dpid2 if it hasn't already been by the above.

---cut---

# Start up the Simple Network Management Protocol (SNMP) daemon

start /usr/sbin/snmpd "$src_running"

# Start up the hostmibd daemon

start /usr/sbin/hostmibd "$src_running"

# Start up the snmpmibd daemon

start /usr/sbin/snmpmibd "$src_running"

# Start up the aixmibd daemon

start /usr/sbin/aixmibd "$src_running"

---cut---

It looks like /usr/sbin/dpid2 functionality has been rolled into one of the above from at least AIX 6 TL08 onwards.

• Backup your existing snmp configuration files

---cut---

/etc/snmpdv3.conf

/etc/clsnmp.conf

/etc/snmpd.boots

/etc/snmpd.peers

---cut---

• Pick one of your servers /etc/snmpd.boots files and make that file uniform across all your servers. Here's an example of its contents:

---cut---

00000002000000000A454172 0000000082

---cut---

The first value is your EngineID, which can be something you made up, or one provided by the vendor. The second is the number of times snmp has been restarted. If you ensure this file is consistent across your AIX servers you can reuse your /etc/snmpdv3.conf file across them all. In turn, you can reuse the credentials when adding the nodes to Solarwinds. The auth/priv keys are married to the EngineID and won't work on another server if the EngineID is different there.

• Generate a new auth key with your local EngineID.

---cut---

pwtokey -e -u auth <auth password> $(cat /etc/snmpd.boots | cut -f2 -d' ')

Display of 16 byte HMAC-MD5 privKey:

5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0

Display of 16 byte HMAC-MD5 localized privKey:

axxxxxxxxxxxxxxxxxxxxxxxxxxxxxx5

---cut---

Make a note of the non-localized key value. E.g 5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0 of the two above. Also ensure you make a note of the passwords of course. You _must_ use the passwords when adding the node to Solarwinds. Attempts to use the key instead met with failure, likely due to the need for the EngineID to be paired up with the key somehow (context field didn't help).

• Generate a new priv key with your local EngineID. Again you're only interested in the non-localized key value. I'll use axxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7 below for this.

---cut---

pwtokey -e -u priv <priv password> $(cat /etc/snmpd.boots | cut -f2 -d' ')

---cut---

• Clear your command history if you're worried about maintaining the privacy of these keys (good habit but a touch paranoid!)

---cut---

> ~/.sh_history

---cut---

• Update the /etc/clsnmp.conf file so it has an entry for local snmp testing. I'm using swro aka Solar-Winds-Read-Only. I might dabble with read-write later and want them segregated. Plug in your freshly generated auth/priv keys.

---cut---

/etc/clsnmp.conf

#winSnmpName  targetAgent  admin  secName  password  context secLevel authProto  authKey                           privProto  privKey

#----------------------------------------------------------------------------------------------------------------------------------------------------------

swro          127.0.0.1    snmpv3 swro     -         -       AuthPriv HMAC-MD5   5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0  DES axxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7

---cut---

You can replace the first "swro" with the local server name if you like. In fact multiple duplicate lines with each server/IP (and consistant snmpd.boots & snmpdv3.conf files) will allow you to kick off clsnmp commands from this server to any other that has been updated with this process. Handy if you want to setup scripting to pull specific MIB/OID values etc from all servers etc.

• Update your snmpd.peers file to ensure it has the details required for the snmpd process to access other components (e.g hostmibd/snmpmibd etc) for specific MIB/OID resources.

---cut---

/etc/snmpd.peers

###############################################################################

#

"gated"     1.3.6.1.4.1.2.3.1.2.1.2     "gated_password"

"dpid2"     1.3.6.1.4.1.2.3.1.2.2.1.1.2 "dpid_password"

"muxatmd" 1.3.6.1.4.1.2.3.1.2.3.1.1 "muxatmd_password"

#

# Enables cpu & volume information visibility to snmpd

"xmtopas"       1.3.6.1.4.1.2.3.1.2.1.3 "xmtopas_pw"

#

## EOF

---cut---

The default AIX set of "passwords" is being used above (and in the following snmpdv3.conf) which should get you sorted. Sing out if you spot any issues with this approach as it depends on locking out non-local access to snmpd via the snmpdv3.conf file except for auth/priv key holders.

For example, I didn't have an entry here for xmtopas. Once I put that in place the SolarWinds discovered resources list suddenly included "Volume Utilization" values aka filesystem and logical volume info. Once selected they appear in the "Asset Inventory" tab under logical volumes.

• Update your /etc/snmpdv3.conf file with the one below. Swap out the auth/priv keys with the ones you generated above.

---cut---

##

## Solar Winds Specific Entries

##

#

USM_USER swro 00000002000000000A454172 HMAC-MD5 5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx0 DES axxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7 N -

#

VACM_GROUP swrogrp - swro readonly

#

VACM_VIEW swroview internet                    - included -

VACM_VIEW swroview 1.3.6.1.4.1.2               - included -

VACM_VIEW swroview 1.3.6.1.4.1.2.2             - included -

VACM_VIEW swroview 1.3.6.1.4.1.2.3             - included -

VACM_VIEW swroview 1.3.6.1.4.1.2.5             - included -

VACM_VIEW swroview 1.3.6.1.4.1.2.6             - included -

VACM_VIEW swroview directory                   - included -

VACM_VIEW swroview mgmt                        - included -

VACM_VIEW swroview mib-2                       - included -

VACM_VIEW swroview system                      - included -

VACM_VIEW swroview aix                         - included -

VACM_VIEW swroview 1.3.6.1.4                   - included -

VACM_VIEW swroview 1.3.6.1.6                   - included -

VACM_VIEW swroview 1.3.6.1.6.3.1.1.5           - included -

VACM_VIEW swroview 1.3.6.1.4.1.2021            - included -

VACM_VIEW swroview 1.3.6.1.4.1.2.3.1.2.2.2.1.4 - included -

#

# Include snmpv3 managed MIBs with this view

VACM_VIEW swroview snmpModules                 - included -

# Include aixmibd managed MIBS with this view

VACM_VIEW swroview 1.3.6.1.4.1.2.6.191         - included -

#

VACM_ACCESS swrogrp - - AuthPriv - swroview - sworoview -

##

## AIX Internal SNMP Agent Specific Entries

##

#

# Allow localhost(only) SNMPv1 general access

COMMUNITY public public noAuthNoPriv 127.0.0.1 255.255.255.255  -

VACM_GROUP group1 SNMPv1  public  -

VACM_ACCESS  group1 - - noAuthNoPriv SNMPv1   defaultView - defaultView -

#

VACM_VIEW defaultView internet                    - included -

# Exclude snmpv3 related MIBs from the default view

VACM_VIEW defaultView snmpModules                 - excluded -

VACM_VIEW defaultView 1.3.6.1.6.3.1.1.4           - included -

VACM_VIEW defaultView 1.3.6.1.6.3.1.1.5           - included -

# Exclude aixmibd managed MIBS from this view

VACM_VIEW defaultView 1.3.6.1.4.1.2.6.191         - excluded -

#

# Access to data from gated/muxatmd/xmservd/dpid

smux   1.3.6.1.4.1.2.3.1.2.1.2   gated_password  # gated

smux   1.3.6.1.4.1.2.3.1.2.3.1.1   muxatmd_password #muxatmd

smux   1.3.6.1.4.1.2.3.1.2.1.3   xmservd_pw   #xmservd

smux   1.3.6.1.4.1.2.3.1.2.2.1.1.2   dpid_password   #dpid

#

#

# These entries appear to be for IBM director at a guess

#  They allow it to participate with the above

#VACM_GROUP director_group SNMPv2c public -

#VACM_ACCESS director_group - - noAuthNoPriv SNMPv2c defaultView - defaultView -

#

# Trap definitions

NOTIFY notify1 traptag trap -

TARGET_ADDRESS Target1 UDP 127.0.0.1 traptag trapparms1 - - -

TARGET_PARAMETERS trapparms1 SNMPv1  SNMPv1  public  noAuthNoPriv -

##

## Global Defaults

##

#

# Set no access unless explicitly allowed by previous entries

DEFAULT_SECURITY no-access - -

#

# Set log location, maximum size, log level

logging         file=/usr/tmp/snmpdv3.log       enabled

#logging         size=100000                     level=0

logging         size=100000                     level=2

#

## EOF

---cut---

Still a work in progress locking down the AIX Internal SNMP agents and of course getting the right set of MIB included in the Solar Winds view. At least now I've something working I can fine tune and will be looking at other related posts here.

• Stop all snmp related services

---cut---

stopsrc -s snmpmibd;stopsrc -s aixmibd;stopsrc -s snmpd;stopsrc -s hostmibd;stopsrc -s dpid2

---cut---

• Start all snmp related servers (excluding the now redundant dpid2)

---cut---

startsrc -s snmpmibd;startsrc -s aixmibd;startsrc -s snmpd;startsrc -s hostmibd

---cut---

• Test things out locally by using the walk option on clsnmp (yup, IBM included a cleverly disguised snmpwalk command). I'm using the "internet" MIB in this example, lots of output!

---cut---

clsnmp -h swro walk internet

---cut---

If you get an error here, odds are you've a mismatched EngineID and auth/priv keys. Check out your /usr/tmp/snmpdv3.log for more details. As mentioned earlier the "swro" here is a reference to the matching line in /etc/clsnmp.conf. A server name could be used here (if defined there) instead and would result in a remote test.

• Once the dust settles, turn down the log level in snmpdv3.conf (level=0) to avoid excessive logging for daily operation.

Solar Winds Configuration

It should now be possible to add this node in Solarwinds.

Key Values

• SNMP Version: SNMPv3
• SNMPv3 Username: swro
• SNMPv3 Authentication Method: MD5
• SNMPv3 Authentication Password: Use the password from pwtokey above (don't use the key, it doesn't work)
• SNMPv3 Privacy Method: DES56
• SNMPv3 Privacy Password: Use the password from pwtokey above (don't use the key, it doesn't work)

Hit the test button to see if all is ok. If it isn't make sure the services have been started and check the snmpdv3.log for more clues (especially the "did solar winds get here at all" clue).

As I've standardized the /etc/snmpd.boot and /etc/snmpdv3.conf files across the estate I can re-use these credentials and have saved them as "aix-swro".

Update 09/11/2015 - Post AIX upgrade (TL08/SP02 -> TL09/SP04) - Repair links to encrypted binaries

After the upgrade the links to the alternate, encrypted binaries for SNMP were reset to the default non-encrypted ones. This wasn't immediately apparent on Solarwinds until you try to "List Resources" for the node, which fails.

Solarwinds error: "<node> is currently down, unreachable, or provided credentials are not valid"

• Login as root on the target server
• Attempt an snmp walk to confirm this is the same problem

> clsnmp -h swro walk internet 1>/dev/null

Error reading file /etc/clsnmp.conf(Line 46):    Invalid securityLevel

clsnmp: 1473-406 Error converting destinationName swro to Entity.

• Check the current snmp binaries being linked to

> ls -la /usr/sbin/snmpd /usr/sbin/clsnmp

lrwxrwxrwx 1 root system 9 Oct 14 12:02 /usr/sbin/snmpd -> snmpdv3ne

lrwxrwxrwx 1 root system 9 Oct 14 12:02 /usr/sbin/clsnmp -> clsnmpne

• Change the links to the encrypted snmpd binary with the command:

> snmpv3_ssw -e

In /etc/rc.tcpip file, comment out the line that contains: dpid2

In /etc/rc.tcpip file, remove the comment from the line that contains: snmpmibd

Stop daemon: snmpdMake the symbolic link from /usr/sbin/snmpd to /usr/sbin/snmpdv3e

Make the symbolic link from /usr/sbin/clsnmp to /usr/sbin/clsnmpe

Start daemon: snmpd

• Check the linked binary again

> ls -la /usr/sbin/snmpd /usr/sbin/clsnmpe

lrwxrwxrwx 1 root system 18 Nov 09 14:49 /usr/sbin/snmpd -> /usr/sbin/snmpdv3e

lrwxrwxrwx 1 root system 18 Nov 09 14:49 /usr/sbin/clsnmp -> /usr/sbin/clsnmpe

• On the solarwinds console, retry "list resources" on the node to confirm all is ok again

Sources

- IBM Doco

-- IBM SNMPv3 Documentation [http://www-01.ibm.com/support/knowledgecenter/api/content/ssw_aix_71/com.ibm.aix.networkcomm/snmpv3_intro.htm]

-- Creating users in SNMPv3 in AIX [http://www-01.ibm.com/support/knowledgecenter/api/content/ssw_aix_71/com.ibm.aix.networkcomm/HT_commadmn_create_snmpv3_user.htm#create_snmpv3_user]

-- AIX 7.1 snmpdv3.conf file setup [http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.files/snmpdv3.conf.htm?lang=en]

-- IBM SNMP key generation [http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.commadmn/doc/commadmndita/snmpv3_genkeys.htm]

-- Using the clsnmp command [http://www-01.ibm.com/support/knowledgecenter/ssw_aix_53/com.ibm.aix.cmds/doc/aixcmds1/clsnmp.htm?cp=ssw_aix_53%2F1-2-0-2-122&lang=en]

--- Useful to test an snmp configuration as this can interrogate your snmp server to confirm its working ok.

- Solar Winds Doco

-- Solarwinds SNMPV3 Implementation Guide [http://www.solarwinds.com/support/Orion/docs/Implementing_SNMPv3r1.pdf]

-- Managing SNMP credentials guidelines [http://www.solarwinds.com/netperfmon/solarwinds/wwhelp/wwhimpl/js/html/wwhelp.htm]

- Solar Winds Forums

-- Example SNMPv3 Configuration on AIX from Solarwinds forum [https://thwack.solarwinds.com/message/249258]

-- Help with SNMPv3.1 on AIX [http://thwack.solarwinds.com/thread/42695]

-- Using SNMPv3.1 on AIX [http://thwack.solarwinds.com/thread/36507]

-- Configuring 3rd party Net-SNMP [http://thwack.solarwinds.com/thread/19323]

-- AIX Specific mods [http://thwack.solarwinds.com/community/application-and-server_tht/server-and-application-monitor/content?filterID=content~category[application-monitor-templates]&filterID=content~objecttype~objecttype[document]&query=unix]

- Make AIX a Solarwinds client on snmpv3 [http://odme.blogspot.com.au/2012/09/make-aix-solarwinds-client-on-snmpv3.html]

- Configuring AIX snmpd for MIB subagent access [http://odme.blogspot.com.au/2012/09/snmpdv3-wont-talk-to-mib-subagents.html]

-- The aixmbid, snmpmibd and similar subsystems depend on snmp to talk to each other! Here's the settings they use to do so.

-- Very detailed setup of the AIX snmpd for MIB subagent config including extra mib defaultviews [http://forums.cacti.net/viewtopic.php?t=19040]

- Nagios setup for snmp [http://nagios.frank4dd.com/howto/aix-snmp-setup.htm]

-- Examples on using snmpwalk

-- Example snmpdv3.conf for Nagios with highlights for nagios specifics

- [http://lparbox.com/how-to/powerha-cluster/21]

- [http://www.mcpressonline.com/system-administration/techtip-monitoring-aix-with-snmp.html]

- Using snmpwalk [http://www.net-snmp.org/tutorial/tutorial-5/commands/snmpv3.html]

Simplify and automate these patches and more with SolarWinds Patch Manager.

Check out the attached spreadsheet for recent patches.

The Patch Manager catalog contains the following products:

Change History (current month + 1):

11/12

• Skype 7.14.85.104 (11/10)
•   SeaMonkey 2.39 (11/08)

11/11

• Adobe Flash Player Plugin 19.0.0.245 (11/10)
• Adobe Flash Player 18.0.0.261 ESR (11/10)
• Adobe  Air 19.0.0.241 (11/10)
• Google Chrome 46.0.2490.86 (11/10)

11/06

• Ultra VNC Server 1.2.0.9 (11/04)
• Ultra VNC Viewer 1.2.0.9 (11/04)

11/04

• Firefox 42.0 (11/03)
• Firefox ESR 38.4.0 (11/03)
• Opera 33.0.1990.55 (11/03)

10/30

• Notepad ++ 6.8.6 (10/29)
  

10/28

• Adobe Shockwave Player 12.2.1.171 (10/27)
• WinZip 20.0.11659 (10/27)
• Opera 33.0.1990.43 (10/27)

10/23

• iTunes 12.3.1.23 (10/21)
• AAS 4.1 (10/21)
• AMDS 9.1.0.6 (10/21)
• Google Chrome 46.0.2490.80 (10/22)
• Skype 7.13.85.101 (10/22)

10/21

• Java Runtime Environment 8u65 (10/20)
• Java Runtime Environment 8u66 (10/20)

10/20

• Notepad++ 6.8.4 (10/16)
• Notepad++ 6.8.5(10/19)

10/19

• Adobe Flash Player 19.0.0.226 (10/16)
• Adobe Flash Player 18.0.0.255 ESR (10/16)

10/14

• Adobe Flash Player 19.0.0.207 (10/13)
• Adobe Flash Player 18.0.0.252 ESR (10/13)
• Adobe  Air 19.0.0.213 (10/13)
• Google Chrome 46.0.2490.71 (10/13)

10/13

• Malwarebytes 2.2.0.1024 (10/12)

10/09

• Dameware Mini Remote Control 12.0 (10/06)
• Foxit Enterprise Reader 7.2.2.0929 (10/08)

10/05

• Skype 7.12.85.101 (10/01)

10/1

• Mozilla Firefox 41.0.1 (9/30)
• Mozilla Thunderbird 38.3.0 (9/30)

9/29

• Opera 32.0.1948.69 (9/29)

9/28

• Google Chrome 45.0.2454.101 (9/24)
• SeaMonkey 2.38 (9/26)

9/24

• Skype 7.11.85.102 (9/23)

9/23

• Google Chrome 45.0.2454.99 (9/21)
• Mozilla Firefox 41.0 (9/22)
• Mozilla Firefox ESR 38.3.0 (9/22)

Most businesses are not worried about being individually targeted by the latest and greatest zero-day malware or other “scary” cyber threats. But what you don’t know can hurt you. The most common security threats today tend to be crimes of opportunity that are often sourced from already known offenders. So, how do you find out about these threats? Enter the webcast, Threat Intelligence with SIEM. Join SolarWinds security experts in this webcast as they discuss: - What are the most common threats today? - Why threat feed intelligence is important? - What are some best practices for securing your environment from these threats? - How does threat intelligence strengthen SIEM?

Download a free, 30-day trial of Log & Event Manager >>

Since the release on NPM 11.5 we've been hard at working building the next round of exciting functionality and improvements in existing functionality.  I'm excited to share the following list of items we're working on:

• Updated UI Look and Feel - The UI is receiving some love this release, spanning across NPM, SAM, VMAN, and the other Orion based products.
• Deep Visibility into F5 load balancers - Covering things like connection counts by VIP and server. NPM F5 info  FEATURE REQUEST - F5 Load Balancers - Show Wide IP status, GTM Server List, etc - Case #448234
• Monitoring for Cisco Switch Stacks (3750, 3850, 2960S) - Covering things like stack master, stack ring health, enumerating members, tracking member changes, per member hardware health, and so on. Cisco Stack monitoring
• Syslog / Traps / search improvements - Search Enhancements
• Unmanage behavior improvements - Unmanage reason (like shutdown tracker in Windows 2008)Silence Alerts While Still MonitoringUnmanage reason (like shutdown tracker in Windows 2008)Show 'Unmanaged' durationSilence Alerts While Still Monitoring
• Manage Pollers improvements - map sysoid values to arbitrary text for node details properties. Unknown Devices: admin mapping of sysObjectID to Vendor and MachineType
• Enhancements to view restrictions
• QoE Transport-Layer Metrics

Ongoing Initiatives:

• Increased scalability per SolarWinds instance (target of 250k elements / instance)
• Improved performance and decreased resource load times via analysis with SolarWinds DPA
• Increased number of pollers possible per instance

Hi

The image above shows the trap i need to get into the Event list on Orion NPM.

"spSensorValue = 1" should be the Trigger Condition, and "spSensorValue = 0" is the Reset Condition.

I have tried to make a custom sql alert, but can't get it right.

Can someone help me on this one?

Thanks,

Simon

UPDATE:

Case is closed, this is the solution:

Trigger Condition:

INNER JOIN traps a

               ON a.nodeid = nodes.nodeid

INNER JOIN trapvarbinds b

               ON a.trapid = b.trapid

                  AND b.oidname = 'spSensorValue'

WHERE  a.traptype = 'HHMSAGENT-MIB:spUnknownStatus.301'

       AND a.datetime >= Dateadd(second, -15, Getdate())

       AND b.oidvalue = '1'

Reset Condition:

INNER JOIN traps a

               ON a.nodeid = nodes.nodeid

INNER JOIN trapvarbinds b

               ON a.trapid = b.trapid

                  AND b.oidname = 'spSensorValue'

WHERE  a.traptype = 'HHMSAGENT-MIB:spUnknownStatus.301'

       AND a.datetime >= Dateadd(second, -15, Getdate())

       AND b.oidvalue = '0'

Big thanks to RichardLetts.

when developing these I start with a database query that returns the set of nodes that match the criteria:

SELECTnodeid,

       traptype,

       oidvalue

FROM   trapsa

       INNERJOINtrapvarbindsb

               ONa.trapid=b.trapid

                  ANDb.oidname='spSensorValue'

WHERE  a.acknowledged=0

       ANDtraptype='HHMSAGENT-MIB:spUnknownStatus.301'

ORDER BY a.datetime

This is much faster at developing and testing that trying to use the alert manager to do this. Then convert this into the bit I paste into the 'alert' ; I like to put a time-bound on traps and allow then to be 'acknowledged'

       INNERJOIN trapsa

               ONa.nodeid=nodes.nodeid

       INNERJOINtrapvarbindsb

               ONa.trapid=b.trapid

                  ANDb.oidname='spSensorValue'

WHERE  a.acknowledged=0

       AND a.traptype='HHMSAGENT-MIB:spUnknownStatus.301'

       ANDa.datetime>=Dateadd(hour,-4,Getdate())

       ANDb.oidvalue='1'

       INNERJOIN trapsa

               ONa.nodeid=nodes.nodeid

       INNERJOINtrapvarbindsb

               ONa.trapid=b.trapid

                  ANDb.oidname='spSensorValue'

WHERE  a.traptype='HHMSAGENT-MIB:spUnknownStatus.301'

       ANDa.datetime>=Dateadd(hour,-4,Getdate())

       ANDb.oidvalue='0'

Hopefully that is enough to get you started in the right direction

[SOAPBOX]

     I have prefer putting the restrictions into the join criteria because they get evaluated how most people expect them to do when you have an outer join.

     I like putting the table restrictions in the join criteria when it makes sense, because if you ever need an outer join they work more like you expect them to

     Note: the Orion trap receiver incorrectly decoded the trap type. if that ever gets fixed you may have to update your clause.

[/SOAPBOX]

You are going to need some more complex SQL, because you have to check that the last trap received trap for the oidvalue:

       INNERJOINtrapsa

               ONa.nodeid=nodes.nodeid

                  ANDa.traptype='HHMSAGENT-MIB:spUnknownStatus.301'

       INNERJOINtrapvarbindsb

               ONa.trapid=b.trapid

                  ANDb.oidname='spSensorValue'

WHERE  a.acknowledged=0

       ANDa.datetime>=Dateadd(hour,-4,Getdate())

       ANDa.datetime=(SELECTMax(datetime)

                         FROM   trapst

                         WHERE  t.nodeid=a.nodeid

                                ANDT.traptype=

                                    'HHMSAGENT-MIB:spUnknownStatus.301')

       ANDb.oidvalue='1'

Thanks alot, Richard!

I did some customization of your examples, and now it works perfect for our use.

Trigger Condition:

INNER JOIN traps a

               ON a.nodeid = nodes.nodeid

INNER JOIN trapvarbinds b

               ON a.trapid = b.trapid

                  AND b.oidname = 'spSensorValue'

WHERE  a.traptype = 'HHMSAGENT-MIB:spUnknownStatus.301'

       AND a.datetime >= Dateadd(second, -15, Getdate())

       AND b.oidvalue = '1'

Reset Condition:

INNER JOIN traps a

               ON a.nodeid = nodes.nodeid

INNER JOIN trapvarbinds b

               ON a.trapid = b.trapid

                  AND b.oidname = 'spSensorValue'

WHERE  a.traptype = 'HHMSAGENT-MIB:spUnknownStatus.301'

       AND a.datetime >= Dateadd(second, -15, Getdate())

       AND b.oidvalue = '0'

This document was generated from the following discussion: Trap to Event List using Custom SQL Alert

Here is the calculation for the Disk / Volume / RAM /below and example.

What object IDs (OIDs) does Orion NPM poll for volume information? What types of volume information does Orion NPM poll?

http://knowledgebase.solarwinds.com/kb/questions/1194/What+object+IDs+%28OIDs%29+does+Orion+NPM+poll+for+volume+information%3F+What+types+of+volume+information+does+Orion+NPM+poll%3F

You can run the MIB Walk for the Node in Question in order to find out what values been returned by the Node for required OID's as below.

Here is MIB walk tool download Link

http://downloads.solarwinds.com/solarwinds/Release/PreRelease/SolarWindsSnmpWalk.zip

******** MIb Walk Result from the Node **********

.1.3.6.1.2.1.25.2.3.1.3.1 = String: "Physical RAM"

.1.3.6.1.2.1.25.2.3.1.3.2 = String: "Virtual Memory"

.1.3.6.1.2.1.25.2.3.1.3.3 = String: "/"

.1.3.6.1.2.1.25.2.3.1.3.4 = String: "/proc"

.1.3.6.1.2.1.25.2.3.1.3.5 = String: "/dev/pts"

.1.3.6.1.2.1.25.2.3.1.3.6 = String: "/proc/bus/usb"

.1.3.6.1.2.1.25.2.3.1.3.7 = String: "/partB"

.1.3.6.1.2.1.25.2.3.1.3.8 = String: "/common"

.1.3.6.1.2.1.25.2.3.1.3.9 = String: "/grub"

.1.3.6.1.2.1.25.2.3.1.3.10 = String: "/dev/shm"

.1.3.6.1.2.1.25.2.3.1.3.11 = String: "/spare"

.1.3.6.1.2.1.25.2.3.1.3.12 = String: "/proc/sys/fs/binfmt_misc"

.1.3.6.1.2.1.25.2.3.1.4.1 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.2 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.3 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.4 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.5 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.6 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.7 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.8 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.9 = INTEGER: 1024

.1.3.6.1.2.1.25.2.3.1.4.10 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.11 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.4.12 = INTEGER: 4096

.1.3.6.1.2.1.25.2.3.1.5.1 = INTEGER: 1021591

.1.3.6.1.2.1.25.2.3.1.5.2 = INTEGER: 512034

.1.3.6.1.2.1.25.2.3.1.5.3 = INTEGER: 4232703

.1.3.6.1.2.1.25.2.3.1.5.4 = INTEGER: 0

.1.3.6.1.2.1.25.2.3.1.5.5 = INTEGER: 0

.1.3.6.1.2.1.25.2.3.1.5.6 = INTEGER: 0

.1.3.6.1.2.1.25.2.3.1.5.7 = INTEGER: 4232699

.1.3.6.1.2.1.25.2.3.1.5.8 = INTEGER: 17502476

.1.3.6.1.2.1.25.2.3.1.5.9 = INTEGER: 252879

.1.3.6.1.2.1.25.2.3.1.5.10 = INTEGER: 510795

.1.3.6.1.2.1.25.2.3.1.5.11 = INTEGER: 8468466

.1.3.6.1.2.1.25.2.3.1.5.12 = INTEGER: 0

.1.3.6.1.2.1.25.2.3.1.6.1 = INTEGER: 447260

.1.3.6.1.2.1.25.2.3.1.6.2 = INTEGER: 52

.1.3.6.1.2.1.25.2.3.1.6.3 = INTEGER: 2837178

.1.3.6.1.2.1.25.2.3.1.6.4 = INTEGER: 0

.1.3.6.1.2.1.25.2.3.1.6.5 = INTEGER: 0

.1.3.6.1.2.1.25.2.3.1.6.6 = INTEGER: 0

.1.3.6.1.2.1.25.2.3.1.6.7 = INTEGER: 2736606

.1.3.6.1.2.1.25.2.3.1.6.8 = INTEGER: 7646375

.1.3.6.1.2.1.25.2.3.1.6.9 = INTEGER: 4303

.1.3.6.1.2.1.25.2.3.1.6.10 = INTEGER: 5422

.1.3.6.1.2.1.25.2.3.1.6.11 = INTEGER: 8256

****************** Physical Disks ******************

hrStorageDescr = .1.3.6.1.2.1.25.2.3.1.3.3 = String: "/"

hrStorageAllocationUnits=   .1.3.6.1.2.1.25.2.3.1.4.3 = INTEGER: 4096

hrStorageSize .1.3.6.1.2.1.25.2.3.1.5.3 = INTEGER: 4232703

hrStorageUsed = .1.3.6.1.2.1.25.2.3.1.6.3 = INTEGER: 2837178

Calculations :

Volume size       4232703 x 4096  17337151488 Bytes   16.146 GB

Volume used       2837178 x 4096  11621081088 Bytes   10.822 GB

space available   17337151488 - 11621081088 = 5716070400 5.323 GB .

Percentage        10.8/16.14 * 100 65%

****************************************************

hrStorageDescr = .1.3.6.1.2.1.25.2.3.1.3.1 = String: "Physical RAM"

hrStorageAllocationUnits= .1.3.6.1.2.1.25.2.3.1.4.1 = INTEGER: 4096

hrStorageSize    .1.3.6.1.2.1.25.2.3.1.5.1 = INTEGER: 1021591

hrStorageUsed = .1.3.6.1.2.1.25.2.3.1.6.1 = INTEGER: 447260

Calculations : Physical RAM

Volume size       1021591 x 4096  4184436736 Bytes       3.897 GB

Volume used       447260 x 4096   1831976960 Bytes       1.706 GB

space available   1831976960 - 4184436736 = 2352459776     2.1 GB

Percentage        1.706/3.897 * 100 43%

*********************************************************

hrStorageDescr = .1.3.6.1.2.1.25.2.3.1.3.2 = String: "Virtual Memory"

hrStorageAllocationUnits= .1.3.6.1.2.1.25.2.3.1.4.2 = INTEGER: 4096

hrStorageSize    .1.3.6.1.2.1.25.2.3.1.5.2 = INTEGER: 512034

hrStorageUsed = .1.3.6.1.2.1.25.2.3.1.6.2 = INTEGER: 52

Calculations : Virtual Memory

Volume size       512034 x 4096  2097291264 Bytes   1.953 GB

Volume used       52 x 4096 212992 Bytes           0.0001 GB

space available   212992 - 2097291264 = 2097078272  1.953 GB .

Percentage 0.0001/1.953 * 100 0.0%

///////////////////////////////////////////////////////////////

Adding Few more explanation here for Cisco Call Manager Showing 99% Memory Utilization in Orion .( just in case someone having this issue )

The reason you are seeing 99% is due to the Cisco MCS returning back those values.

Why its doing this, you have to Open Cisco TAC.

NPM is returning back the values its being supplied with, in this case showing 99%.

Poller type: Used .

1. N.Memory.SNMP.CpqHostPhysicalMemory =

Memory poller for OpenVMS systems which support proprietary CPQ-Host MIB

Poller utilizes GetSubtree request with two OIDs to gather total and used memory:

            cpqHoPhysicalMemorySize = 1.3.6.1.4.1.232.11.2.13.1.

            cpqHoPhysicalMemoryFree = 1.3.6.1.4.1.232.11.2.13.2.

Total memory is computed using this formula:

            memoryTotal = cpqHoPhysicalMemorySize * 1024

Used memory is computed using this formula:

            memoryUsed = memoryTotal - (cpqHoPhysicalMemoryFree * 1024)

MIB Walk shows:

            .1.3.6.1.4.1.232.11.2.13.1.0 = INTEGER: 3990 = cpqHoPhysicalMemorySize

            .1.3.6.1.4.1.232.11.2.13.2.0 = INTEGER: 60 = cpqHoPhysicalMemoryFree

memoryTotal = 3990 * 1024 = 4085760

memoryUsed = 4085760 - (60 * 1024) = 4085760 – 61440 = 4024320

PercentUsed = 4024320/4085760 = 98.496% = 99%

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

For Linux Server for Poller assigned N.Memory.SNMP.NetSnmpReal

If poller assigned

N.Memory.SNMP.NetSnmpReal

/// memTotalReal = 1.3.6.1.4.1.2021.4.5.
/// memAvailReal = 1.3.6.1.4.1.2021.4.6.
/// memCached = 1.3.6.1.4.1.2021.4.15.
/// memBuffer = 1.3.6.1.4.1.2021.4.14.
/// UsedMemory = (totalMem - availMem - cachedMem - bufferMem) * 1024

.1.3.6.1.4.1.2021.4.5.0 = INTEGER: 2073620 (Total )

.1.3.6.1.4.1.2021.4.6.0 = INTEGER: 1869968 (Available )

.1.3.6.1.4.1.2021.4.15.0 = INTEGER: 88008 (cachedMemory )

.1.3.6.1.4.1.2021.4.14.0 = INTEGER: 51868 (BufferMemory )

( 2073620– 1869968– 88008– 51868 ) = 63776  

63776 * 1024 = 65306624 bytes

********************

Vmware ESX Memory Calculation

Supported Poller

The "Memory" Usage and the "Physical Memory" Utilization are calculated differently by Orion NPM.

Memory:

The Memory is the sum of the memory used by the processes listed in the HOST-RESOURCES-MIB:hrSWRunPerfTable (1.3.6.1.2.1.25.5.1).

The OID used is the hrSWRunPerfMem (1.3.6.1.2.1.25.5.1.1.2).

Physical Memory:

The Physical Memory is calculated based on the values populated in the HOST-RESOURCES-MIB:hrStorageTable (1.3.6.1.2.1.25.2.2.3).

The OIDs used are hrStorageSize (1.3.6.1.2.1.25.2.3.1.5), hrStorageUsed (1.3.6.1.2.1.25.2.3.1.6) and hrStorageAllocationUnits (1.3.6.1.2.1.25.2.3.1.4) .

Calculation :-

Used: .1.3.6.1.2.1.25.5.1.1.2

Free:  .1.3.6.1.2.1.25.5.1.1.1

Total. 5933728 + 7325457 = 13259185

% Used : 5933728 % 13259185 = 44.751 %