IBM DataPower off-device logging: a configuration example - United States datapower; 7993; machine type 7993; logging; off device; off device logging; log target; off-device; syslog; debug log level; debug; log level TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 This technote gives an example of how to enable off-device logging on an IBM® WebSphere® DataPower® appliance. This can be very helpful when a debug log level is needed to help isolate a problem or monitor behavior over a long period of time or can be used in production environment as DataPower only keeps a limited number of log files (the default is 3 files) in the file system in a rotational basis.

CAUSE
The DataPower device has a finite amount of space to hold larger than average log files or long term logging needs in production environment.

It should be noted that if used while debugging at load or in a capacity issue, log events may be dropped. There is a prioritization of events within the device and client traffic always comes first. Log events dropped can be confirmed or counted from the Status>Log Targets menu in the WebGUI.

RESOLVING THE PROBLEM
Configure a server daemon to listen and capture the DataPower devices events. First, this example shows how to create the log target sending events to a syslog destination. The other examples listed below will have their own strong points to be considered depending on the problem scenario.

Syslog:

* The syslog protocol operates over UDP which has no guarantee on packet delivery - a fast fire and forget method.

To create the new log target, go into the default domain: Objects > Log Targets

Configure the log target with settings as follows from the WebGUI:

 1. Name the log target
 2. Select Target Type of syslog
 3. Fill in the Local Identifier with a descriptive string that may be used by a remote recipient to identify this specific log target
 4. Enter the Remote Host Address and the Remote IP Port as in the screen shot wherex.x.x.x is the IP address of the remote syslog server that listens on port 514
 5. Take all other defaults

[/support/docview.wss?uid=swg21269136&aid=1] [/support/docview.wss?uid=swg21269136&aid=1]

Under the Event Subscriptions tab, you can select all and debug as indicated here:

[/support/docview.wss?uid=swg21269136&aid=2] [/support/docview.wss?uid=swg21269136&aid=2]

 *
 * 6. Generate log events in the DataPower by using some transactions, for example by saving the configuration from the WebGUI or running some test load into a domain.

Syslog-ng, will simply work over the TCP protocol to ensure the packet is delivered to the destination. The same consideration should be applied to a device under high load in that a log event could be dropped if not able to be wrote out to the network within time.

NFS:
All NFS limitations apply from file size and permission settings. This is a more common approach using an NFS static mount to capture a log target and can allow for slightly more reliable messages to be logged. The limiting factor again is the speed of the network and NFS server response time.

File type:

Using a file type log target with a backup method as seen here:
[/support/docview.wss?uid=swg21269136&aid=3] [/support/docview.wss?uid=swg21269136&aid=3]
This will allow log events to quickly be written to a local file on the device's file system. Once the file reaches its set size a connection to the destination will upload the file from the device. This will upload with a unique time and date stamp on the uploaded file.

This is a useful method to capture sporadic problems. This is also useful for long running transactions that may span more than one file depending on device load.

HTTP service:
Finally a clever method that may be used in some senarios would be an HTTP service on the device.
Using an HTTP service configured in the following way,
[/support/docview.wss?uid=swg21269136&aid=4] [/support/docview.wss?uid=swg21269136&aid=4]

Once the file type log is created in the logtemp:/// directory, a client browser or wget type client can easily retrieve the file from the device.

This is very useful when there is no local or accessible remote log storage location to the device, due to firewall or network restrictions.

This is also removing the limitation of the device dropping the log event should it not be able to write the log event to the network.

Each of the above methods are very useful and work better in some scenarios than others depending on the network, load, problem, and information needed. This is intended to be a guide to help you decide which method would be best for your scenario.

Question: Limitations of sylog-TCP log target using DataPower Gateways I am trying to configure a log target using type syslog-TCP. It seems that the payload is getting truncated when I try to record custom logs to it using log-action. Why is this? How can I workaround it.

Answer: Configure a server daemon to listen and capture the DataPower devices events.


IBM Java Health Center Client -  a low overhead monitoring tool - United States  TECHNOTE (FAQ)

QUESTION
 How can I monitor and diagnose my WebSphere Application Server environment without impacting performance?

ANSWER
Also see: Extracting data from Java Health Center [http://www.ibm.com/support/docview.wss?uid=swg21423006]

 *  Java™ Health Center:
   Health Center is a very low overhead monitoring tool. It runs alongside an IBM Java application with a very small impact on the application's performance (less than 1%). Health Center monitors several application areas, using the information to provide recommendations and analysis that help you improve the performance and efficiency of your application. Health Center can save the data obtained from monitoring an application and load it again for analysis at a later date.

   Starting with IBM Java 5 SR8 or IBM Java 6 SR1, The Health Center client can be installed within the IBM Support Assistant Team Server [http://www.ibm.com/software/support/isa/teamserver.html].

   The Health Center client has several different views, each reporting data regarding a different aspect of the JVM. The views available are:

    *  Classes: Information about classes being loaded CPU: Processor usage for the application and the system on which it is running Environment: Details of the configuration and system of the monitored application Garbage Collection: Information about the Java heap and pause times I/O: Information about I/O activities that take place Locking: Information about contention on inflated locks Method Trace: Information about method use over time Native Memory: Information about the native memory usage Profiling: Provides a sampling profile of Java methods including call paths Threads: Information about the live threads of the monitored JVM WebSphere® Real Time for Linux: Information about real-time applications

   To Install Health Center:
   Health Center Client is installed as a plug-in for IBM Support Assistant (ISA), therefore, ISA must be installed prior to obtaining the Java Health Center.

    *
    * IBM How to Install and Update Problem Determination Tools for IBM Support Assistant Team Server
    *
    *
    * Generic JVM Argument
    *
    *
    *
    *
      (WebSphere) Java Version Generic JVM Argument Java 7.0 and later
      Java 6.0 SR5 and later
      Java 5.0 SR10 and later -Xhealthcenter Java 6.0 SR1 through SR4
      Java 5.0 SR8 and SR9 -agentlib:healthcenter -Xtrace:output=perfmon.out

 *   *
    * Connecting to an application by using the Health Center client
    *
    *
    *
    * BEST PRACTICE:
    *
    *
    * Platform Requirements
    *
    *
    *
    * Note
    *
    * inside
    *
    *
    *  Start the Health Center client, open the help documentation, then navigate to IBM Support Assistant>Problem Determination Tools>IBM Monitoring and Diagnostic Tools - Health Center>Monitoring a running application>Installing Health Center and install the agent from there.
    *

RELATED INFORMATION
 IBM Monitoring and Diagnostic Tools for Java - Health C [http://www.ibm.com/developerworks/java/jdk/tools/healthcenter]
HC 01 Overview of Health Center features [http://www.youtube.com/watch?v=5Tcktcl0qxs]
IBM Monitoring and Diagnostic Tools for Java - Getting [http://www.ibm.com/developerworks/java/jdk/tools/healthcenter/getting_started.html]
IBM Support Assistant [http://www.ibm.com/software/support/isa/]

Question:
Profiler for WebSphere 8

Hi,

We noticed that a was application server uses 1,4 Go of memory at startup, we would like to understand what java classes are using most part of it.

Is a Java profiler provided by default with WAS 8.0 ? Is there something to activate via the WAS admin console ?

Thanks a lot

Answer: Health Center is a very low overhead monitoring tool. It runs alongside an IBM Java application with a very small impact on the application's performance (less than 1%). Health Center monitors several application areas, using the information to provide recommendations and analysis that help you improve the performance and efficiency of your application.


IBM Security Bulletin: API Connect is affected by a Node.js denial of service vulnerability (CVE-2017-14919) - United States  SECURITY BULLETIN

SUMMARY
 IBM API Connect has addressed the following vulnerability.

Node.js is vulnerable to a denial of service, caused by an uncaught exception flaw in the zlib module. By making 8 an invalid value for the windowBits parameter, a remote attacker could exploit this vulnerability to cause the application to crash.

VULNERABILITY DETAILS
CVEID:CVE-2017-14919 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-14919]
DESCRIPTION:Node.js is vulnerable to a denial of service, caused by an uncaught exception flaw in the zlib module. By making 8 an invalid value for the windowBits parameter, a remote attacker could exploit this vulnerability to cause the application to crash.
CVSS Base Score: 7.5
CVSS Temporal Score: Seehttps://exchange.xforce.ibmcloud.com/vulnerabilities/134286 [https://exchange.xforce.ibmcloud.com/vulnerabilities/134286]for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H)

AFFECTED PRODUCTS AND VERSIONS

Affected API Connect Affected Versions IBM API Connect 5.0.0.0-5.0.6.4 IBM API Connect 5.0.7.0-5.0.7.2 IBM API Connect 5.0.8.0-5.0.8.1 REMEDIATION/FIXES

Affected Product Addressed in VRMF APAR Remediation/First Fix IBM API Connect

V5.0.0.0 - 5.0.6.4 5.0.6.5 LI79994 Addressed in IBM API Connect V5.0.6.5.

Follow this link and find the "APIConnect-Portal" package.

https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm%7EWebSphere&product=ibm/WebSphere/IBM+API+Connect&release=5.0.6.4&platform=All&function=all [https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm%7EWebSphere&product=ibm/WebSphere/IBM+API+Connect&release=5.0.6.2&platform=All&function=all] IBM API Connect

V5.0.7.0 - 5.0.7.2 5.0.8.2 LI79994 Addressed in IBM API Connect V5.0.8.2.

Follow this link and find the "APIConnect_Management" and "APIConnect-Portal" packages.

https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm%7EWebSphere&product=ibm/WebSphere/IBM+API+Connect&release=5.0.8.1&platform=All&function=all [https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm%7EWebSphere&product=ibm/WebSphere/IBM+API+Connect&release=5.0.6.2&platform=All&function=all] IBM API Connect

V5.0.8.0 - 5.0.8.1 5.0.8.2 LI79994 Addressed in IBM API Connect V5.0.8.2.

Follow this link and find the "APIConnect_Management" and "APIConnect-Portal" packages.

https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm%7EWebSphere&product=ibm/WebSphere/IBM+API+Connect&release=5.0.8.1&platform=All&function=all [https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm%7EWebSphere&product=ibm/WebSphere/IBM+API+Connect&release=5.0.6.2&platform=All&function=all] GET NOTIFIED ABOUT FUTURE SECURITY BULLETINS
 Subscribe to My Notifications [ http://www-01.ibm.com/software/support/einfo.html] to be notified of important product support alerts like this.

REFERENCES
Complete CVSS v3 Guide [http://www.first.org/cvss/user-guide]
On-line Calculator v3 [http://www.first.org/cvss/calculator/3.0]

RELATED INFORMATION
IBM Secure Engineering Web Portal [http://www.ibm.com/security/secure-engineering/bulletins.html]
IBM Product Security Incident Response Blog [http://www.ibm.com/blogs/psirt]

CHANGE HISTORY
 08 February 2018: original document published

*The CVSS Environment Score is customer environment specific and will ultimately impact the Overall CVSS Score. Customers can evaluate the impact of this vulnerability in their environments by accessing the links in the Reference section of this Security Bulletin.

DISCLAIMER
According to the Forum of Incident Response and Security Teams (FIRST), the Common Vulnerability Scoring System (CVSS) is an "industry open standard designed to convey vulnerability severity and help to determine urgency and priority of response." IBM PROVIDES THE CVSS SCORES "AS IS" WITHOUT WARRANTY OF ANY KIND, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. CUSTOMERS ARE RESPONSIBLE FOR ASSESSING THE IMPACT OF ANY ACTUAL OR POTENTIAL SECURITY VULNERABILITY.

Question: Help with Security Bulletin: API Connect is affected by a Node.js denial of service vulnerability (CVE-2017-14919) I need to understand details regarding Security Bulletin: API Connect is affected by a Node.js denial of service vulnerability (CVE-2017-14919). Where can I find this information?

Answer:
CVEID:CVE-2017-14919 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-14919]
DESCRIPTION:Node.js is vulnerable to a denial of service, caused by an uncaught exception flaw in the zlib module. By making 8 an invalid value for the windowBits parameter, a remote attacker could exploit this vulnerability to cause the application to crash.
CVSS Base Score: 7.5
CVSS Temporal Score: Seehttps://exchange.xforce.ibmcloud.com/vulnerabilities/134286 [https://exchange.xforce.ibmcloud.com/vulnerabilities/134286]for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H)


IBM Rule Team Server/Decision Center is timing out while performing certain operations - United States  TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 How to resolve "Transaction timed out after 601 seconds" error while performing certain operations in Rule Team Server (RTS)/Decision Center (DC)?

SYMPTOM
While generating ruleapps in RTS/DC, the following error is thrown after a few minutes (typically 5 or 10 minutes):

On WebLogic:

Rule Team Server has encountered an unexpected error. The internal error message is: #{SelectionBean.asynchAction}: javax.faces.el.EvaluationException: java.lang.RuntimeException: java.sql.SQLException: The transaction is no longer active - status: 'Marked rollback. [Reason=weblogic.transaction.internal.TimedOutException: Transaction timed out after 601 seconds BEA1-5057DF8C316E1C1224CD]'. No further JDBC access is allowed within this transaction.

...

On WebSphere:
TimeoutManage I WTRN0006W: Transaction XXX has timed out after 300 seconds.
TimeoutManage I WTRN0124I: When the timeout occurred the thread with which the transaction is, or was most recently, associated was Thread[WebContainer : 3,5,main]. The stack trace of this thread when the timeout occurred was:
ilog.rules.brl.parsing.parser.earley.IlrEarleyParser.errorRecovery(IlrEarleyParser.java:1046)
ilog.rules.brl.parsing.parser.earley.IlrEarleyParser.buildCharts(IlrEarleyParser.java:772)
ilog.rules.brl.parsing.parser.earley.IlrEarleyParser.parse(IlrEarleyParser.java:511)
...

CAUSE
The transaction timeout for RTS/DC is set to 10 minutes by default in the teamserver web module deployment descriptor (WEB-INF/web.xml), and application servers also have container level transaction timeouts that are configurable. If a transactional operation performed by RTS/DC takes longer than these timeouts to complete, the transaction is rolled back and the operation is not completed.

RESOLVING THE PROBLEM
If you perform time consuming operations in large repositories, you can increase the timeout value in the web.xml file of the RTS/DC EAR file (jrules-teamserver-<appserver>.ear\teamserver.war\WEB-INF) by changing the value of the property ilog.rules.teamserver.transaction.timeout.

You will find the property in the file web.xml defined as below:
...
<context-param>
<description>Modify the timeout value that is associated with transactions (in seconds)</description>
<param-name>ilog.rules.teamserver.transaction.timeout</param-name>
<param-value>600</param-value>
</context-param>
...

Another place to look for are application server specific transaction timeout configurations. For example, for WebSphere Application Server, check the "Maximum transaction timeout" and increase it as needed as described here [http://pic.dhe.ibm.com/infocenter/wasinfo/v8r0/topic/com.ibm.websphere.nd.doc/info/ae/ae/tjta_settlog.html].

Cross reference information Segment Product Component Platform Version Edition Business Integration IBM Operational Decision Manager Platform Independent 7.5 Enterprise Business Integration IBM Operational Decision Manager Platform Independent 8.0.1, 8.0 Enterprise

Question: ODM 8.7 TeamServer users active authoring rules and they get kicked out - transaction timeout or session timeout? Some of my users are being kicked out of TeamServer while actively authoring rules. What value would be controlling this behavior? Session timeout is at it's default 30 minutes. Does transaction timeout come in to play here or is there a different setting that might be causing this. Thanks.

Answer:
If you perform time consuming operations in large repositories, you can increase the timeout value in the web.xml file of the RTS/DC EAR file (jrules-teamserver-<appserver>.ear\teamserver.war\WEB-INF) by changing the value of the property ilog.rules.teamserver.transaction.timeout.

You will find the property in the file web.xml defined as below:
...
<context-param>
<description>Modify the timeout value that is associated with transactions (in seconds)</description>
<param-name>ilog.rules.teamserver.transaction.timeout</param-name>
<param-value>600</param-value>
</context-param>


IBM Supported Upgrade and Downgrade paths for DataPower Virtual Edition - United States  TECHNOTE (FAQ)

QUESTION
 Which WebSphere DataPower virtual edition fix pack can I upgrade to and downgrade from?

CAUSE
Not all scrypt4 fix packs are compatible with the 5.0.0.0 OVA files that were released in November 2012 and March 2013. If you upgrade or downgrade with an incompatible fix pack, unexpected errors might occur.

ANSWER
 Use the following links to determine which fix packs are compatible with your virtual deployment.

 * 5.0.0.0 from 30 November 2012 (build 220750)
 * 5.0.0.0 from 29 March 2013 (build 227007)
 * 5.0.0.5
 * 5.0.0.6
 * 5.0.0.8 or later
 * 6.0.0.0 or later

5.0.0.0 from 30 November 2012 (build 220750)
When your virtual deployment is the OVA file from 30 November 2012, you can install any fix pack from Fix Central, except for the 5.0.0.0 fix pack. On Fix Central, the 5.0.0.0 fix pack is for the 5.0.0.0 OVA release from 29 March 2013.

Back to top

5.0.0.0 from 29 March 2013 (build 227007)
When your virtual deployment is the OVA file from 29 March 2013, you can install only the 5.0.0.0 fix pack and 5.0.0.8 or later fix packs from Fix Central. On Fix Central, the 5.0.0.5 and 5.0.0.6 fix packs are for the 5.0.0.0 OVA release from 30 November 2013.

Back to top

5.0.0.5
When your virtual deployment is 5.0.0.5, you can install any fix pack from Fix Central, except for the 5.0.0.0 fix pack. On Fix Central, the 5.0.0.0 fix pack is for the 5.0.0.0 OVA release from 29 March 2013.

Back to top

5.0.0.6
When your virtual deployment is 5.0.0.6, you can install any fix pack from Fix Central, except for the 5.0.0.0 fix pack. On Fix Central, the 5.0.0.0 fix pack is for the 5.0.0.0 OVA release from 29 March 2013.

Back to top

5.0.0.8 or later
When your virtual deployment is 5.0.0.8 or later, do not install any fix pack from Fix Cental that is earlier than the 5.0.0.8 fix pack. The 5.0.0.8 fix pack has critical fixes. For more information, see Critical Updates [http://www-01.ibm.com/support/docview.wss?uid=swg21390112#part2].

Note: You must install the 5.0.0.8 or later fix pack to upgrade to 6.0.0.0 or later. The 5.0.0.8 fix pack contains a required fix to allow the larger sized firmware image to fit.

Back to top

6.0.0.0 or later
When your virtual deployment is the 6.0.0.0 OVA file or a firmware upgrade through a 6.0.0.0 or later fix pack, do not install any fix pack from Fix Cental that is earlier than the 5.0.0.8 fix pack. The 5.0.0.8 fix pack has critical fixes. For more information, see Critical Updates [http://www-01.ibm.com/support/docview.wss?uid=swg21390112#part2].

Back to top

Cross reference information Segment Product Component Platform Version Edition Business Integration WebSphere DataPower Service Gateway XG45 Virtual Edition VMware ESXi, VMware ESX 5.0.0, 6.0.0 Edition Independent

Question: How do I upgrade my virtual DataPower Appliance from 5.0 firmware to 6.0+ firmware? When attempting to upgrade my Virtual DataPower Appliance from 5.0.0.8 firmware version to 6.0+ firmware versions, I receive a failure message.

Answer: You must install the 5.0.0.8 or later fix pack to upgrade to 6.0.0.0 or later. The 5.0.0.8 fix pack contains a required fix to allow the larger sized firmware image to fit.


IBM web server Plugin may need LD_LIBRARY_PATH when used with Apache 2.2 - United States  TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 IBM web server Plug-in provides the connection between the web server tier and the IBM WebSphere Application Server (WAS) tier.

When using Apache 2.2 web server on Linux operating systems, the IBM web server Plug-in may need to have the LD_LIBRARY_PATH environment variable set in a certain way, in order to be able to locate the IBM Global Security Kit (GSKit) libraries needed for SSL (HTTPS).

Without the GSKit libraries, the IBM web server Plug-in will be unable to initialize SSL, and will use only HTTP for connections to the WAS appservers.

SYMPTOM
During web server start-up, errors like the following will be logged into the Plug-in log file (http_plugin.log):

ERROR: lib_security: loadSecurityLibrary: Failed to load gsk library from /opt/IBM/WebSphere/Plugins/gsk8/gsk8_64
ERROR: ws_transport: transportInitializeSecurity: Unable to load security library
ERROR: ws_server: serverAddTransport: Failed to initialize security
ERROR: ws_server: serverAddTransport: HTTPS Transport is skipped

RESOLVING THE PROBLEM
To resolve the problem, you must set the LD_LIBRARY_PATH environment variable to point to the library directory of the GSKit that comes with the IBM web server Plug-in. The precise directory location will depend of the version of Plug-in, and on the bit depth being used (32-bit or 64-bit).

For example.....

If you installed the 32-bit Plug-in v7.0 and are using with a 32-bit Apache 2.2, use this:

export LD_LIBRARY_PATH=/opt/IBM/WebSphere/Plugins/gsk7/lib/

If you installed the 64-bit Plug-in v7.0 and are using it with a 32-bit Apache 2.2, use this:

export LD_LIBRARY_PATH=/opt/IBM/WebSphere/Plugins/gsk7/gsk7_32/lib/

If you installed the 64-bit Plug-in v7.0 and are using it with a 64-bit Apache 2.2, use this:

export LD_LIBRARY_PATH=/opt/IBM/WebSphere/Plugins/gsk7/gsk7_64/lib64/

If you installed the 32-bit Plug-in v8.x and are using with a 32-bit Apache 2.2, use this:

export LD_LIBRARY_PATH=/opt/IBM/WebSphere/Plugins/gsk8/lib/

If you installed the 64-bit Plug-in v8.x and are using it with a 32-bit Apache 2.2, use this:

export LD_LIBRARY_PATH=/opt/IBM/WebSphere/Plugins/gsk8/gsk8_32/lib/

If you installed the 64-bit Plug-in v8.x and are using it with a 64-bit Apache 2.2, use this:

export LD_LIBRARY_PATH=/opt/IBM/WebSphere/Plugins/gsk8/gsk8_64/lib64/

Please note that for Apache 2.2 on Linux, simply setting the LD_LIBRARY_PATH environment variable from the command line prior to starting the web server, MAY NOT WORK. In that case, you must set this variable as follows...

For SLES and SUSE Linux add the export command into the /usr/sbin/envvars file.

For RHEL Linux add the export command into the /etc/sysconfig/httpd file.

By adding the export command into the files shown above, it means that the environment variable will be set when the web server is started. That will enable the Plug-in to be able to locate the appropriate GSKit libraries and be able to initialize SSL transports.

RELATED INFORMATION
#Web server plug-in setupGSKitLibPath script contains in [http://www-01.ibm.com/support/docview.wss?uid=swg21441018]

Question:
Why Plug-in log file reports an error message after install & configure the WebSphere Plugin 8.5 version on a Apache 2.4 webserver.

During web server start-up, errors like the following will be logged into the Plug-in log file (http_plugin.log):

ERROR: lib_security: loadSecurityLibrary: Failed to load gsk library from /opt/IBM/WebSphere/Plugins/gsk8/gsk8_64 ERROR: ws_transport: transportInitializeSecurity: Unable to load security library ERROR: ws_server: serverAddTransport: Failed to initialize security ERROR: ws_server: serverAddTransport: HTTPS Transport is skipped

Note: This is after install & configure the WebSphere Plugin 8.5 version on a Apache 2.4 webserver  on a Linux platform

Answer: To resolve the problem, you must set the LD_LIBRARY_PATH environment variable to point to the library directory of the GSKit that comes with the IBM web server Plug-in. The precise directory location will depend of the version of Plug-in, and on the bit depth being used (32-bit or 64-bit).


IBM Download IBM Tivoli Network Manager IP Edition Version 4.2.0.1 for AIX, Linux - United States  DOWNLOADABLE FILES

ABSTRACT
 This document describes how to download and install IBM Tivoli Network Manager IP Edition Version 4.2 Fix Pack 1 using the Passport Advantage Web site.

DOWNLOAD DESCRIPTION

IBM® Tivoli® Network Manager IP Edition discovers networks, including layer 1 optical networks, and layer 2 and 3 IP networks. Network Manager also discovers mobile networks (including LTE). In addition, Network Manager integrates network topology information with alert data from IBM Tivoli Netcool/OMNIbus so that operators can visualize and understand the layout of complex networks, and diagnose and solve network problems in real time.

After logging into IBM's Passport Advantage Online Web site at https://www.ibm.com/software/howtobuy/passportadvantage/pao_customers.htm [http://www.ibm.com/software/howtobuy/passportadvantage/pao_customers.htm], click Software download & media access, then click Download finder, then select Find by product description and type in Tivoli Network Manager. Expand the Tivoli Software eAssembly, and select the images you wish to download from that list. All Tivoli Network Manager IP Edition download images are described below.

More information
The IBM Tivoli Network Manager IP Edition V4.2 library of publications are available from IBM Knowledge Center at https://www.ibm.com/support/knowledgecenter/SSSHRK_4.2.0/itnm/ip/wip/common/welcome.html [https://www.ibm.com/support/knowledgecenter/SSSHRK_4.2.0/itnm/ip/wip/common/welcome.html]

Downloading and installing the product

 1. Download all of the parts that you need, both required and optional. The parts are listed in the sections below.
 2. Unpack each of the parts into a single temporary directory on your system.
 3. Follow the instructions in the Installing [https://www.ibm.com/support/knowledgecenter/SSSHRK_4.2.0/install/task/ins_installing.html] section of the IBM Knowledge Center to install the product.

Supported operating systems

On IBM® PowerPC-based systems, the following versions are supported:

 * AIX® 6.1 iSeries and pSeries
 * AIX 7.1 iSeries and pSeries

On Intel and Advanced Micro Devices (AMD) x86 processors, the following versions are supported:

 * Red Hat Enterprise Linux 6 (x86-64)
 * Red Hat Enterprise Linux 7 (x86-64)
 * SuSE Linux Enterprise Server (SLES) 11.0 (x86-64) SP2 and SP3

Downloadable parts

There are downloadable parts for each supported platform upon which IBM Tivoli Network Manager IP Edition can be installed. Refer to the tables below for details about what each part contains. At a minimum, you must download the required parts.

Required parts

To install IBM Tivoli Network Manager IP Edition V4.2.0.1 Multilingual, you need to download the following parts:

Part # Description CNE9VEN IBM Tivoli Network Manager IP Edition 4.2.0.1 for Linux Includes the base product including the core components, the GUI components, and the topology database scripts.

(English, Brazilian Portuguese, French, German, Italian, Spanish, Japanese, Korean, Hungarian, Polish, Russian, Simplified Chinese, and Traditional Chinese)

CNE9WEN IBM Tivoli Network Manager IP Edition 4.2.0.1 for AIX Includes the base product including the core components, the GUI components, and the topology database scripts.

(English, Brazilian Portuguese, French, German, Italian, Spanish, Japanese, Korean, Hungarian, Polish, Russian, Simplified Chinese, and Traditional Chinese)

CNE9UML IBM Tivoli Network Manager IP Edition V4.2.0.1 Quick Start Guide.
DB2 is the default topology database in Network Manager 4.2. Network Manager 4.2 also supports Oracle Database version 12c Enterprise Edition with Partitioning option.

Part # Description CRYY2ML IBM DB2 10.5 Enterprise Server Edition for Tivoli Netcool/OMNIbus, Network Manager, and Netcool Configuration Manager

Optional parts

These optional parts are included in this eAssembly.

Part # Description CRYY6ML IBM Tivoli Monitoring V6.3.0.2 and Monitoring Agent for Network Manager IP Edition V4.2 CN933EN IBM Tivoli Monitoring Agent for Network Manager V4.2 for Linux CN934EN IBM Tivoli Monitoring Agent for Network Manager V4.2 for AIX CN935EN IBM Tivoli Monitoring Agent for Network Manager V4.2 for Windows CJ0R8ML Jazz for Service Management v1.1.3 for Tivoli Netcool/OMNIbus, Network Manager, Netcool Configuration Manager, and Network Performance Insight CJ0S1ML Websphere Application Server 8.5.5.9 and SDK Java 7 for Tivoli Netcool/OMNIbus, Network Manager, and Netcool Configuration Manager
PREREQUISITES
For software and hardware prerequisites, see the Planning for installation section in the Network Manager 4.2 documentation on the IBM Knowledge Center.

URL LANGUAGE SIZE(Bytes) Planning for installation [https://www.ibm.com/support/knowledgecenter/SSSHRK_4.2.0/install/task/pln_planninginst.html] English 1
INSTALLATION INSTRUCTIONS
To install the product, follow the instructions in the IBM Tivoli Network Manager IP Edition V4.2 Quick Start Guide. The Quick Start Guide is in multiple languages.

URL LANGUAGE SIZE(Bytes) Quick Start Guide [https://www-01.ibm.com/support/docview.wss?uid=swg27048605] English 1
DOWNLOAD PACKAGE
Download IBM Tivoli Network Manager IP Edition V4.2.0.1 from the Passport Advantage Online Web site.

Download RELEASE DATE LANGUAGE SIZE(Bytes) Download Options Download Network Manager 4.2.0.1 for AIX and Linux 22 Sep 2016 English 1 HTTPS [https://www-01.ibm.com/software/how-to-buy/passportadvantage/pao_customers.htm]
TECHNICAL SUPPORT

 You can access IBM Tivoli Product support at https://www.ibm.com/software/sysmgmt/products/support/IBMTivoliNetworkManagerIPEdition.html [https://www.ibm.com/software/sysmgmt/products/support/IBMTivoliNetworkManagerIPEdition.html]

 You can access Passport Advantage Online support at https://www.ibm.com/software/howtobuy/passportadvantage/paocustomer/docs/en_US/ecare.html [https://www.ibm.com/software/howtobuy/passportadvantage/paocustomer/docs/en_US/ecare.html]

Question: Where I can get ITNM 4.2.0.1 GA version download details with Part number?. How to download and install IBM Tivoli Network Manager IP Edition Version 4.2 Fix Pack 1 (GA) using the Passport Advantage Web site?.

Answer:
Download all of the parts that you need, both required and optional. The parts are listed in the sections below.
 2. Unpack each of the parts into a single temporary directory on your system.
 3. Follow the instructions in the Installing [https://www.ibm.com/support/knowledgecenter/SSSHRK_4.2.0/install/task/ins_installing.html] section of the IBM Knowledge Center to install the product.


IBM Deleting Box folders and content from IBM Case Manager 5.3.0 - United States  TECHNOTE (FAQ)

QUESTION
 After configuring Box as a repository for Case Manager, Case Client users would like to delete documents and/or folders created under the Box collaboration folder. There does not appear to be any mechanism to perform this deletion.

ANSWER
There is no support for deleting documents and/or folders from a Box Collaboration folder created via Case Manager at this time. Case workers who created the Box collaboration folder will have Viewer/Uploader permission on the Box Collaboration folder, and therefore, will not be able to delete content within a Box collaboration folder as this may result in missing data in the associated cases. Users who are invited to collaborate on the Box collaboration folder have the same restriction.

Here is a link to Box collaboration permissions to clarify what permissions a Viewer/Uploader has
https://community.box.com/t5/How-To-Guides-for-Sharing/What-Are-The-Different-Access-Levels-For-Collaborators/ta-p/144 [https://community.box.com/t5/How-To-Guides-for-Sharing/What-Are-The-Different-Access-Levels-For-Collaborators/ta-p/144]

Question:
Cannot delete folder in Box collab folder

In an ICM 5.3.0 environment integrated with Box, after setting up Box Collaboration, case worker can create Box collaboration folder, and can create folders and files under Box collaboration folder, however, they cannot delete the their folders and files.

Case worker's default permission is 'Viewer Uploader' and case worker cannot add permission more than editor (Editor/Co-owner/Owner). So case worker can add new folder/file and edit file, but cannot delete folder/file.

The question is, why cannot case worker have permission more than editor?

Answer: There is no support for deleting documents and/or folders from a Box Collaboration folder created via Case Manager at this time.


IBM Access and download SPSS software - United States entitlements; software; download TECHNOTE (FAQ)

QUESTION
 Can I download the software without active Subscription and Maintenance?

CAUSE
expired entitlements

ANSWER

Official statement by IBM eCustomer Care Team (Passport Advantage Online help desk):

We would like to inform you that you can only access and download software that has active Subscription and Maintenance from Passport Advantage Online.
If the Software Subscription and Maintenance has expired you will not be able to download the software from Passport Advantage Online.
In this case, you would need to renew your software Subscription and Maintenance.

You can order the Media CD copy of the software your company purchased. The software version is the one up to the time your subscription has expired.
The sales department will be able to advise you if the Media CD is available.
You can find your local contact for the IBM Sales Department at: https://www.ibm.com/planetwide/ [https://www.ibm.com/planetwide/]

Question: I need to move my perpetual license of SPSS v24 to my new machine. I have my authorization code first given to me, but since we did not renew our support contract we can't seem to download the software. I was originally told that I could download the trial version and it would have the authorization wizard where I could input the code and move forward. But the only downloadable version is subscription version with no authorization wizard. How do I move forward?

Answer:
If the Software Subscription and Maintenance has expired you will not be able to download the software from Passport Advantage Online.
In this case, you would need to renew your software Subscription and Maintenance.

You can order the Media CD copy of the software your company purchased. The software version is the one up to the time your subscription has expired.


IBM No data visible in Web Sites workspace for ITCAM for IIS (Q7) Agent - United States 5724C0FWATR; 5742C0FWA; 5724B97ISTR; 5724B97IS TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 After successfully installing and configuring the ITCAM agent for Microsoft IIS (q7), no data appears for IIS websites and no alerts occur when the web server is stopped. Similar issue may also occur for ITCAM agent for Microsoft .NET Framework (qf).

SYMPTOM
ITCAM agent for IIS agent

 Agent shows no data for Web Sites running on the IIS Server in TEP.

ITCAM agent for .NET (qf)
Following symptoms may occur:
1) The Windows Application event log shows following event, and some data :
Faulting application name: kqfclient.exe, version: 0.0.0.0, time stamp: 0x52ce696f
Faulting module name: kqfclient.exe, version: 0.0.0.0, time stamp: 0x52ce696f
...
Faulting application path: C:\PROGRA~2\ibm\ITM\TMAITM~1\kqfclient.exe

2) No data is displayed in some workspaces, such as Web Sites view of Web Server workspace.

CAUSE
The required IIS role services and the IIS v6.0 Management Compatibility pack are not installed.

RESOLVING THE PROBLEM
To resolve this problem, install IIS 6.0 management compatibility pack on your Windows Server.

Note the following:
1) The symptom, cause and the remedial action mentioned in this technote applies to both .NET Framework agent and the ITCAM for IIS agent.
2) Even if you have IIS 7.0 - you should install the IIS 6.0 Management Compatibility Pack. This is not a part of IIS 6.0, but instead is a compatibility pack that is needed for higher IIS versions (IIS 7, 7.5, 8, 8.5). This option is found under the role services for IIS Web server.

Detailed instructions are as follows:

(i) Ensure the following IIS Role Services are installed:
-------------------------------------------------------------------------------
IIS 6 Management Compatibility
IIS 6 Metabase Compatibility
IIS 6 WMI Compatibility
IIS 6 Scripting Tools
IIS 6 Management Console

[/support/docview.wss?uid=swg21685441&aid=1] [/support/docview.wss?uid=swg21685441&aid=1]

(ii) Ensure the IIS v6.0 Management Compatibility pack is installed:
---------------------------------------------------------------------------------------------------
1. On the Master or Replica host, open the Server Manager and select the Roles option.
Click the Add Roles button.
The first page of the Add Roles Wizard appears.

2. Click Next. The Select Server Roles window appears.

3. Select the Web Server (IIS) check box, and then click Next.
A pop-up message appears, asking you whether to add features required for Web Server (IIS).

4. Click Add Required Features.
The Select Server Roles window appears.

5. Click Next.
The Web Server (IIS) window appears.

6. Click Next.
The Select Role Service window appears.

7. On the Role Services list, select the IIS 6 Management Capability check box.

8. Click Next, and follow the Wizard instructions until the completion of the installation.

Restart the agent.

Question:
Need to understand what exact counters are needed for the ITM IIS agent? Need to understand what exact counters are needed for the ITM IIS agent to collect website details and web site status. unable to find what
counters are required in the IIS doc.

Answer: To resolve this problem, install IIS 6.0 management compatibility pack on your Windows Server.


IBM NUM_IOSERVERS=AUTOMATIC behaviour is changed from 10.5FP7 - United States  TECHNOTE (FAQ)

QUESTION
 After updating to 10.5FP7 from an older fixpack level, I noticed that AUTOMATIC value for NUM_IOSERVERS is changed even though any settings are not changed. Is it expected bahavior?

ANSWER
Yes, it is expected. Due to IT09990, we changed the fomula for calculating the number of prefetchers to start at database activation time.
Until 10.5FP6, we use following fomula:
number of prefetchers =
max( max( max over all table spaces( parallelism setting ),
number of cores * number of SMT threads ), 16 )

Starting from 10.5FP7, we use following fomula:
number of prefetchers =
max( max( max over all table spaces( parallelism setting ),
number of cores * number of SMT threads ), 12 )

So you can see the following output in 10.5FP6 and FP7:

FP6:
$ db2 get db cfg | grep NUM_IOSERVERS
Number of I/O servers (NUM_IOSERVERS) = AUTOMATIC(16)

FP7
$ db2 get db cfg | grep NUM_IOSERVERS
Number of I/O servers (NUM_IOSERVERS) = AUTOMATIC(12)

RELATED INFORMATION
 num_ioservers - Number of I/O servers configuration par [http://www.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.admin.config.doc/doc/r0000333.html]

Question: Why AUTOMATIC value for NUM_IOSERVERS is changed after applying 10.5FP7? Why AUTOMATIC value for NUM_IOSERVERS is changed after applying 10.5FP7?

Answer: Yes, it is expected. Due to IT09990, we changed the fomula for calculating the number of prefetchers to start at database activation time.


IBM FileNet Workplace XT NullPointerException when bootstrapping the object store or accessing Site Preferences/Access Role - United States java.lang.NullPointerException; workplacext bootstrap; user display name TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 After logging into FileNet Workplace XT, a java.lang.NullPointerException error is logged when bootstrapping the object store or when accessing Site Preferences/Access Role.

SYMPTOM
Error Stack Trace extracted from the J2EE Application Server error log:

java.lang.NullPointerException

 at org.apache.jasper.runtime.JspWriterImpl.write(JspWriterImpl.java:421)

 at com.filenet.wcm.apps.server.ui.info.prefs.PrefAccessRolesInfoPage.renderRoleMembers(PrefAccessRolesInfoPage.java:254)

 at com.filenet.wcm.apps.server.ui.info.prefs.PrefAccessRolesInfoPage.renderRole(PrefAccessRolesInfoPage.java:220)

 at com.filenet.wcm.apps.server.ui.info.prefs.PrefBootstrapInfoPage.renderAdminRole(PrefBootstrapInfoPage.java:131)

 at com.filenet.wcm.apps.server.ui.info.prefs.PrefBootstrapInfoPage.render(PrefBootstrapInfoPage.java:303)

CAUSE
This behavior may occur if an invalid entry has been specified in the User Display Name Attribute or Group Display Name Attribute setting in the Directory Services Configuration in FileNet Enterprise Manager (FEM).

Workplace XT attempts to resolve the user names and groups defined within the Access Roles based on the Directory Service configuration defined in FEM. If the user name query resolves to null or if there is an invalid entry a NullPointerException error may occur.

RESOLVING THE PROBLEM
 In FEM, change the User Display Name Attribute or Group Display Name Attribute to a valid LDAP entry such as cn, displayName or uid and restart the FileNet Content Engine and Workplace XT instances.

Question:
We got a NullPointerException when bootstrapping the object store or accessing Site Preferences/Access Role on WorkplaceXT

After logging into FileNet Workplace XT, a java.lang.NullPointerException error is logged when bootstrapping the object store or when accessing Site Preferences/Access Role.

Here's the error stack,

Error Stack Trace extracted from the J2EE Application Server error log:

java.lang.NullPointerException

at org.apache.jasper.runtime.JspWriterImpl.write(JspWriterImpl.java:421)

at com.filenet.wcm.apps.server.ui.info.prefs.PrefAccessRolesInfoPage.renderRoleMembers(PrefAccessRolesInfoPage.java:254)

at com.filenet.wcm.apps.server.ui.info.prefs.PrefAccessRolesInfoPage.renderRole(PrefAccessRolesInfoPage.java:220)

at com.filenet.wcm.apps.server.ui.info.prefs.PrefBootstrapInfoPage.renderAdminRole(PrefBootstrapInfoPage.java:131)

at com.filenet.wcm.apps.server.ui.info.prefs.PrefBootstrapInfoPage.render(PrefBootstrapInfoPage.java:303)

Answer: In FEM, change the User Display Name Attribute or Group Display Name Attribute to a valid LDAP entry such as cn, displayName or uid and restart the FileNet Content Engine and Workplace XT instances.


IBM Replacement of an expiring certificate on the IBM WebSphere DataPower SOA Appliance - United States SSL HTTPS TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 What is the procedure for replacing an expiring certificate that is in use on the DataPower appliance with a newly obtained certificate?

CAUSE
Once you have obtained a new certificate from your certificate authority and are ready to replace the expiring one currently in use by your DataPower configuration with this new certificate, there is no single prescribed manner in which to perform the replacement operation. The following information presents some options for formulating an approach that best fits your environment.

RESOLVING THE PROBLEM
First, know your certificate material. It's not uncommon for newer certificates to be chained, so it is not a one for one replacement; the existing certificate on the DataPower appliance may need to be replaced by more than one certificate. Therefore, it is important to ensure you have the full certificate chain, as appropriate (i.e, the personal, intermediate and root certificates). Certificate authorities may send two certificates in the chain and expect the third one to be downloaded from their website.

Here's a screenshot of a sample chain. Notice that you can check the 'Certification Path' tab if there are any remaining certificates in the chain. Also note the chain of "Issued To" and "Issued By" identities.
ExampleCertificateChain.jpg [/support/docview.wss?uid=swg21500046&aid=2]ExampleCertificateChain.jpg [/support/docview.wss?uid=swg21500046&aid=1]

As for the certificate configuration on DataPower, the appliance will reference the actual key and certificate files stored in the cert: directory by file name. Navigation in the WebGUI to the SSL Proxy Profile -> Crypto Profile -> Crypto Key and Certificate objects will show the current files referenced by the configuration.

[/support/docview.wss?uid=swg21500046&aid=3] [/support/docview.wss?uid=swg21500046&aid=3]

For example, your DataPower configuration may currently reference the existing files cert:///current-key.pem and cert:///current-cert.pem.

[/support/docview.wss?uid=swg21500046&aid=4] [/support/docview.wss?uid=swg21500046&aid=4]

As long as the replacement file(s) are NOT named the same as these files (i.e. the new ones are named cert:///new-key.pemand cert:///new-cert.pem for example), it is safe to load the new file(s) into the cert: directory at any time ahead of the replacement operation. Your existing configuration will not use the new files, so long as the Crypto Key and Certificate Objects still refer to the files associated with the expiring certificate (i.e. cert:///current-key.pem and cert:///current-cert.pem).

[/support/docview.wss?uid=swg21500046&aid=5] [/support/docview.wss?uid=swg21500046&aid=5]

Having both expiring and replacement files in the cert: directory at once allows for the capability to modify the configuration to point to the new files or revert it back to the expiring files relatively easily or to configure a new SSL Proxy Profile object that references the new files. The new SSL Proxy Profile could be used to test independently of the currently running configuration using a new test service such as an XML Firewall.

You can navigate directly to the key and certificate objects using these paths:
Objects → Crypto Configuration → Crypto Certificate.
Objects → Crypto Configuration → Crypto Key

Another view is from Control Panel -> Keys and Certificate Management.

[/support/docview.wss?uid=swg21500046&aid=6] [/support/docview.wss?uid=swg21500046&aid=6]

Either way, note how the configuration ultimately references the files in the cert: directory and plan ahead for how you prefer to reference the new files.

They may be swapped out within the existing Crypto Configuration objects (i.e. update the objects to reference the new file(s) or to revert to the expiring ones as needed).

Alternatively, a new separate set of Crypto Configuration objects that reference the new files may be created and replacement performed at the Crypto object level by updating the configuration to reference the new replacement Crypto Configuration objects.

Important Note: If the replacement certificate has the same file name of the expired certificate, you MUST disable/enable the certificate object in order to clear all cached references to the expired certificate.

For more information on certificates, refer to the product documentation specific to your firmware and appliance type. Navigate to the Administrator's Guide, under "Managing the Appliance" - "Working with keys and certificates" from the WebSphere DataPower Product Documentation Portal [http://www-01.ibm.com/support/docview.wss?uid=swg21377654].

Question:
Why do I still get "certificate expired" error after adding new certificate?

The remote SSL vendor has provided us a new certificate to replace an expiring one. We have added the new certificate to the appropriate Datapower validation credential, however, connections are still failing with "certificate expired":

mpgw(Inbound_MPG): tid(51005)[request][10.20xx.xx] gtid(51345): certificate validation failed for '/C=US/O=People Services/OU=IT/CN=test.site.com' against 'My_ValCred': certificate has expired

Answer: If the replacement certificate has the same file name of the expired certificate, you MUST disable/enable the certificate object in order to clear all cached references to the expired certificate.


IBM Portal 8 installation unable to detect the WebSphere Application Server - United States portal reinstall fails; install into existing WAS fails TECHNOTE (TROUBLESHOOTING)

PROBLEM
The IBM Installation Manager is unable to detect that a WebSphere Application Server exists when attempting to install Portal.

SYMPTOM
This issue seems to manifest when an existing Portal has been uninstalled and WebSphere Application Server is not uninstalled. When Portal install is attempted again, but this time to install into the existing WebSphere Application Server then Installation Manager will throw an excecption that notes that it is unable to detect WebSphere Application Server on the system

We have also seen an issue where the install fails indicating the version of WAS detected is insufficient and we see an exception like this in IIM logs:

49 INFO 00:03.14 This Cumulative Fix requires that IBM
WebSphere Portal Server version 8.0.0.0 be previously installed. You
must first install the base release and then update it with the
Cumulative Fix. You cannot install them at the same time.
50 ERROR 00:03.15 The required version 8.0.0.5 or later of
WebSphere Application Server has not been detected on this system.
Select the WebSphere Application Server Package to install WebSphere
Application Server and WebSphere Portal.
51 INFO 00:03.21 Stopping Installation Manager.
52 INFO 00:03.28 No log properties file found in:
/home/wasadmin/var/ibm/InstallationManager/logs

CAUSE
The issue is caused by Portal files that are not correctly removed from the AppServer folder structure when Portal is uninstalled.

ENVIRONMENT
Portal 8.0.0.x and WAS 8.0.0.x

RESOLVING THE PROBLEM
Navigate to the AppServer folder and remove the following files (they are regenerated with a reinstall of Portal):

lib/ext/commons-codec-1.3.jar
lib/ext/commons-httpclient-3.0.1.jar
lib/ext/openid4java-full-0.9.5.jar
lib/ext/wp.auth.base.sua_RedirectServletFilter.jar
lib/ext/wp.auth.base.sua_loginmodule.jar
lib/ext/wp.auth.tai.jar
lib/wp.user.connections.jar
lib/wp.wire.jar
plugins/com.ibm.patch.was.plugin.jar
plugins/com.ibm.wp.was.plugin.jar
plugins/wp.ext.jar
properties/jndi.properties

Select the Recheck Status button. It should now say "All prerequisites are met."

Question:
After uninstalling Portal 8.0 only from my WAS environment my subsequent attempts to install Portal 8.0 into that same existing WAS fails.

I installed Portal 8.0.0.1 CF12 with an existing WAS 8.0.0.8 application server and everything was successful. I want to script my install so I uninstalled Portal 8.0.0.1 only and was left with an existing WAS 8.0.0.8...as before. However, when I attempt to reinstall Portal 8.0.0.1 into that same existing WAS 8.0.0.8 to test my scripting I get the following error immediately:

ERROR: The required version 8.0.0.5 or later of WebSphere Application Server has not been detected on this system. Select the WebSphere Application Server Package to install WebSphere Application Server and WebSphere Portal.

IIM shows WAS as installed, so I am not sure what is causing this behavior when it installed fine just a few hours ago.

Answer:
Navigate to the AppServer folder and remove the following files (they are regenerated with a reinstall of Portal):

lib/ext/commons-codec-1.3.jar
lib/ext/commons-httpclient-3.0.1.jar
lib/ext/openid4java-full-0.9.5.jar
lib/ext/wp.auth.base.sua_RedirectServletFilter.jar
lib/ext/wp.auth.base.sua_loginmodule.jar
lib/ext/wp.auth.tai.jar
lib/wp.user.connections.jar
lib/wp.wire.jar
plugins/com.ibm.patch.was.plugin.jar
plugins/com.ibm.wp.was.plugin.jar
plugins/wp.ext.jar
properties/jndi.properties


IBM Wrong codepoints for non-ASCII characters inserted in UTF-8 database using CLP - United States CLP; insert; codepage; unicode; wrong codepoints; CharConversionException TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 During insert from the CLP there is no codepage conversion if operating system codepage and database codepage are both UTF-8. In this case data to be inserted should also be in UTF-8 encoding.

If data has a different encoding then the database codepage (this can be verified using any hex editor), then the operating system codepage should be changed to match the data's encoding in order to enforce the data conversion to the database codepage.

SYMPTOM
Error executing Select SQL statement. Caught by java.io.CharConversionException. ERRORCODE=-4220

 Caused by:
java.nio.charset.MalformedInputException: Input length = 4759
at com.ibm.db2.jcc.b.u.a(u.java:19)
at com.ibm.db2.jcc.b.bc.a(bc.java:1762)

CAUSE
During an insert of data using CLP characters, they do not go through codepage conversion.
If operating system and database codepage both are UTF-8, but the data to be inserted is not Unicode, then data in the database might have incorrect codepoints (not-Unicode) and the above error will be a result during data retrieval.

To verify the encoding for data to be inserted you can use any editor that shows hex representation of characters. Please verify the codepoints for non-ASCII characters that you try to insert. If you see only 1 byte per non-ASCII characters then you need to force the database conversion during insert from CLP to UTF-8 database.

To force codepage conversion during insert from the CLP make sure that the operating system codepage is non-Unicode and matching to the codepage of data when you insert data to Unicode database from non-Unicode data source.

Problem Details
An example problem scenario is as follows:

 1.  Create a database of type UTF-8:

     CREATE DATABASE <db> USING CODESET utf-8 TERRITORY US

 2.  Create a table that holds character data:

     CREATE TABLE test (col char(20))

 3.  Check operating system locale:

     locale LANG=en_US.UTF-8
     LC_CTYPE="en_US.UTF-8"

 4.  Insert the non-ASCII characters 'Ã' , '³', '©' which have codepoint 0x'C3', 0x'B3',0x'A9' in codepage 819 into the table:

     INSERT INTO test VALUES ('Ã')
     INSERT INTO test VALUES ('³')
     INSERT INTO test VALUES ('©')

 5.  By running the following statement, you can see that all INSERT statements caused only one byte to be inserted into the table:

     SELECT col, HEX(col) FROM test

     Ã C3
     ³ B3
     © A9

     However, the UTF-8 representation of those characters are: 0x'C383' for 'Ã', 0x'C2B3' for '³', and 0x'C2A9' for '©'.
     So these three rows in the table contain invalid characters in UTF-8.

 6.  When selecting from a column using the JDBC application, the following error will occur. This is expected because the table contains invalid UTF-8 data:
     Error executing Select SQL statement. Caught by java.io.CharConversionException. ERRORCODE=-4220
     Caused by:
     java.nio.charset.MalformedInputException: Input length = 4759
     at com.ibm.db2.jcc.b.u.a(u.java:19)
     at com.ibm.db2.jcc.b.bc.a(bc.java:1762)

 7.  Delete all rows with incorrect Unicode codepoints from the test table:
     DELETE * from test

 8.  Change the locale to one that matching codepage of data to be inserted:
     export locale=en_us. One of the way to determine the codepage for your data can be found here: http://www.codeproject.com/Articles/17201/Detect-Encoding-for-In-and-Outgoing-Text [http://www.codeproject.com/Articles/17201/Detect-Encoding-for-In-and-Outgoing-Text]. If you prepare data yourself using some editor please check the documentation for your editor to find out how to set up the codepage for data being prepared by the editor.

 9.  Insert data to the table:
     INSERT INTO test VALUES ('Ã')
     INSERT INTO test VALUES ('³')
     INSERT INTO test VALUES ('©')

 10. Verify that inserted data were converted to UTF-8 during insert:
     SELECT col, HEX(col) FROM test

     Ã C383
     ³ C2B3
     © C2A9

 11. Run your java application selecting Unicode data. No exception should be reported.

ENVIRONMENT
UNIX, Linux, Unicode database

DIAGNOSING THE PROBLEM
Verify that non-ASCII data have a proper Unicode codepoints in Unicode database

RESOLVING THE PROBLEM
Reinsert data with codepage conversion enforced by setting the operation system codepage matching to the codepage of data to be inserted

RELATED INFORMATION
 Export data: [http://www.ibm.com/developerworks/db2/library/techarticle/0210lau/0210lau.html]

Question:
Table ""."" could not be found

When doing a table mapping with DB2 as source, selecting a table which contains special characters on column comments like:

COMMENT ON COLUMN .. IS 'Test comment ‘1’ complete';

Mapping fails with following error message: Table . could not be found in the database.

Answer: Reinsert data with codepage conversion enforced by setting the operation system codepage matching to the codepage of data to be inserted


IBM How to delete a document hierarchy node in IBM Datacap Taskmaster Capture - United States dco document hierarchy node delete remove hide blank page TECHNOTE (FAQ)

QUESTION
 How do I delete a document hierarchy node, such as a blank page, so that it no longer processes rules or appears in a batchview listing?

CAUSE
It is sometimes desired to remove pages or documents from a batch, as they are no longer needed or to simplify processing for a Verify operator.

ANSWER
The DeleteChildType from the Validations library can be used by running it from the parent of the object to be deleted and specifying the specific child.

For example, to remove blank pages from a batch:

 1. Create a special page type attached at the batch level called something like DeleteMe.
 2. Use an action like BlankPagesIDBySize to identify the blank page:  * BlankPagesIDBySize (1000,DeleteMe)

 3. Create a rule with the action DeleteChildType(DeleteMe) and attach it at the batch level to have it remove all pages with page type DeleteMe.

If building a custom action, the DeleteChild API method is invoked from the parent object.

 *
 *

Question:
Hide a page in dotedit step  I am working on datacap 8.1 version. I have 2 pages in a batch. Page 1 with status as 1, Page 2 with status as 20. Both are of same page type. In verification step (dotedit) I want to hide the page which is with status 20. I have given status 20 in ignored page status in web administrator for verify. But still when I navigate through pages in dotedit batch view the respective panel is loading. I am able to submit the batch even if I have one page in a batch with 20 as status.

Is there any way to delete the page (we have multiple same type of pages in a batch) from dotedit based on the page status?

Answer: The DeleteChildType from the Validations library can be used by running it from the parent of the object to be deleted and specifying the specific child.


IBM Security Bulletin: IBM MQ and IBM MQ Appliance are vulnerable to SWEET32 Birthday attack (CVE-2016-2183) - United States  SECURITY BULLETIN

SUMMARY
 A security vulnerability affects IBM MQ and IBM MQ Appliance, that could allow an attacker to obtain sensitive information when using a channel CipherSpec that uses the Triple-DES algorithm. The affected CipherSpecs are:

- TRIPLE_DES_SHA_US
- FIPS_WITH_3DES_EDE_CBC_SHA
- ECDHE_ECDSA_3DES_EDE_CBC_SHA256
- ECDHE_RSA_3DES_EDE_CBC_SHA256

VULNERABILITY DETAILS
CVEID: CVE-2016-2183 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-2183]
DESCRIPTION: OpenSSL could allow a remote attacker to obtain sensitive information, caused by an error in the DES/3DES cipher, used as a part of the SSL/TLS protocol. By capturing large amounts of encrypted traffic between the SSL/TLS server and the client, a remote attacker able to conduct a man-in-the-middle attack could exploit this vulnerability to recover the plaintext data and obtain sensitive information. This vulnerability is known as the SWEET32 Birthday attack.
CVSS Base Score: 3.7
CVSS Temporal Score: See https://exchange.xforce.ibmcloud.com/vulnerabilities/116337 [https://exchange.xforce.ibmcloud.com/vulnerabilities/116337] for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:N/AC:H/PR:N/UI:N/S:U/C:L/I:N/A:N)

AFFECTED PRODUCTS AND VERSIONS
The following versions are affected:

 * IBM MQ  * Maintenance levels between 8.0.0.0 and 8.0.0.5
    * 9.0.0.0 only

 * IBM MQ Appliance  * Maintenance levels between 8.0.0.0 and 8.0.0.5

 * IBM WebSphere MQ  * Maintenance levels between 7.0.1.0 and 7.0.1.14
    * Maintenance levels between 7.1.0.0 and 7.1.0.8
    * Maintenance levels between 7.5.0.0 and 7.5.0.7

REMEDIATION/FIXES
 IBM MQ V9.0

 * Apply Fix Pack 9.0.0.1 [http://www-01.ibm.com/support/docview.wss?uid=swg27048059]

IBM MQ V8.0 & IBM MQ Appliance V8.0  * Apply Fix Pack 8.0.0.6 [https://www.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm~WebSphere&product=ibm/WebSphere/WebSphere+MQ&release=8.0.0.6&platform=All&function=all].
 * Apply Fix Pack 8.0.0.6 for IBM MQ Appliance [https://www.ibm.com/support/fixcentral/swg/selectFixes?parent=ibm~WebSphere&product=ibm/WebSphere/IBM+MQ+Appliance+M2000&release=8.0.0.6&platform=All&function=all].

IBM MQ V7.5.0  * Apply APAR IV90867 [http://www.ibm.com/support/fixcentral/swg/quickorder?parent=ibm%7EWebSphere&product=ibm/WebSphere/WebSphere+MQ&release=7.5.0.7&platform=All&function=aparId&apars=IV90867&source=fc].

IBM MQ V7.1.0  * Apply APAR IV90867 [http://www.ibm.com/support/fixcentral/swg/quickorder?parent=ibm%7EWebSphere&product=ibm/WebSphere/WebSphere+MQ&release=7.1.0.8&platform=All&function=aparId&apars=IV90867&source=fc].

IBM MQ V7.0.1  * Users of MQ 7.0.1 should contact IBM support to request an interim fix for APAR IV90867.

WORKAROUNDS AND MITIGATIONS
All versions of MQ can mitigate the vulnerability either by switching to an alternative CipherSpec or by enabling secret key reset.
Resetting SSL/TLS secret keys [http://www.ibm.com/support/knowledgecenter/SSFKSJ_9.0.0/com.ibm.mq.sec.doc/q014310_.htm]

GET NOTIFIED ABOUT FUTURE SECURITY BULLETINS
 Subscribe to My Notifications [ http://www-01.ibm.com/software/support/einfo.html] to be notified of important product support alerts like this.

IMPORTANT NOTE
 IBM strongly suggests that all System z customers be subscribed to the System z Security Portal to receive the latest critical System z security and integrity service. If you are not subscribed, see the instructions on the System z Security web site [http://www.ibm.com/systems/z/solutions/security_subintegrity.html]. Security and integrity APARs and associated fixes will be posted to this portal. IBM suggests reviewing the CVSS scores and applying all security or integrity fixes as soon as possible to minimize any potential risk.

REFERENCES
Complete CVSS v3 Guide [http://www.first.org/cvss/user-guide]
On-line Calculator v3 [http://www.first.org/cvss/calculator/3.0]

RELATED INFORMATION
IBM Secure Engineering Web Portal [http://www.ibm.com/security/secure-engineering/bulletins.html]
IBM Product Security Incident Response Blog [http://www.ibm.com/blogs/psirt]

CHANGE HISTORY
 20 February 2017: Original version published
21 March 2017: Removed applicability to 9.0.1 release
4 April 2017: Added details for 7.0.1 fix
24 May 2017: Details for 9.0.0.1 added
19 June 2017: Removed HP & Solaris Specifics for 9.0.0.1

*The CVSS Environment Score is customer environment specific and will ultimately impact the Overall CVSS Score. Customers can evaluate the impact of this vulnerability in their environments by accessing the links in the Reference section of this Security Bulletin.

DISCLAIMER
According to the Forum of Incident Response and Security Teams (FIRST), the Common Vulnerability Scoring System (CVSS) is an "industry open standard designed to convey vulnerability severity and help to determine urgency and priority of response." IBM PROVIDES THE CVSS SCORES "AS IS" WITHOUT WARRANTY OF ANY KIND, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. CUSTOMERS ARE RESPONSIBLE FOR ASSESSING THE IMPACT OF ANY ACTUAL OR POTENTIAL SECURITY VULNERABILITY.

RELATED INFORMATION
 A Japanese translation is available [http://www.ibm.com/support/docview.wss?uid=swg21999690]

Cross reference information Segment Product Component Platform Version Edition Business Integration WebSphere MQ SSL Platform Independent 9.0, 8.0, 7.5, 7.1, 7.0.1 All Editions

Question: Is MQ or MQ Appliance impacted by Sweet32 Birthday Attack if we are running triple-DES cipher.? Is MQ or MQ Appliance impacted by Sweet32 Birthday Attack if we are running triple-DES cipher.?

Answer:
The following versions are affected:

 * IBM MQ  * Maintenance levels between 8.0.0.0 and 8.0.0.5
    * 9.0.0.0 only

 * IBM MQ Appliance  * Maintenance levels between 8.0.0.0 and 8.0.0.5

 * IBM WebSphere MQ  * Maintenance levels between 7.0.1.0 and 7.0.1.14
    * Maintenance levels between 7.1.0.0 and 7.1.0.8
    * Maintenance levels between 7.5.0.0 and 7.5.0.7


IBM Action required for IBM Integration Bus Hypervisor Edition V9.0 and WebSphere Message Broker Hypervisor Edition V8.0 for security vulnerabilities in Red Hat Linux - United States  FLASH (ALERT)

ABSTRACT
 IBM Integration Bus Hypervisor Edition V9.0 and WebSphere Message Broker Hypervisor Edition V8.0 require customer action for security vulnerabilities in Red Hat Linux.

CONTENT
IBM Integration Bus Hypervisor Edition V9.0 and WebSphere Message Broker Hypervisor Edition V8.0 ship with versions of Red Hat Enterprise Linux (RHEL) Server that are vulnerable to CVE-2016-8677, CVE-2016-8678 CVE-2016-9298, CVE-2014-9915, CVE-2016-10046, CVE-2016-10047, CVE-2016-10048, CVE-2016-10049, CVE-2016-10050, CVE-2016-10051, CVE-2016-10052, CVE-2016-10053, CVE-2016-10054, CVE-2016-10055, CVE-2016-10056, CVE-2016-10057, CVE-2016-10058, CVE-2016-10059, CVE-2016-10060, CVE-2016-10061, CVE-2016-10062, CVE-2016-10063, CVE-2016-10064, CVE-2016-10065, CVE-2016-10066, CVE-2016-10067, CVE-2016-10068, CVE-2016-10069, CVE-2016-10070, CVE-2016-10071, CVE-2016-10144, CVE-2016-10145, CVE-2016-10146, CVE-2017-5506, CVE-2017-5507, CVE-2017-5508, CVE-2017-5511, CVE-2017-6498, CVE-2017-6499, CVE-2017-6500, CVE-2017-6501, CVE-2017-6502, CVE-2016-10252 and CVE-2017-6497

IBM Integration Bus Hypervisor Edition V9.0 and WebSphere Message Broker Hypervisor Edition V8.0 ship with Red Hat Enterprise Linux (RHEL) Server 6.2.

Remediation:

IBM strongly recommends that you contact Red Hat to obtain and install fixes for Red Hat Enterprise Linux (RHEL) Server 6.2, as applicable.

Cross reference information Segment Product Component Platform Version Edition Business Integration WebSphere Message Broker Hypervisor Edition Linux 8.0
PRODUCT ALIAS/SYNONYM
 WMB HVE IIB HVE

Question: Help with Action required for IIIB V9 and WMB V8 Hypervisor Editions for security vulnerabilities in Red Hat Linux I need to understand details regarding Action required for IBM Integration Bus Hypervisor Edition V9.0 and WebSphere Message Broker Hypervisor Edition V8.0 for security vulnerabilities in Red Hat Linux. Where can I find this information?

Answer:
IBM Integration Bus Hypervisor Edition V9.0 and WebSphere Message Broker Hypervisor Edition V8.0 ship with versions of Red Hat Enterprise Linux (RHEL) Server that are vulnerable to CVE-2016-8677, CVE-2016-8678 CVE-2016-9298, CVE-2014-9915, CVE-2016-10046, CVE-2016-10047, CVE-2016-10048, CVE-2016-10049, CVE-2016-10050, CVE-2016-10051, CVE-2016-10052, CVE-2016-10053, CVE-2016-10054, CVE-2016-10055, CVE-2016-10056, CVE-2016-10057, CVE-2016-10058, CVE-2016-10059, CVE-2016-10060, CVE-2016-10061, CVE-2016-10062, CVE-2016-10063, CVE-2016-10064, CVE-2016-10065, CVE-2016-10066, CVE-2016-10067, CVE-2016-10068, CVE-2016-10069, CVE-2016-10070, CVE-2016-10071, CVE-2016-10144, CVE-2016-10145, CVE-2016-10146, CVE-2017-5506, CVE-2017-5507, CVE-2017-5508, CVE-2017-5511, CVE-2017-6498, CVE-2017-6499, CVE-2017-6500, CVE-2017-6501, CVE-2017-6502, CVE-2016-10252 and CVE-2017-6497

IBM Integration Bus Hypervisor Edition V9.0 and WebSphere Message Broker Hypervisor Edition V8.0 ship with Red Hat Enterprise Linux (RHEL) Server 6.2.


IBM The 'Manage Group' Add User search in the Process Admin Console does not return available users for IBM Business Process Manager (BPM) - United States  TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 When you search for a user within the 'Manage Group' in the Process Admin Console for IBM Business Process Manager, a more specific search keyword does not result in any entries. In addition, an LDAP timeout message is added to the SystemOut.log file.

SYMPTOM
When you do a broad search with a small keyword, you expect to receive many entries in return. When your search is more specific, you use a longer keyword to try to narrow the search. However, when you expect fewer returns from your search, it does not happen. The user list remains empty.

You can find the following exception in the SystemOut.log file:

[8/7/12 14:02:59:435 CEST] 000000xx exception E com.ibm.ws.wim.
adapter.ldap.LdapConnection search(String, String, Object[],
SearchControls) CWWIM4520E The 'javax.naming.NamingException: LDAP
response read timed out, timeout used:20000ms.; Remaining name:
'DC=xxx,DC=xxx,DC=xxx,DC=com'; Resolved object: 'com.sun.jndi.
ldap.LdapCtx@196fbd6a'' naming exception occurred during processing.
[8/7/12 14:02:59:435 CEST] 00000068 exception E com.ibm.ws.wim.
adapter.ldap.LdapConnection search(String, String, Object[],
SearchControls)
com.ibm.websphere.wim.exception.
WIMSystemException: CWWIM4520E The 'javax.naming.NamingException: LDAP
response read timed out, timeout used:20000ms.; Remaining name:
'DC=xxx,DC=xxx,DC=xxx,DC=com'; Resolved object: 'com.sun.jndi.
ldap.LdapCtx@196fbd6a'' naming exception occurred during processing.
at com.ibm.ws.wim.adapter.ldap.LdapConnection.search
(LdapConnection.java:2951)
at com.ibm.ws.wim.adapter.ldap.LdapConnection.checkSearchCache
(LdapConnection.java:2781)
at com.ibm.ws.wim.adapter.ldap.LdapConnection.search
(LdapConnection.java:2971)
...

CAUSE
In some cases, LDAP might take longer to respond on a more specific search, which results in an LDAP timeout.

RESOLVING THE PROBLEM
To resolve this issue, increase the LDAP timeout limit by using one of the following options:

 * Option 1
   Use the wsadmin command:
   $AdminTask updateIdMgrLDAPServer {-id AD -connectTimeout 0}
   $AdminConfig save

   After it is saved, view the wimconfig.xml file and make sure you see connectTimeout="0"

   You can find the wimconfig.xml file in the following locations in the deployment manager configuration:  * {WAS_HOME}\profiles\profile_name\config\cells\cell_name\wim\config\wimconfig.xml

    * profile_root/conf/cells/cell_name/wim/config/wimconfig.xml

   A complete restart of the WebSphere Application Server / IBM Business Process Manager environment is required to make this change take effect.

 * Option 2
   You will edit the wimconfig.xml file directly. However, before you edit the file, make a backup copy of it. Edit the wimconfig.xml file and search forconnectTimeout. Change this value from connectTimeout="20" to connectTimeout="0" and save the file.

   A complete restart of the WebSphere Application Server / IBM Business Process Manager environment is required to make this change take effect.

Note: The connectTimeout increment value is in seconds of time. The example value 0 is to wait indefinitely. Different values can be set accordingly.

Cross reference information Segment Product Component Platform Version Edition Business Integration IBM Business Process Manager Standard Security AIX, Linux, Linux zSeries, Solaris, Windows 8.5, 8.0.1, 8.0, 7.5.1, 7.5 Business Integration IBM Business Process Manager Express Security Linux, Windows 8.5, 8.0.1, 8.0, 7.5.1, 7.5

Question:
BPM AppTarget startup hangs during startup

During startup of the apptarget, JVM hangs with the following messages

[9/8/14 14:11:01:709 CDT] 0000001f wle W com.lombardisoftware.server.core.GroupCore getAllGroups() CWLND0004W - The system has been waiting for the group replication lock to be released for the last 10 minutes. [9/8/14 14:11:32:810 CDT] 00000022 ThreadMonitor W WSVR0605W: Thread "server.startup : 0" (0000001f) has been active for 654582 milliseconds and may be hung. There is/are 1 thread(s) in total in the server that may be hung. at java.lang.Thread.sleep(Native Method) at java.lang.Thread.sleep(Thread.java:896) at com.lombardisoftware.server.core.GroupCore.getAllGroups(GroupCore.java:545) at com.lombardisoftware.server.ejb.security.SecurityCore.listAllGroups(SecurityCore.java:566) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:611) ... [9/8/14 14:21:02:662 CDT] 0000001f wle W com.lombardisoftware.server.core.GroupCore getAllGroups() CWLND0004W - The system has been waiting for the group replication lock to be released for the last 20 minutes. [9/8/14 14:31:03:193 CDT] 0000001f wle W com.lombardisoftware.server.core.GroupCore getAllGroups() CWLND0004W -The system has been waiting for the group replication lock to be released for the last 30 minutes. [9/8/14 14:41:03:749 CDT] 0000001f wle W com.lombardisoftware.server.core.GroupCore getAllGroups() CWLND0004W -The system has been waiting for the group replication lock to be released for the last 40 minutes.

Answer:
You will edit the wimconfig.xml file directly. However, before you edit the file, make a backup copy of it. Edit the wimconfig.xml file and search forconnectTimeout. Change this value from connectTimeout="20" to connectTimeout="0" and save the file.

   A complete restart of the WebSphere Application Server / IBM Business Process Manager environment is required to make this change take effect.


IBM Diagnosing a java.lang.StackOverflowError - United States StackOverflow; overflowing; recursion; recursively; jit; JustInTime; Hot; Spot; WebSphere; Just-In-Time TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 Stack Overflow exceptions can occur when a thread stack continues to grow in size until reaching the maximum limit.

SYMPTOM
 A stack overflow can result from:

 * A deeply nested application
 * An infinite loop within an application
 * A problem in just-in-time (JIT) compiled code
 * Applications requiring a larger stack size, especially ones relying on XML, GUI, or java2D classes.
 * Native method calls

Stack overflow issues are frequently masked by Out of Memory exceptions. By resolving the memory constraints, the stack overflow can be resolved.
CAUSE
When a stack overflow occurs, the amount of stack space required by the program exceeds what is configured for the stack in the Java™ Virtual Machine (JVM) process, or the native stack size configured by the operating system.
Some applications require stacks that are larger than the default size; for example, a graphics-intensive Java program can require a larger stack, which may require an increase in the stack size to avoid StackOverflow.

DIAGNOSING THE PROBLEM
Look for either Out of Memory messages or java.lang.StackOverflow in the server logs. The process may continue to run after either of these messages are seen.

If a crash did occur, a javacore should have generated on IBM SDK. You will either see the signal as SIGSEGV, SIGILL, SIGABRT, or SIGBUS. Usually the current thread will indicate the following

pending=java.lang.StackOverflowError
RESOLVING THE PROBLEM

[/support/docview.wss?uid=swg21162896&aid=1]Explanations and Solutions
[/support/docview.wss?uid=swg21162896&aid=2]Detailed Configuration Options

Explanations and Solutions

Infinite Recursion
If an application is performing recursion, the maximum stack size can easily be reached and a Stack Overflow exception is thrown. The thread stack has a limited size and eventually its space will run out as the thread stack grows without bounds.

Some traits of recursion:
- Large thread stacks that appear to repeat
- An infinite loop that continuously spawns off threads
- Very large XML documents loaded into the Document Object Model (DOM)
- JSP or servlets calling itself (usually by executing forward or include to itself)
- Repeated calls in native functions

Increasing the thread stack size allows for larger thread stacks. However if the recursion continues to trigger a stack overflow, the next step is to identify what code is causing the recursion from javacores, thread dumps, or even system core files.

A thread stack that indicates it's too large to display can be an indicator of stack overflow. This is especially true if the stack appears to repeat (such as recursive method calls).

JIT/HotSpot Compiled Code

The JIT/HotSpot compiler (JIT) is designed to speed up the JVM execution times by compiling method calls. This can speed up execution time, but as more aggressive optimizations are used, this can inadvertently cause recursion, resulting in stack overflow or crash. The documents linked below explain how to debug JIT and HotSpot compiler issues:  * JIT Problem Determination for IBM SDK 5.0 and 6.0 using -Xjit [http://www.ibm.com/support/docview.wss?rs=0&uid=swg21294023]
 * Debugging HotSpot failures on the Solaris and HP-UX operating environment [http://www.ibm.com/support/docview.wss?rs=0&uid=swg21081179]

Depleted Native Stack
It is almost guaranteed that a crash will occur if the native stack runs out of space. System cores can be used to identify long and possibly recursively looping native thread stacks.

To resolve, increase the native stack size (usually by adjusting the operating system limit for stack size) to accommodate the recursive native calls; however, identifying the recursive native calls will help determine the root cause of the stack overflow.

[/support/docview.wss?uid=swg21162896&aid=3] Back to top

[http://www.ibm.com/i/v14/rules/dotted_blue_rule_443.gif]

Detailed Configuration Options

Adjusting the Stack Sizes (Xss and Xmso) options
If this does not work, you can adjust the stack sizes. Doubling the stack size is suggested as a first step; however, every thread that the JVM creates will consume memory. Be careful to not exhaust your physical and heap memory resources.

For every Java thread, there are two stacks that are utilized. One is for Java code for method calls, and the other is for native C code (on Solaris and HP-UX, only the native stack is utilized). These are adjustable and can be raised to provide more room on the stacks to prevent an overflow.

 * Maximum Thread Stack Size (-Xss)
 *
 *
 *
 *
 *  -Xss<size>
 *
 * <size>
 *
 * nn[k|m|g|K|M|G]
 *
 *
 *
 *
 *
 *
 * there is no distinction between native and thread stack sizes
 *
 * only
 *
 * HP-UX
 *
 *
 *
 *  -XX:MainThreadStackSize=<size>
 *
 *
 *
 *
 *
 *
 *
 * Initial Thread Stack Size (-Xiss)
 *
 * This is only for distributed platforms (AIX, Linux, Windows)
 *
 *
 *
 *
 *
 *  -Xiss<size>
 *
 * <size>
 *
 * nn[k|m|g|K|M|G]
 *
 *
 *
 *
 * Initial Native Stack Size (-Xmso)
 *
 * This is only for distributed platforms (AIX, Linux, Windows)
 *
 *
 *
 *
 *
 *
 *  -Xmso<size>
 *
 * <size>
 *
 * nn[k|m|g|K|M|G]
 *
 *
 *
 *

Default Values
This is a chart of default sizes for the various stack size arguments. This is subject to change due to upgrades of the SDK.

[/support/docview.wss?uid=swg21162896&aid=4]
Value [/support/docview.wss?uid=swg21162896&aid=4]
AIX [/support/docview.wss?uid=swg21162896&aid=4]
Linux [/support/docview.wss?uid=swg21162896&aid=4]
Windows [/support/docview.wss?uid=swg21162896&aid=4]
Solaris [/support/docview.wss?uid=swg21162896&aid=4]
HP-UX [/support/docview.wss?uid=swg21162896&aid=4]
Initial Thread Stack Size [/support/docview.wss?uid=swg21162896&aid=4]
2KB [/support/docview.wss?uid=swg21162896&aid=4]
2KB [/support/docview.wss?uid=swg21162896&aid=4]
2KB [/support/docview.wss?uid=swg21162896&aid=4]
Not Adjustable [/support/docview.wss?uid=swg21162896&aid=4]
Not Adjustable [/support/docview.wss?uid=swg21162896&aid=4]
Maximum Thread Stack Size [/support/docview.wss?uid=swg21162896&aid=4]
256KB (32-bit)
512KB (64-bit) [/support/docview.wss?uid=swg21162896&aid=4]
256KB (32-bit)
512KB (64-bit) [/support/docview.wss?uid=swg21162896&aid=4]
256KB (32-bit)
512KB (64-bit) [/support/docview.wss?uid=swg21162896&aid=4]
Not Adjustable [/support/docview.wss?uid=swg21162896&aid=4]
Not Adjustable [/support/docview.wss?uid=swg21162896&aid=4]
Native Stack Size [/support/docview.wss?uid=swg21162896&aid=4]
256KB [/support/docview.wss?uid=swg21162896&aid=4]
256KB [/support/docview.wss?uid=swg21162896&aid=4]
32KB (32-bit)
256KB (64-bit) [/support/docview.wss?uid=swg21162896&aid=4]
512KB (32-bit)
1024KB (64-bit) [/support/docview.wss?uid=swg21162896&aid=4]
64KB (PA-RISC)
1024KB (Itanium) [/support/docview.wss?uid=swg21162896&aid=4]
[/support/docview.wss?uid=swg21162896&aid=4]
[/support/docview.wss?uid=swg21162896&aid=4]
[/support/docview.wss?uid=swg21162896&aid=4]
[/support/docview.wss?uid=swg21162896&aid=4]
[/support/docview.wss?uid=swg21162896&aid=4]
[/support/docview.wss?uid=swg21162896&aid=5] Back to top

Examples of Stack Overflow

Server Logs
These logs may contain references to StackOverflow, alongside other messages. Thread stacks may accompany these messages, indicating if there is a recursive call.
 * SystemOut.log example

 * [3/14/15 3:14:15:926 CST] 31415926 BaseMap W CWOBJ0006W: An exception occurred:
 *
 * com.ibm.websphere.objectgrid.ObjectGridRuntimeException: java.lang.StackOverflowError
 *
 * <<Thread stack may follow this message>>
 *

 *
 *
 *
 * SystemErr.log example

 * [3/14/15 3:14:15:926 PST] 31415926 SystemErr R Caused by: java.lang.StackOverflowError
 *
 *  at java.util.Hashtable.get(Hashtable.java:461)
 *
 *  at com.ibm.ws.webcontainer.srt.SRTServletRequest.getParameter(SRTServletRequest.java:1257)
 *
 *  at psdi.webclient.system.session.WebClientSession.applySkin(WebClientSession.java:295)
 *
 *  at psdi.webclient.system.controller.AppInstance.render(AppInstance.java:1177)
 *
 *  at psdi.webclient.system.controller.AppInstance.render(AppInstance.java:1213)
 *
 *  at psdi.webclient.system.controller.AppInstance.render(AppInstance.java:1213)
 *
 * ...
 *

 *

Javacore

In javacores, look for "StackOverflowError" in the thread name entry: 3XMTHREADINFO "ORB.thread.pool : 19" (TID:10167F78, sys_thread_t:E072BCE0, state:R, native ID:E06BEBA0) prio=5: pending=java.lang.StackOverflowError
System Dump (processed with jextract and DTFJ Dump Analyzer)

Look for this message near the end of a very large stack. This particular example had 1000 recursively called entries. Java stack frames dump is too long: <analyzererror>

Cross reference information Segment Product Component Platform Version Edition Application Servers Runtimes for Java Technology Java SDK

Question:
Getting java.lang.StackOverflowError,  how to resolve it?

Getting the following StackOverflowError when executing a dump namespace:

TRACE START:

        java.lang.StackOverflowError at java.util.regex.Pattern$Branch.match(Pattern.java:4513) at java.util.regex.Pattern$GroupTail.match(Pattern.java:4626) at java.util.regex.Pattern$Curly.match1(Pattern.java:4196) at java.util.regex.Pattern$Curly.match(Pattern.java:4145) at java.util.regex.Pattern$GroupHead.match(Pattern.java:4567) at java.util.regex.Pattern$Branch.match(Pattern.java:4511) at java.util.regex.Matcher.match(Matcher.java:1232) at java.util.regex.Matcher.matches(Matcher.java:570) at com.ibm.rmi.corba.IIOPAddrImpl.(IIOPAddrImpl.java:107) at com.ibm.rmi.corba.Corbaloc.parseObjAddr(Corbaloc.java:231) at com.ibm.rmi.corba.Corbaloc.parseObjAddrList(Corbaloc.java:181) at com.ibm.rmi.corba.Corbaloc.parseCorbaloc(Corbaloc.java:145) at com.ibm.rmi.corba.Corbaloc.(Corbaloc.java:120) at com.ibm.rmi.corba.ORB.createObjectURL(ORB.java:3731) at com.ibm.rmi.corba.ORB.createObjectURL(ORB.java:3698) at com.ibm.rmi.corba.ORB.string_to_object(ORB.java:3694)

Answer:
When a stack overflow occurs, the amount of stack space required by the program exceeds what is configured for the stack in the Java™ Virtual Machine (JVM) process, or the native stack size configured by the operating system.
Some applications require stacks that are larger than the default size; for example, a graphics-intensive Java program can require a larger stack, which may require an increase in the stack size to avoid StackOverflow.


IBM ITCAM Data Collector connection to the Managing Server fails with Connection Refused. - United States  TECHNOTE (TROUBLESHOOTING)

PROBLEM(ABSTRACT)
 The Data Collector (DC) connection may fail with “Connection Refused” if the Managing Server’s(MS) host name can resolve to an IP address that cannot reach the MS.

CAUSE
The DC obtains the MS kernel’s network address from the MS. If a host name is specified for the kernel, and any of it’s resolved IP addresses cannot reach the MS from the DC, ITCAM’s connection attempt will fail if one of these "invalid” IP’s is used.

RESOLVING THE PROBLEM
Instead of host name, specify a “valid” IP address for the MS in the <MS_HOME>/bin/setenv.sh file, and in ITCAM 6.1, the ms.properties file. This solution usually works except when there is a firewall with Network Address Translation (NAT) enabled. In that case, you must make network changes so that all the resolved IPs can reach the MS.

Change as directed:

ITCAM 6.0 ( or earlier )
In MS_home/bin/setenv.sh:
KERNEL_HOST01=<ip addr of MS>
KERNEL_HOST02=<ip addr of MS>

ITCAM 6.1
In MS_home/bin/setenv.sh:
KERNEL_HOST01=<ip addr of MS>
am_socket_bindip=<ip addr of MS>
am_socket_exportip=<ip addr of MS>
In MS_HOME/etc/ms.properties, modify “kernel.hosts” specifying MS IP address:
kernel.hosts=<ip addr of MS>CBSPort:RFSPort:RMIPort - usually -
kernel.hosts=<ip addr of MS>9122:9120:9118

Example:
kernel.hosts=9.12.20.220:9122:9120:9118

Example of the error message:
This example is from z/OS, but the problem can occur on any platform.

<PPECONTROLLER, 60014b53-f7e9-db01-4272-2a9f50388c89.511,
9.12.20.161> Unable to join Kernel WSRT7.pdl.pok.ibm.com:9120 -
java.net.ConnectException: EDC8128I Connection refused.
(errno2=0x76630291); nested exception is:
java.net.ConnectException: EDC8128I Connection refused.
(errno2=0x76630291)

To determine IP addresses for a given MS host name [issue on the DC]:
nslookup <MS_host_name>

To test basic connectivity, for each IP address listed by NSLOOKUP, issue:
ping <ip_address>

Question: Why is Connection Refused on the ITCAM Managing Server? ITCAM Data Collector connection to the Managing Server fails with Connection Refused.

Answer: Instead of host name, specify a “valid” IP address for the MS in the <MS_HOME>/bin/setenv.sh file, and in ITCAM 6.1, the ms.properties file.


IBM Security Bulletin: Multiple vulnerabilities in IBM Java Runtime affect WebSphere Message Broker and IBM Integration Bus - United States WMB IIB SECURITY BULLETIN

SUMMARY
 Multiple security vulnerabilities exist in the IBM® Runtime Environment Java™ Technology Edition Version 6 Service Refresh 16 Fix Pack 41 and earlier releases used by WebSphere Message Broker, and the IBM® Runtime Environment Java™ Technology Edition Version 7 Service Refresh 10 Fix Pack 1 and earlier releases used by WebSphere Message Broker and IBM Integration Bus, and the IBM® Runtime Environment Java™ Technology Edition Version 7R1 Service Refresh 4 Fix Pack 1 and earlier releases used by IBM Integration Bus. These issues were disclosed as part of the IBM Java SDK updates in April 2017. WebSphere Message Broker and IBM Integration Bus have addressed the applicable CVEs

VULNERABILITY DETAILS
If you run your own Java code using the IBM Java Runtime delivered with this product, you should evaluate your code to determine whether the complete list of vulnerabilities are applicable to your code. For a complete list of vulnerabilities, refer to the “IBM Java SDK Security Bulletin”, located in the References section for more information.

CVEID: CVE-2017-3511 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-3511]
DESCRIPTION: An unspecified vulnerability in Oracle Java SE related to the Java SE, Java SE Embedded, JRockit JCE component could allow an unauthenticated attacker to take control of the system.
CVSS Base Score: 7.7
CVSS Temporal Score: See https://exchange.xforce.ibmcloud.com/vulnerabilities/124890 [https://exchange.xforce.ibmcloud.com/vulnerabilities/124890] for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:L/AC:H/PR:N/UI:R/S:C/C:H/I:H/A:H)

CVEID: CVE-2017-1289 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-1289]
DESCRIPTION: IBM SDK, Java Technology Edition is vulnerable XML External Entity Injection (XXE) error when processing XML data. A remote attacker could exploit this vulnerability to expose highly sensitive information or consume memory resources.
CVSS Base Score: 8.2
CVSS Temporal Score: See https://exchange.xforce.ibmcloud.com/vulnerabilities/125150 [https://exchange.xforce.ibmcloud.com/vulnerabilities/125150] for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:L)

CVEID: CVE-2016-9840 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-9840]
DESCRIPTION: zlib is vulnerable to a denial of service, caused by an out-of-bounds pointer arithmetic in inftrees.c. By persuading a victim to open a specially crafted document, a remote attacker could exploit this vulnerability to cause a denial of service.
CVSS Base Score: 3.3
CVSS Temporal Score: See https://exchange.xforce.ibmcloud.com/vulnerabilities/120508 [https://exchange.xforce.ibmcloud.com/vulnerabilities/120508] for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:L)

CVEID: CVE-2016-9841 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-9841]
DESCRIPTION: zlib is vulnerable to a denial of service, caused by an out-of-bounds pointer arithmetic in inftrees.c. By persuading a victim to open a specially crafted document, a remote attacker could exploit this vulnerability to cause a denial of service.
CVSS Base Score: 3.3
CVSS Temporal Score: See https://exchange.xforce.ibmcloud.com/vulnerabilities/120509 [https://exchange.xforce.ibmcloud.com/vulnerabilities/120509] for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:L)

CVEID: CVE-2016-9842 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-9842]
DESCRIPTION: zlib is vulnerable to a denial of service, caused by an undefined left shift of negative number. By persuading a victim to open a specially crafted document, a remote attacker could exploit this vulnerability to cause a denial of service.
CVSS Base Score: 3.3
CVSS Temporal Score: See https://exchange.xforce.ibmcloud.com/vulnerabilities/120510 [https://exchange.xforce.ibmcloud.com/vulnerabilities/120510] for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:L)

CVEID: CVE-2016-9843 [http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-9843]
DESCRIPTION: zlib is vulnerable to a denial of service, caused by a big-endian out-of-bounds pointer. By persuading a victim to open a specially crafted document, a remote attacker could exploit this vulnerability to cause a denial of service.
CVSS Base Score: 3.3
CVSS Temporal Score: See https://exchange.xforce.ibmcloud.com/vulnerabilities/120511 [https://exchange.xforce.ibmcloud.com/vulnerabilities/120511] for the current score
CVSS Environmental Score*: Undefined
CVSS Vector: (CVSS:3.0/AV:L/AC:L/PR:N/UI:R/S:U/C:N/I:N/A:L)

AFFECTED PRODUCTS AND VERSIONS
IBM Integration Bus V10.0.0.0- 10.0.0.8

IBM Integration Bus V9.0.0.0- 9.0.0.7

WebSphere Message Broker V8.0.0.0 - 8.0.0.8

REMEDIATION/FIXES

Product VRMF APAR Remediation/Fix IBM Integration Bus V10.0.0.0- 10.0.0.8 IT20410 The APAR is available in fix pack 10.0.0.9 (on all platforms except HP) http://www-01.ibm.com/support/docview.wss?uid=swg24043686 [http://www-01.ibm.com/support/docview.wss?uid=swg24043686]

IBM Integration Bus V9.0.0.0- 9.0.0.7 IT20410 The APAR is available in fix pack 9.0.0.8 (on all platforms except HP) http://www-01.ibm.com/support/docview.wss?uid=swg24043751 [http://www-01.ibm.com/support/docview.wss?uid=swg24043751]

WebSphere Message Broker V8.0.0.0 - 8.0.0.8 IT20410 The APAR is available in fix pack 8.0.0.9 (on all platforms except HP) https://www.ibm.com/support/docview.wss?uid=swg24043806 [https://www.ibm.com/support/docview.wss?uid=swg24043806]

For unsupported versions of the product IBM recommends upgrading to a fixed, supported version/release/platform of the product.
The planned maintenance release dates for WebSphere Message Broker and IBM Integration Bus are available at :
http://www.ibm.com/support/docview.wss?uid=swg27006308 [http://www.ibm.com/support/docview.wss?uid=swg27006308]

GET NOTIFIED ABOUT FUTURE SECURITY BULLETINS
 Subscribe to My Notifications [ http://www-01.ibm.com/software/support/einfo.html] to be notified of important product support alerts like this.

IMPORTANT NOTE
 IBM strongly suggests that all System z customers be subscribed to the System z Security Portal to receive the latest critical System z security and integrity service. If you are not subscribed, see the instructions on the System z Security web site [http://www.ibm.com/systems/z/solutions/security_subintegrity.html]. Security and integrity APARs and associated fixes will be posted to this portal. IBM suggests reviewing the CVSS scores and applying all security or integrity fixes as soon as possible to minimize any potential risk.

REFERENCES
Complete CVSS v3 Guide [http://www.first.org/cvss/user-guide]
On-line Calculator v3 [http://www.first.org/cvss/calculator/3.0]
IBM Java SDK Security Bulletin [http://www.ibm.com/support/docview.wss?uid=swg22002169]
RELATED INFORMATION
IBM Secure Engineering Web Portal [http://www.ibm.com/security/secure-engineering/bulletins.html]
IBM Product Security Incident Response Blog [http://www.ibm.com/blogs/psirt]

CHANGE HISTORY
 30 Jun 2017 - Original version Published

*The CVSS Environment Score is customer environment specific and will ultimately impact the Overall CVSS Score. Customers can evaluate the impact of this vulnerability in their environments by accessing the links in the Reference section of this Security Bulletin.

DISCLAIMER
According to the Forum of Incident Response and Security Teams (FIRST), the Common Vulnerability Scoring System (CVSS) is an "industry open standard designed to convey vulnerability severity and help to determine urgency and priority of response." IBM PROVIDES THE CVSS SCORES "AS IS" WITHOUT WARRANTY OF ANY KIND, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. CUSTOMERS ARE RESPONSIBLE FOR ASSESSING THE IMPACT OF ANY ACTUAL OR POTENTIAL SECURITY VULNERABILITY.

RELATED INFORMATION
#

Cross reference information Segment Product Component Platform Version Edition Business Integration WebSphere Message Broker 8.0
PRODUCT ALIAS/SYNONYM
 WMB IIB

Question: Help with Security Bulletin: Multiple vulnerabilities in IBM Java Runtime affect WMB and IIB I need to understand details regarding Security Bulletin: Multiple vulnerabilities in IBM Java Runtime affect WebSphere Message Broker and IBM Integration Bus. Where can I find this information?

Answer: If you run your own Java code using the IBM Java Runtime delivered with this product, you should evaluate your code to determine whether the complete list of vulnerabilities are applicable to your code. For a complete list of vulnerabilities, refer to the “IBM Java SDK Security Bulletin”, located in the References section for more information.