SSL/HTTPS - problems with kdb files

I'm active Experts Exchange contributor, and there's this SSL/kdb problem. I indulge myself into publishing my comment to one of the questions (http://www.experts-exchange.com/Networking/Protocols/Application_Protocols/SSL/Q_27794894.html) here (with some edits):

It's on CMS format (IBM Key Database file) and problems with opening it with your ikeyman tool (with WebSphere java):

• for CMS it is IBM proprietary format (like LTPA) and is not available in non-IBM JRE/JDKs, BUT I also had this issue that WAS's JDK could not open CMS (kdb) files - can't really say why as I did not troubleshoot it. but the workaround that worked for me was to run ikeyman not from /opt/IBM/WebSphere/AppServ

er/java but from different WAS package JRE - like UpdateInstaller or InstallationManager - I'm sure you have either installed on your machine so try them.
I just now checked how it looks like when running ikeyman from: C:\Program Files (x86)\IBM\WebSphere\AppServer\java\jre\bin I can operate on CMS files but when running  from: C:\Program Files (x86)\IBM\Java60\jre\bin I can't, so it might be something with your java paths. If you can't figure it out, try the workaround I suggested above (UI or IM java)

• difference between kdb and p12 is - at least this is "emiprical" difference experienced by me - that kdb usually houses many certificates (signer&personal) for use by applications, whereas p12 is usually used to carry one certificate from an issuer to the owner (for instance I get my corporate certificate in p12 from supplier). just "any" java's keytool or any gsk7 won't be able to open kdb file, it must me somewhere near ;) WebSphere  

• if you use kdb file for your IHS, don't forget to indicate your certificate as "default" in the kdb file. I was looking for the way to set cert alias to use from within httpd.conf file, but it seems to be impossible

• I thought that IHS uses ONLY kdb database to get certifcates from but I just found that you may simply supply crt file - PEM encoded (example: http://rimuhosting.com/howto/modssl.jsp) 

some reference - different product, but usage of gsk7 commands is given:
http://pic.dhe.ibm.com/inf

ocenter/tivihelp/v3r1/index.jsp?topic=%2Fcom.ibm.tivoli.itws.doc_8.5.1%2Ffipsensurenetwork.htm

Java exception: org.omg.CORBA.BAD_INV_ORDER

It may happen that all of a sudden you will see the following exception in your SystemOut.log of your managed server (that is the one managed from deployment manager console via nodeagent process). That may occur during either server run or (more often) during server startup attempted from shell level.

[1/25/12 11:25:36:870 CET] 0000002c WSSecurityCon A   Error creating client_auth_token in initSecContext, reason: Major Code[15] Minor Code[0] Message[ create_gss_initial_context_token: Java exception: org.omg.CORBA.BAD_INV_ORDER: ORB has been shut down  vmcid: OMG  minor code: 4  completed: No]
[1/25/12 11:25:36:884 CET] 0000002c IPCConnectorC E   ADMC0076E: Failed to get security token due to Major Code[15] Minor Code[0] Message[ create_gss_initial_context_token: Java exception: org.omg.CORBA.BAD_INV_ORDER: ORB has been shut down  vmcid: OMG  minor code: 4  completed: No]
        at com.ibm.ISecurityUtilityImpl.CSIUtil.create_gss_initial_context_token(CSIUtil.java:979)
        at com.ibm.ISecurityLocalObjectTokenBaseImpl.WSSecurityContextLTPAImpl.initSecContext(WSSecurityContextLTPAImpl.java:179)
        at com.ibm.ISecurityLocalObjectTokenBaseImpl.WSSecurityContextLTPAImpl.initSecContext(WSSecurityContextLTPAImpl.java:133)
        at com.ibm.ws.management.connector.ipc.IPCConnectorClient.fillInCredential(IPCConnectorClient.java:307)
        at com.ibm.ws.management.connector.ipc.IPCConnectorClient.queryNames(IPCConnectorClient.java:792)
        at com.ibm.ws.management.AdminClientImpl.queryNames(AdminClientImpl.java:108)
        at com.ibm.ws.management.discovery.transport.MBeanDiscoveryHelper.discovery(MBeanDiscoveryHelper.java:507)
        at com.ibm.ws.management.discovery.transport.MulticastServer.rediscover(MulticastServer.java:406)
        at com.ibm.ws.management.discovery.transport.MulticastServer.run(MulticastServer.java:279)
        at java.lang.Thread.run(Thread.java:736)

It usually means that your server has lost communication with it's nodeagent, and the reasons for that may be numerous, but the most common are:

• nodeagent process crashed or was killed any other way - that is the easiest: just bring up back the nodeagent
• something went wrong with network interfaces (that's more applicable to startup problem) - in that case you need to try to bring up/down lo interface (read here) or check if in the meantime your hostname or ip address hadn't been changed (little chance, but you never know)
• if above fail, check if nodeagent NDS (node discovery service) port, among with other ports (soap etc.) are locally available with telnet. if not, you need to diagnose further system's ip sockets setup.

Good luck!

WebSphere JSSL0080E: javax.net.ssl.SSLHandshakeException - The client and server could not negotiate the desired level of security.

If you see the following exception while or after starting your server (nodeagent or app server): 

JSSL0080E: javax.net.ssl.SSLHandshakeException - The client and server could not negotiate the desired level of security. 

then big chance is that there's something wrong with internal certificates between WAS instances, either:
1. Hostname has been changed and hostname verification is turned on in SSL setup. you may check this here:
http://www.ibm.com/developerworks/websphere/techjournal/0612_birk/0612_birk.html
in paragraph:

The default trust manager, IbmX509, performs fundamental certificate validation, including the certificate signature validation (ensuring it has not been modified) and certificate expiration validation (ensuring it has not expired). This trust manager does not perform hostname verification by default, although you can set the com.ibm.ssl.performURLHostNameVerification=true property in the security custom properties to enable this function for URL connections only. If this is done, the trust manager will ensure that for URL connections, the hostname specified on the connection matches the SubjectDN in the certificate returned by the server (just as Web browsers do).
(recommend reading the whole article btw.)

2. Something went horribly wrong in terms of cell synchronization, so I'd suggest stopping servers and nodeagent on remote machine and running ./syncNode.sh from profile's bin directory. not sure it will work as probably wsadmin script client won't be able to connect to SOAP port due to root cause. hence, we're back at point 1. and I guess only manual copying of trust/keystores from dmgr machine to remote machine might help.

hope this helps, or at least points to the right direction.

WebSphere: disable SSL between IHS and WAS (IHS SSL offload)

Ever wondered how to disable SSL between IHS and WebSphere App Server? In some environments, your IHS and WAS remain in secure zone, and you don't want to spend any CPU time for doing SSL between web server and backend server.
In case of IBM HTTP Server + WAS combination there's a simple way to do it:

• assuming you have Plugin already configured for IHS, you need to edit it's config (plugin-cfg.xml) with some editor eg. vi

• find section for your backend server cluster you want to SSL off-load:

<ServerCluster CloneSeparatorChange="false" GetDWLMTable="false" IgnoreAffinityRequests="true" LoadBalance="Round Robin" Name="clustername" PostBufferSize="64" PostSizeLimit="-1" RemoveSpecialHeaders="true" RetryInterval="60">
      <Server ConnectTimeout="5" ExtendedHandshake="false" MaxConnections="-1" Name="PTS2CSIP01Node_STS2CSIP01S01" ServerIOTimeout="60" WaitForContinue="false">
         <Transport Hostname="backend_hostname" Port="4300" Protocol="http"/>
         <Transport Hostname="backend_hostname" Port="4343" Protocol="https">
            <Property Name="keyring" Value="/opt/ibm/HTTPServer/Plugins/config/plgcfgname/plugin-key.kdb"/>
            <Property Name="stashfile" Value="/opt/ibm/HTTPServer/Plugins/config/plgcfgname/plugin-key.sth"/>
         </Transport>
      </Server>
   </ServerCluster>

 and comment out second Transport definition to:

      <!--Transport Hostname="backend_hostname" Port="4343" Protocol="https">
            <Property Name="keyring" Value="/opt/ibm/HTTPServer/Plugins/config/plgcfgname/plugin-key.kdb"/>
            <Property Name="stashfile" Value="/opt/ibm/HTTPServer/Plugins/config/plgcfgname/plugin-key.sth"/>
      </Transport-->
 

• restart your IHS server and check in error_log if plugin has been loaded properly 

To check, you may turn on plugin tracing:

<Log LogLevel="Trace" Name="/opt/ibm/HTTPServer/Plugins/logs/plgcfgname/http_plugin.log"/>

Restart IHS once again and tail  http_plugin.log while sending request to your https port of IHS. You shall get trace like that:
TRACE: ws_common: websphereFindTransport: Finding the transport
DETAIL: ws_common: websphereFindTransport: Setting the transport(case 3): backend_hostname on port 4300 <--- PLEASE NOTE THIS IS HTTP PORT OF WAS!
TRACE: ws_common: websphereExecute: Executing the transaction with the app server reqInfo is OKuseExistingStream=0, client->stream=(nil)
DEBUG: ws_common: websphereGetStream: Getting the stream to the app server
TRACE: ws_transport: transportStreamDequeue: Checking for existing stream from the queue
TRACE: ws_common: websphereGetStream: Have a connect timeout of 5; Setting socket to not block for the connect
TRACE: ws_common: websphereGetStream: Reseting socket to block
TRACE: ws_common: websphereGetStream: Setting socket to non-block for ServerIOTimeout over HTTP
DEBUG: ws_common: websphereGetStream: socket 13 connected to backend_hostname:4300
DEBUG: lib_stream: openStream: Opening the stream
DETAIL: ws_common: websphereGetStream: Created a new stream; queue was empty, socket = 13

Any questions? Good Luck!!! Feel free to comment&leave feedback.