Error while regenerating application token when IS is acting as key manager

Tried the scenario in following version :
WSO2 Identity Server 5.0.0
WSO2 API Manger 1.7.1

In this scenario WSO2 IS acts as key manager and I was following following guide
https://docs.wso2.com/display/CLUSTER420/Configuring+WSO2+Identity+Server+as+the+Key+Manager

After all the configuration is done, created an API, published it, subscribed and created an application token.

When regenerating the token I observed exact issue reported at https://wso2.org/jira/browse/APIMANAGER-2738

[2014-09-09 11:12:04,737] ERROR - APIStoreHostObject Error in getting new accessToken
org.apache.axis2.AxisFault: Error in getting new accessToken
    at org.apache.axis2.util.Utils.getInboundFaultFromMessageContext(Utils.java:531)
    at org.apache.axis2.description.OutInAxisOperationClient.handleResponse(OutInAxisOperation.java:370)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:445)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
    at org.wso2.carbon.apimgt.keymgt.stub.subscriber.APIKeyMgtSubscriberServiceStub.renewAccessToken(APIKeyMgtSubscriberServiceStub.java:1187)
    at org.wso2.carbon.apimgt.keymgt.client.SubscriberKeyMgtClient.regenerateApplicationAccessKey(SubscriberKeyMgtClient.java:83)
    at org.wso2.carbon.apimgt.hostobjects.APIStoreHostObject.jsFunction_refreshToken(APIStoreHostObject.java:3219)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
    at org.mozilla.javascript.FunctionObject.call(FunctionObject.java:386)
    at org.mozilla.javascript.optimizer.OptRuntime.callN(OptRuntime.java:52)
    at org.jaggeryjs.rhino.store.modules.subscription.c4._c_anonymous_3(/store/modules/subscription/key.jag:61)
    at org.jaggeryjs.rhino.store.modules.subscription.c4.call(/store/modules/subscription/key.jag)
    at org.mozilla.javascript.ScriptRuntime.applyOrCall(ScriptRuntime.java:2430)
    at org.mozilla.javascript.BaseFunction.execIdCall(BaseFunction.java:269)
    at org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97)
    at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
    at org.jaggeryjs.rhino.store.modules.subscription.c0._c_anonymous_10(/store/modules/subscription/module.jag:35)
    at org.jaggeryjs.rhino.store.modules.subscription.c0.call(/store/modules/subscription/module.jag)
    at org.mozilla.javascript.optimizer.OptRuntime.callN(OptRuntime.java:52)
    at org.jaggeryjs.rhino.store.site.blocks.subscription.subscription_add.ajax.c0._c_anonymous_1(/store/site/blocks/subscription/subscription-add/ajax/subscription-add.jag:206)
    at org.jaggeryjs.rhino.store.site.blocks.subscription.subscription_add.ajax.c0.call(/store/site/blocks/subscription/subscription-add/ajax/subscription-add.jag)
    at org.mozilla.javascript.optimizer.OptRuntime.call0(OptRuntime.java:23)
    at org.jaggeryjs.rhino.store.site.blocks.subscription.subscription_add.ajax.c0._c_script_0(/store/site/blocks/subscription/subscription-add/ajax/subscription-add.jag:3)
    at org.jaggeryjs.rhino.store.site.blocks.subscription.subscription_add.ajax.c0.call(/store/site/blocks/subscription/subscription-add/ajax/subscription-add.jag)
    at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
    at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
    at org.jaggeryjs.rhino.store.site.blocks.subscription.subscription_add.ajax.c0.call(/store/site/blocks/subscription/subscription-add/ajax/subscription-add.jag)
    at org.jaggeryjs.rhino.store.site.blocks.subscription.subscription_add.ajax.c0.exec(/store/site/blocks/subscription/subscription-add/ajax/subscription-add.jag)
    at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:570)
    at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273)
    at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:432)
    at org.jaggeryjs.jaggery.core.JaggeryServlet.doPost(JaggeryServlet.java:29)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
    at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
    at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379)
    at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
    at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
    at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
[2014-09-09 11:12:04,749] ERROR - subscription-add:jag org.wso2.carbon.apimgt.api.APIManagementException: Error in getting new accessToken

The solution is I've missed following point in the guide...

Copy the section from the identity.xml file in the API Manager (/repository/conf/identity.xml) and replace the section of the identity.xml file of the IS (/repository/conf/identity.xml).

Thought of sharing this hoping that someone will come across the same issue :)

Common pitfalls in Kerberos Authentication Using WSO2 Products

Recently I was going through the article written by Amilj on "Kerberos Authentication Using WSO2 Products" and ran into some issues. Since theses are common mistakes, thought of putting my solutions in this blog post so that this will be useful to anyone who's trying above article

Following are the WSO2 products I used

• WSO2 Identity Server 5.0.0 (Soon to be released with lots of new and cool features :) )
• WSO2 ESB 4.8.1

After setting up everything according to the article, and when running the client I came across following issues.

Issue 01 :

[java] log4j:WARN No appenders could be found for logger (org.apache.axiom.om.util.StAX
Utils).
     [java] log4j:WARN Please initialize the log4j system properly.
     [java] Calling Echo service with parameter - Hello World
     [java] org.apache.axis2.AxisFault: Error in building kereberos token
     [java]     at org.apache.rampart.handler.RampartSender.invoke(RampartSender.java:76)
     [java]     at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
     [java]     at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
     [java]     at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
     [java]     at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:426)
     [java]     at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
     [java]     at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
     [java]     at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
     [java]     at org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:554)
     [java]     at org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:530)
     [java]     at org.wso2.identity.esb.kerberos.KerberosClient.run(KerberosClient.java:111)
     [java]     at org.wso2.identity.esb.kerberos.KerberosClient.main(KerberosClient.java:43)
     [java] Caused by: org.apache.rampart.RampartException: Error in building kereberos token
     [java]     at org.apache.rampart.builder.BindingBuilder.getKerberosTokenBuilder(BindingBuilder.java:936)
     [java]     at org.apache.rampart.builder.SymmetricBindingBuilder.doSignBeforeEncrypt(SymmetricBindingBuilder.java:425)
     [java]     at org.apache.rampart.builder.SymmetricBindingBuilder.build(SymmetricBindingBuilder.java:86)
     [java]     at org.apache.rampart.MessageBuilder.build(MessageBuilder.java:144)
     [java]     at org.apache.rampart.handler.RampartSender.invoke(RampartSender.jav


Observed above exception in java client side.

Solution :

- Checked wso2carbon.log and found some warnings

TID: [0] [IS] [2014-04-22 22:18:31,628]  WARN {org.apache.directory.server.kerberos.shared.store.operations.StoreUtils} -  No server entry found for kerberos principal name pavithra@WSO2.ORG {org.apache.directory.server.kerberos.shared.store.operations.StoreUtils}
TID: [0] [IS] [2014-04-22 22:18:31,629]  WARN {org.apache.directory.server.kerberos.protocol.KerberosProtocolHandler} -  Client not found in Kerberos database (6) {org.apache.directory.server.kerberos.protocol.KerberosProtocolHandler}


If you need more details enable debug logs by adding following entry in IS_HOME/repository/conf/log4j.properties

log4j.logger.org.apache.directory.server.kerberos=DEBUG

- But I've created client principal correctly

- So checked the KDC (i.e. Embedded LDAP in Identity Server) through Apache DS browser to make sure that the client principal name has been added correctly.

 - There I observed that "_carbon.super"  has been added to client principal name. So when I added a user named pavithra through users and roles, the client principal name has become pavithra_carbon.super@WSO2.ORG

- Change following property in policy-1.xml in java client

Previously it was :



Change it to :



- Now if you again run the client, it should be able to successfully return the response from the secured service

- But I came across another issue :)

Issue 02 :

- When running the java client, it threw following exception

org.apache.axis2.AxisFault: Connection has been shutdown: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
    at org.apache.axis2.AxisFault.makeFault(AxisFault.java:430)
    at org.apache.axis2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:78)
    at org.apache.axis2.transport.http.AxisRequestEntity.writeRequest(AxisRequestEntity.java:84)
    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:499)
    at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
    at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
    at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:622)
    at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:193)
    at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
    at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:451)
    at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:278)
    at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:430)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:225)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:149)
    at org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:554)
    at org.apache.axis2.client.ServiceClient.sendReceive(ServiceClient.java:530)
    at org.wso2.identity.esb.kerberos.KerberosClient.main(KerberosClient.java:91)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: com.ctc.wstx.exc.WstxIOException: Connection has been shutdown: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
    at com.ctc.wstx.sw.BaseStreamWriter.finishDocument(BaseStreamWriter.java:1692)
    at com.ctc.wstx.sw.BaseStreamWriter.close(BaseStreamWriter.java:288)
    at org.apache.axiom.util.stax.wrapper.XMLStreamWriterWrapper.close(XMLStreamWriterWrapper.java:46)
    at org.apache.axiom.om.impl.MTOMXMLStreamWriter.close(MTOMXMLStreamWriter.java:188)
    at org.apache.axiom.om.impl.dom.NodeImpl.serializeAndConsume(NodeImpl.java:844)
    at org.apache.axis2.transport.http.SOAPMessageFormatter.writeTo(SOAPMessageFormatter.java:74)
    ... 24 more
Caused by: javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1325)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1337)
    at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:44)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
    at java.io.BufferedOutputStream.write(BufferedOutputStream.java:104)
    at org.apache.commons.httpclient.WireLogOutputStream.write(WireLogOutputStream.java:68)
    at org.apache.commons.httpclient.ChunkedOutputStream.flushCacheWithAppend(ChunkedOutputStream.java:121)
    at org.apache.commons.httpclient.ChunkedOutputStream.write(ChunkedOutputStream.java:179)
    at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:96)
    at com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:214)
    at com.ctc.wstx.sw.BufferingXmlWriter.close(BufferingXmlWriter.java:194)
    at com.ctc.wstx.sw.BaseStreamWriter.finishDocument(BaseStreamWriter.java:1690)
    ... 29 more
Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
    at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:190)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1692)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1675)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1601)
    at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:94)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
    at java.io.BufferedOutputStream.write(BufferedOutputStream.java:104)
    at org.apache.commons.httpclient.WireLogOutputStream.write(WireLogOutputStream.java:68)
    at org.apache.commons.httpclient.ChunkedOutputStream.flushCacheWithAppend(ChunkedOutputStream.java:121)
    at org.apache.commons.httpclient.ChunkedOutputStream.write(ChunkedOutputStream.java:179)
    at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:96)
    at com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:214)
    at com.ctc.wstx.sw.BaseStreamWriter.flush(BaseStreamWriter.java:311)
    at org.apache.axiom.util.stax.wrapper.XMLStreamWriterWrapper.flush(XMLStreamWriterWrapper.java:50)
    at org.apache.axiom.om.impl.MTOMXMLStreamWriter.flush(MTOMXMLStreamWriter.java:198)
    at org.apache.axiom.om.impl.dom.NodeImpl.serializeAndConsume(NodeImpl.java:842)
    ... 25 more
Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
    at sun.security.validator.PKIXValidator.(PKIXValidator.java:57)
    at sun.security.validator.Validator.getInstance(Validator.java:161)
    at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.getValidator(X509TrustManagerImpl.java:108)
    at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:204)
    at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:249)
    at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1185)
    at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
    at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
    at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:637)
    at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:89)
    ... 36 more
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty
    at java.security.cert.PKIXParameters.setTrustAnchors(PKIXParameters.java:183)
    at java.security.cert.PKIXParameters.(PKIXParameters.java:103)
    at java.security.cert.PKIXBuilderParameters.(PKIXBuilderParameters.java:87)
    at sun.security.validator.PKIXValidator.(PKIXValidator.java:55)
    ... 48 more
Could not create service client

Solution :

- Obviously the truststore path was invlid :)

Now everything looks fine. So ran the client and again got an error in ESB side

Issue 03 :

- Observed following error on ESB console.

TID: [0] [ESB] [2014-04-22 14:32:20,151] ERROR {org.apache.ws.security.processor.KerberosTokenProcessor} -  java.lang.IllegalStateException: Cannot get keys at REQ_OK state
    at sun.security.krb5.KrbAsReqBuilder.checkState(KrbAsReqBuilder.java:398)
    at sun.security.krb5.KrbAsReqBuilder.getKeys(KrbAsReqBuilder.java:155)
    at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:724)
    at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:580)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
    at org.apache.ws.security.processor.KerberosTokenProcessor.authenticate(KerberosTokenProcessor.java:433)
    at org.apache.ws.security.processor.KerberosTokenProcessor.verifyXMLSignature(KerberosTokenProcessor.java:294)
    at org.apache.ws.security.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:308)
    at org.apache.ws.security.processor.SignatureProcessor.handleToken(SignatureProcessor.java:124)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:332)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:249)
    at org.apache.rampart.RampartEngine.process(RampartEngine.java:187)
    at org.apache.rampart.handler.RampartReceiver.invoke(RampartReceiver.java:92)
    at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
    at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
    at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
 {org.apache.ws.security.processor.KerberosTokenProcessor}
javax.security.auth.login.LoginException: java.lang.IllegalStateException: Cannot get keys at REQ_OK state
    at sun.security.krb5.KrbAsReqBuilder.checkState(KrbAsReqBuilder.java:398)
    at sun.security.krb5.KrbAsReqBuilder.getKeys(KrbAsReqBuilder.java:155)
    at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:724)
    at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:580)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
    at org.apache.ws.security.processor.KerberosTokenProcessor.authenticate(KerberosTokenProcessor.java:433)
    at org.apache.ws.security.processor.KerberosTokenProcessor.verifyXMLSignature(KerberosTokenProcessor.java:294)
    at org.apache.ws.security.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:308)
    at org.apache.ws.security.processor.SignatureProcessor.handleToken(SignatureProcessor.java:124)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:332)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:249)
    at org.apache.rampart.RampartEngine.process(RampartEngine.java:187)
    at org.apache.rampart.handler.RampartReceiver.invoke(RampartReceiver.java:92)
    at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
    at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
    at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:886)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
    at org.apache.ws.security.processor.KerberosTokenProcessor.authenticate(KerberosTokenProcessor.java:433)
    at org.apache.ws.security.processor.KerberosTokenProcessor.verifyXMLSignature(KerberosTokenProcessor.java:294)
    at org.apache.ws.security.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:308)
    at org.apache.ws.security.processor.SignatureProcessor.handleToken(SignatureProcessor.java:124)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:332)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:249)
    at org.apache.rampart.RampartEngine.process(RampartEngine.java:187)
    at org.apache.rampart.handler.RampartReceiver.invoke(RampartReceiver.java:92)
    at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
    at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
    at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
TID: [0] [ESB] [2014-04-22 14:32:20,154] ERROR {org.apache.axis2.engine.AxisEngine} -  An error was discovered processing the header (Kerberos login failed) {org.apache.axis2.engine.AxisEngine}
org.apache.axis2.AxisFault: An error was discovered processing the header (Kerberos login failed)
    at org.apache.rampart.handler.RampartReceiver.setFaultCodeAndThrowAxisFault(RampartReceiver.java:180)
    at org.apache.rampart.handler.RampartReceiver.invoke(RampartReceiver.java:95)
    at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
    at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
    at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
    at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:172)
    at org.apache.axis2.transport.http.AxisServlet.doPost(AxisServlet.java:146)
    at org.wso2.carbon.core.transports.CarbonServlet.doPost(CarbonServlet.java:231)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
    at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:68)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
    at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
    at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:49)
    at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
    at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
    at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.ws.security.WSSecurityException: An error was discovered processing the header (Kerberos login failed)
    at org.apache.ws.security.processor.KerberosTokenProcessor.verifyXMLSignature(KerberosTokenProcessor.java:332)
    at org.apache.ws.security.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:308)
    at org.apache.ws.security.processor.SignatureProcessor.handleToken(SignatureProcessor.java:124)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:332)
    at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:249)
    at org.apache.rampart.RampartEngine.process(RampartEngine.java:187)
    at org.apache.rampart.handler.RampartReceiver.invoke(RampartReceiver.java:92)
    ... 38 more

Solution :

- According to http://sourceforge.net/p/spnego/discussion/1003769/thread/167a25ec/, change java version from 7 to 6

When java 6 is used, the client runs without any errors and retruns the correcct response :)

Hope this will be helpful for anyone who's trying the article "Kerberos Authentication Using WSO2 Products"