[jacorb-developer] Client Application of Jacorb3.0 hangs on IBM WebSphere
Phil Mesnier
mesnierp at objectcomputing.com
Thu Mar 23 12:21:14 CET 2017
Hi Prakash,
Please note that this community discussion board is a low priority channel.
If you are looking for commercial grade support, please contact one of the
providers listed on jacorb.org. I know that Avaya has worked with OCI in
the past.
Regarding known issues in JacORB 3.0, I don't know about implicit
deadlocks, but the request processor's thread pool management code was
refactored in JacORB 3.7 specifically to improve performance in high volume
uses.
You should get the latest release of JacORB and use that to try your
scenario again. If you still have trouble then we should take a deeper look.
Best regards,
Phil
On Tue, Mar 21, 2017 at 11:28 AM, Rungta, Prakash (Prakash) <
prungta at avaya.com> wrote:
> Hello Team,
>
> Would appreciate if anyone can look into this.
>
> Thanks,
> Prakash R
>
>
>
> _____________________________________________
> From: Rungta, Prakash (Prakash)
> Sent: Sunday, March 19, 2017 11:51 PM
> To: jacorb-developer at lists.spline.inf.fu-berlin.de
> Cc: Siddique, Md Anjum (Anjum); Rashinkar, Anand (Anand)
> Subject: Client Application of Jacorb3.0 hangs on IBM WebSphere
>
>
> Hello Team,
>
> I am developing a Jacorb 3.0 based Application (Client) . However during
> traffic conditions(Medium) the application(client application) hangs . I
> see the following errors in the client logs (Jacorb Logging increased)
> during start and at random times as well.
>
> 2017-03-19 16:19:28,843 [ClientMessageReceptor1] jacorb.poa.queue WARN -
> Request queue is full, consider increasing jacorb.poa.queue_max (currently:
> 100)
> 2017-03-19 16:19:28,844 [ClientMessageReceptor1] jacorb.poa.queue WARN -
> Request queue is full, consider increasing jacorb.poa.queue_max (currently:
> 100)
> 2017-03-19 16:19:28,844 [ClientMessageReceptor1] jacorb.poa.queue WARN -
> Request queue is full, consider increasing jacorb.poa.queue_max (currently:
> 100)
> 2017-03-19 16:19:28,844 [ClientMessageReceptor1] jacorb.poa.queue WARN -
> Request queue is full, consider increasing jacorb.poa.queue_max (currently:
> 100)
>
> 2017-03-19 16:19:28,845 [RequestController-2] jacorb.poa.controller WARN -
> Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max
> (currently: 20)
> 2017-03-19 16:19:28,845 [RequestController-2] jacorb.poa.controller WARN -
> Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max
> (currently: 20)
> 2017-03-19 16:19:28,845 [RequestController-2] jacorb.poa.controller WARN -
> Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max
> (currently: 20)
> 2017-03-19 16:19:28,845 [RequestController-2] jacorb.poa.controller WARN -
> Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max
> (currently: 20)
> 2017-03-19 16:19:28,846 [RequestController-2] jacorb.poa.controller WARN -
> Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max
> (currently: 20)
> 2017-03-19 16:19:28,846 [RequestController-2] jacorb.poa.controller WARN -
> Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max
> (currently: 20)
> 2017-03-19 16:19:28,846 [RequestController-2] jacorb.poa.controller WARN -
> Thread pool exhausted, consider increasing jacorb.poa.thread_pool_max
> (currently: 20)
>
> On the Server side , I see the CORBA TIMEOUT EXCEPTION being raised
>
> e._info=system exception, ID 'IDL:omg.org/CORBA/TIMEOUT:1.0'
>
> When the client application hangs , I have generated a javacore , I see
> that there are lot of Thread blocking . Also there are lot of
> RequestProcessor threads which are in blocking state
>
> 3LKWAITNOTIFY "RequestProcessor-53" (J9VMThread:
> 0x00000000A8742F00)
> 2LKMONINUSE sys_mon_t:0x00007F933628F2D0 infl_mon_t:
> 0x00007F933628F348:
> 3LKMONOBJECT org/jacorb/poa/RequestProcessor at 0x000000000B8E9B70:
> <unowned>
> 3LKNOTIFYQ Waiting to be notified:
>
> 3XMTHREADINFO "RequestProcessor-53" J9VMThread:0x00000000A8742F00,
> j9thread_t:0x00007F933DA77D30, java/lang/Thread:0x0000000006DC92C0,
> state:CW, prio=10
> 3XMJAVALTHREAD (java/lang/Thread getId:0x30FC92, isDaemon:true)
> 3XMTHREADINFO1 (native thread ID:0x7A47, native priority:0xA,
> native policy:UNKNOWN, vmstate:CW, vm thread flags:0x00000101)
> 3XMTHREADINFO2 (native stack address range
> from:0x00007F92B3BF6000, to:0x00007F92B3C37000, size:0x41000)
> 3XMCPUTIME CPU usage total: 0.128906816 secs
> 3XMTHREADBLOCK Waiting on: org/jacorb/poa/RequestProcessor at 0x0000000006DC92C0
> Owned by: <unowned>
> 3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
> 3XMTHREADINFO3 Java callstack:
> 4XESTACKTRACE at java/lang/Object.wait(Native Method)
> 4XESTACKTRACE at java/lang/Object.wait(Object.java:167(Compiled
> Code))
> 4XESTACKTRACE at org/jacorb/poa/RequestProcessor.run(
> RequestProcessor.java:793(Compiled Code))
> 5XESTACKTRACE (entered lock: org/jacorb/poa/
> RequestProcessor at 0x0000000006DC92C0, entry count: 1)
> 3XMTHREADINFO3 Native callstack:
> 4XENATIVESTACK (0x00007F9388B37912 [libj9prt26.so+0x13912])
>
> Application overview
> Server is a C++ CORBA based and Client is JAVA (Jacorb 3.0 based idl and
> runtime is both 3.0).Server does callback's to client application and
> client does processing.(For simplicity I have commented all the logic
> inside the client callback functions except some log statements during
> entering and leaving the functions.) still the issue is there.
>
> The client application is deployed on RHEL 6.5 (64 bit ) on IBM WebSphere
> (8.5.5.4) .
>
> Following are some of Jacorb Properties that are set
>
> jacorb.net.server_socket_factory.port.min=52000
> jacorb.net.server_socket_factory.port.max=53000
> jacorb.security.support_ssl=on
>
> jacorb.security.ssl.client.supported_options=40
> jacorb.security.ssl.client.required_options=40
>
> jacorb.security.ssl.server.supported_options=0
> jacorb.security.ssl.server.required_options=0
>
> jacorb.security.jsse.server.key_manager_algorithm=IbmX509
> jacorb.security.jsse.server.trust_manager_algorithm=IbmX509
> jacorb.security.jsse.client.key_manager_algorithm=IbmX509
> jacorb.security.jsse.client.trust_manager_algorithm=IbmX509
>
> jacorb.poa.thread_pool_max=20
> jacorb.poa.thread_pool_min=5
>
> jacorb.poa.queue_wait=off
> jacorb.poa.queue_max=100
> jacorb.poa.queue_min=10
>
> Please let me know if there are any known issues with Jacorb with respect
> to thread hangs /Deadlocks and way to resolve this Jacorb issue.
>
> Thanks,
> Prakash R
>
>
>
>
> _______________________________________________
> jacorb-developer maillist - jacorb-developer at lists.spline.
> inf.fu-berlin.de
> https://lists.spline.inf.fu-berlin.de/mailman/listinfo/jacorb-developer
>
--
Phil Mesnier
Principal Engineer & Partner
OCI | WE ARE SOFTWARE ENGINEERS.
tel (314) 579-0066 x225 <(314)%20579-0066>
ObjectComputing.com
<http://objectcomputing.com/>
More information about the jacorb-developer
mailing list