[jacorb-developer] Unexpected behaviour in JacORB 3.6 and 3.6.1(Bug 1011 and 1012)

[Phil Mesnier]

HI Folks,

As an interesting coincidence I found that the thread-local invocation context stack in the Delegate is affected by a non-thread-local cleanup flag. Although I did not have a test case, I saw the potential for trouble due to concurrent requests involving interceptors. I have a fix pending to ensure invocation context stack manipulation stays within the single thread context.

Nicolas, it looks like you have a testable use case.

Best regards,
Phil

> On Jul 10, 2015, at 12:20 PM, Hugo Roenick <hroenick at gmail.com> wrote:
> 
> Hi, Nicolas.
> 
> Sure it is a possibility and we thought about it.
> At first we just wanted to isolate the problem to help fully understand the issue and share with you all.
> 
> PS: We are now addressing a new possible bug regarding the usage of ForwardRequest with concurrent request to a same CORBA Object (2 threads making request to the same CORBA reference). We will probably present this case as soon we isolate in a simple JacORB test case, and in a new jacorb-dev thread.
> 
> Regards,
> -- Hugo
> 
> On Jul 9, 2015, at 3:46 AM, Nicolas Aguilé <na.ais40 at wanadoo.fr> wrote:
> 
>> Hi,
>> We have the same problem, we have investigate it and in our cas it seems to happen the first time whe pass through interceptors.
>> We narrow a CORBA service inside the interceptor and we think that it involves the pop of the invocation context to 0
>> 
>> We have protect the call for poping the invocation context like this :
>> 
>>              if ( !Delegate.getInvocationContext().isEmpty() )             {                 Delegate.getInvocationContext().pop ();             }
>> 
>> It seems to work, but we don't really know the impact of this protection and why invoking a service in the send_request dequeu thecontext
>> 
>> If it can help...
>> regards
>> Nicolas Aguilé
>> ----- message d'origine -----
>> De : "Hugo Roenick" <hroenick at gmail.com>
>> date mer. 08/07/2015 20:00 (GMT +02:00)
>> À : "JacORB Developer" <jacorb-developer at lists.spline.inf.fu-berlin.de>
>> Objet : [jacorb-developer] Unexpected behaviour in JacORB 3.6 and 3.6.1(Bug 1011 and 1012)
>> 
>> Hi guys.
>> 
>> Theres is a while since I noticed that the JacORB update to 3.6 caused one of our test case to fail, but I didn't manage to reserve the necessary time to isolate the problem and report to you guys.
>> Sorry about that.
>> 
>> Well, I finally did manage to save some time and I'm working on this on the last few days.
>> The problem is a little trick to explain, so I will try to be as clear that I can. 
>> I'm sorry in advance for the extension of this mail.
>> 
>> The main problem in this specific test is that we start to receive an unexpected error on client side. This is the stack trace is something like (this one is from the isolated demo that I created and will attach in a bug issue):
>> 
>> Exception in thread "main" java.util.NoSuchElementException
>> at java.util.ArrayDeque.removeFirst(ArrayDeque.java:278)
>> at java.util.ArrayDeque.pop(ArrayDeque.java:507)
>> at org.jacorb.orb.portableInterceptor.ClientInterceptorIterator.invoke(ClientInterceptorIterator.java:169)
>> at org.jacorb.orb.portableInterceptor.AbstractInterceptorIterator.iterate(AbstractInterceptorIterator.java:66)
>> at org.jacorb.orb.portableInterceptor.ClientInterceptorIterator.iterate(ClientInterceptorIterator.java:79)
>> at org.jacorb.orb.portableInterceptor.DefaultClientInterceptorHandler.invokeInterceptors(DefaultClientInterceptorHandler.java:331)
>> at org.jacorb.orb.portableInterceptor.DefaultClientInterceptorHandler.handle_send_request(DefaultClientInterceptorHandler.java:135)
>> at org.jacorb.orb.Delegate._invoke_internal(Delegate.java:1268)
>> at org.jacorb.orb.Delegate.invoke_internal(Delegate.java:1171)
>> at org.jacorb.orb.Delegate.invoke(Delegate.java:1159)
>> at org.omg.CORBA.portable.ObjectImpl._invoke(ObjectImpl.java:475)
>> at demo._DoorStub.canIComeIn(_DoorStub.java:119)
>> at demo.door.DoorClient.main(DoorClient.java:41)
>> 
>> I checked that this kind of error does not occur in JacORB 3.5 and is present in versions 3.6 and 3.6.1.
>> 
>> This error is related to making 2 or more remote calls during send_request interception point and doing it while attending to a forward request attempt. (It probably is not a common scenario, but we used this strategy to build a handshake protocol between CORBA parties)
>> 
>> While isolating this problem I stumbled in another unexpected behaviour. Instead of making 2 or more calls inside the send_request interception point, just make 1 more call that is also going to receive an exception and resolve it through a forward_request. In this case I noticed that there is an extra send_request call and the previous one apparently is not completed (does not arrives in server side). 
>> 
>> Let me try to make myself clear with an example:
>> 
>> [client][server]
>> ---- calls A() ---->
>> <---- NO_PERMISSION (op = A) ----
>> 
>> receive_exception:
>> do forward_request( op = A )
>> 
>> ---- calls A() ---->
>> ---- send_request( op = A )  ---->
>> ---- calls B() ---->
>> <---- NO_PERMISSION (op = B) ----
>> 
>> receive_exception:
>> do forward_request( op = B )
>> 
>> ---- calls B() ---->
>> <---- successful reply (op = B) ----
>> --- complete the send_request( op = A) --->
>> // "apparently this completed send_request does not arrives to the server"
>> // why ???
>> 
>> // an extra call is sent. Why??
>> ---- calls A() ---->
>> <---- successful reply (op = A) ----
>> 
>> 
>> Placed two bug issues in bugzilla with the demo code (its the same demo, with a different case test enum configured in DoorClient.java):
>> - Bug 1011 - NoSuchElementException while making remote calls while attending to a forward_request
>> - Bug 1012 - Missed send_request while completing a forward request
>> 
>> Regards,
>> -- Hugo
>> 
>> _______________________________________________
>> jacorb-developer maillist  -  jacorb-developer at lists.spline.inf.fu-berlin.de
>> https://lists.spline.inf.fu-berlin.de/mailman/listinfo/jacorb-developer
>> _______________________________________________
>> jacorb-developer maillist  -  jacorb-developer at lists.spline.inf.fu-berlin.de
>> https://lists.spline.inf.fu-berlin.de/mailman/listinfo/jacorb-developer
> 
> _______________________________________________
> jacorb-developer maillist  -  jacorb-developer at lists.spline.inf.fu-berlin.de
> https://lists.spline.inf.fu-berlin.de/mailman/listinfo/jacorb-developer

--
Phil Mesnier
Principal Software Engineer and Partner,   http://www.ociweb.com
Object Computing, Inc.                     +01.314.579.0066 x225