Online Forums
Technical support is provided through Support Forums below. Anybody can view them; you need to Register/Login to our site (see links in upper right corner) in order to Post questions. You do not have to be a licensed user of our product.
Please read Rules for forum posts before reporting your issue or asking a question. OPC Labs team is actively monitoring the forums, and replies as soon as possible. Various technical information can also be found in our Knowledge Base. For your convenience, we have also assembled a Frequently Asked Questions page.
Do not use the Contact page for technical issues.
MultiThreading and performance
17 Feb 2015 16:29 #2802
by support
Replied by support on topic MultiThreading and performance
I have investigated the issue. I cannot claim that I fully understand what is happening, but I believe that I understand enough of it to provide an answer. It is a complicated matter, but the answer in short is:
Here is one part of the original code provided:
When I added more timing measurements, I have found that the TPL takes around 0.5 second to start each task (interestingly, this timespan is mentioned in some articles about TPL, related to optimizations it performs after some time). With 30 tasks, this means that they are not started immediately - it takes at least 15 seconds just to get to the last one. The ones that have started indeed do begin to do something, but then they pause. As far as I can tell, our own code is not responsible for that. Probably the TPL detected that new tasks are arriving, and decided to pause all the current work in order to evaluate the newly incoming work, or something to that effect. This is why we do not actually do the first Read up until much later.
If you change the last line in the body of the loop to the line below, the issue will go away:
The fact that just using this different task creation option (LongRunning) is a hint that the problem is not within our component It shows that tasks need to be scheduled differently - which is a TPL issue, not ours (the flag tells the TPL that a dedicated thread would be better for the task).
Using plain .NET threads resolves the issue as well, e.g.:
I stayed away from using TPL internally in the component, precisely for the reason that it has its own scheduling paradigms that would be hard to predict and to explain. This is the first time I have been presented with such issue, but I think it kind of proves the point, though for the TPL usage *outside* the component. I do not have much expertize in TPL, but those who use it should - otherwise they will run into problems. I believe that using the TPL is good for achieving a degree of parallelism and a cleaner code, but the cost is that the details of the scheduling are at hands of the TPL internal implementation.
Best regards
If you decide to use .NET Task Parallel Library (TPL), be prepared to live with how it behaves - and learn it well.
Here is one part of the original code provided:
var tasks = new List<Task>();
for (int i = 0; i < count; i++)
{
int i1 = i;
tasks.Add(Task.Run(() => MakeOpcCall(endpointDescriptor, tags, i1)));
}
Task.WaitAll(tasks.ToArray());When I added more timing measurements, I have found that the TPL takes around 0.5 second to start each task (interestingly, this timespan is mentioned in some articles about TPL, related to optimizations it performs after some time). With 30 tasks, this means that they are not started immediately - it takes at least 15 seconds just to get to the last one. The ones that have started indeed do begin to do something, but then they pause. As far as I can tell, our own code is not responsible for that. Probably the TPL detected that new tasks are arriving, and decided to pause all the current work in order to evaluate the newly incoming work, or something to that effect. This is why we do not actually do the first Read up until much later.
If you change the last line in the body of the loop to the line below, the issue will go away:
tasks.Add(Task.Factory.StartNew(() => MakeOpcCall(endpointDescriptor, tags, i1), TaskCreationOptions.LongRunning));The fact that just using this different task creation option (LongRunning) is a hint that the problem is not within our component It shows that tasks need to be scheduled differently - which is a TPL issue, not ours (the flag tells the TPL that a dedicated thread would be better for the task).
Using plain .NET threads resolves the issue as well, e.g.:
var threads = new List<Thread>();
for (int i = 0; i < count; i++)
{
int i1 = i;
// ReSharper disable once AssignNullToNotNullAttribute
var thread = new Thread(() => MakeOpcCall(endpointDescriptor, tags, i1));
thread.Start();
threads.Add(thread);
}
threads.ForEach(thread => thread.Join());
I stayed away from using TPL internally in the component, precisely for the reason that it has its own scheduling paradigms that would be hard to predict and to explain. This is the first time I have been presented with such issue, but I think it kind of proves the point, though for the TPL usage *outside* the component. I do not have much expertize in TPL, but those who use it should - otherwise they will run into problems. I believe that using the TPL is good for achieving a degree of parallelism and a cleaner code, but the cost is that the details of the scheduling are at hands of the TPL internal implementation.
Best regards
The following user(s) said Thank You: cdunlap
Please Log in or Create an account to join the conversation.
16 Feb 2015 20:25 #2793
by support
Replied by support on topic MultiThreading and performance
I see your point now; you are right, that's what I would expect as well. I will have a look at it. Please allow for some time - I have most of tomorrow booked.
Best regards
Best regards
Please Log in or Create an account to join the conversation.
16 Feb 2015 20:18 #2791
by cdunlap
Replied by cdunlap on topic MultiThreading and performance
Thanks. I understand that the underlying read calls are synchronous.
I notice though, that the first Read() call does not get sent from the first Task I create until after the last Task is created. I would think that when the first Task starts, it would immediately issue a Read(), and if other Tasks start, they will have to wait until the resource is free before being able to Read().
So something along the timeline of:
Etc...
If you pass in a parameter such as 30 to those methods in my sample program, it will attempt to create 30 Tasks that each do a Read(). It actually takes ~22 seconds from the time that the first Task is ran until I see a Read() go through (via CTT and Wireshark) to the OPC Server at all. Is that to be expected?
I notice though, that the first Read() call does not get sent from the first Task I create until after the last Task is created. I would think that when the first Task starts, it would immediately issue a Read(), and if other Tasks start, they will have to wait until the resource is free before being able to Read().
So something along the timeline of:
- Task1 is started
- Task1 calls Read()
- Task2 is started
- Task2 tries to Read() but is blocked
- Task3 is started
- Task1 completes Read()
- Task1 done
- Task2 calls Read()
- Task3 tries to Read() but is blocked
Etc...
If you pass in a parameter such as 30 to those methods in my sample program, it will attempt to create 30 Tasks that each do a Read(). It actually takes ~22 seconds from the time that the first Task is ran until I see a Read() go through (via CTT and Wireshark) to the OPC Server at all. Is that to be expected?
Please Log in or Create an account to join the conversation.
16 Feb 2015 19:25 #2787
by support
Replied by support on topic MultiThreading and performance
Hi C.
what you describe is actually to be expected. The reason for it is that the underlying OPC UA operations are always synchronous, at least in the current implementation. Therefore performing one operation blocks the others.
I am sorry because that's probably not what you'd prefer to hear...
ZZ
what you describe is actually to be expected. The reason for it is that the underlying OPC UA operations are always synchronous, at least in the current implementation. Therefore performing one operation blocks the others.
I am sorry because that's probably not what you'd prefer to hear...
ZZ
Please Log in or Create an account to join the conversation.
16 Feb 2015 13:43 #2779
by cdunlap
MultiThreading and performance was created by cdunlap
Hello,
I have been working on testing the performance of the EasyUAClient with multi-threading and have come across a problem I cannot figure out.
I have created a small test console application that attempts to accomplish the same task in 3 different ways. I will send this to you in an email, along with the TOP Server configuration file that I am using to read the tags from.
I would expect and do see similar performance out of method 2 and method 3 since "Isolated" property is not involved.
Here is my problem though:
It seems that the whichever method that I call first that spins up the list of tasks to run (either approach 2 or 3), it takes a lot of time to complete. The first task never finishes running until the last task has been started. I'm not sure if it is blocking somewhere that I am missing.
I used the CTT to confirm that in the cases described above, there are no Read() service calls from the time that the first task is ran until the last task is ran. Once the last task starts, I see a lot of Read() calls.
If I call method 2 or method 3 individually, they both exhibit this behavior.
If I call method 2 and then method 3 immediately after 2, method 3 finishes almost immediately - but the first task still does not complete until after the last one is started. This is the same behavior if I call method 3 and then method 2.
I am guessing this is probably a problem on my end, but I cannot figure it out and was hoping to get your input. Is there something that I am doing wrong here?
Thank you for all of the help.
I have been working on testing the performance of the EasyUAClient with multi-threading and have come across a problem I cannot figure out.
I have created a small test console application that attempts to accomplish the same task in 3 different ways. I will send this to you in an email, along with the TOP Server configuration file that I am using to read the tags from.
- The first approach calls ReadMultiple() in a loop for a given number of times.
- The second approach (I will call it method 2) runs a given number of Tasks in parallel that each call ReadMultiple() on a local EasyUAClient (each task creates its own client).
- The third approach (I will call it method 3) runs a given number of Tasks in parallel that each call ReadMultiple() on a global EasyUAClient.
I would expect and do see similar performance out of method 2 and method 3 since "Isolated" property is not involved.
Here is my problem though:
It seems that the whichever method that I call first that spins up the list of tasks to run (either approach 2 or 3), it takes a lot of time to complete. The first task never finishes running until the last task has been started. I'm not sure if it is blocking somewhere that I am missing.
I used the CTT to confirm that in the cases described above, there are no Read() service calls from the time that the first task is ran until the last task is ran. Once the last task starts, I see a lot of Read() calls.
If I call method 2 or method 3 individually, they both exhibit this behavior.
If I call method 2 and then method 3 immediately after 2, method 3 finishes almost immediately - but the first task still does not complete until after the last one is started. This is the same behavior if I call method 3 and then method 2.
I am guessing this is probably a problem on my end, but I cannot figure it out and was hoping to get your input. Is there something that I am doing wrong here?
Thank you for all of the help.
Please Log in or Create an account to join the conversation.
Moderators: support
Time to create page: 0.064 seconds