Quick Refresh : Multi-threading : How to decide pool size for Thread Pools?

 How to decide pool size for Thread Pools?

Thread pool is really an important topic to understand if you are developing an application, which is designed with multi-threading approaches. Thread pool is a pool of live worker threads. A worker thread is a thread that accepts a task, completes it, and comes back to the thread pool to accept another task. In such a way the application can use already exist threads multiple times instead of creating new threads every time. By using a thread pool in your application, you can boost its performance and utilize resources efficiently.

But in this article, we are not talking about how to create a thread pool and how does thread pool works. Here we will discuss how to decide pool size for thread pool? I mean, on which factors we could decide that how many threads, we should create in our thread pool for better time and resource utilization.

Most of the time people don’t think about pool size for their thread pool and they create any number of threads in the pool with their choice without thinking much about it. But believe me, if you don’t know how to decide a number of threads in the pool that could be more dangerous and could spoil your performance and memory management badly. Therefore, we will discuss in this article which factors thread pool size depends on and how an appropriate pool size affects the performance of your application?

There are few factors mentioned below on which thread pool size depends:

Available Processors: This is the first and most important factor on which thread pool size matters a lot. Because the purpose behind creating a thread pool or making any program multi-threaded is to utilize all available resources like processors and unused CPU cycles. If your pool size would be less than available processors in your system means you are not using all available processors and not utilizing resources fully. On another side, if your pool size is greater than your available processor that means you are creating more threads than a thread pool can handle. It is not possible to provide a CPU cycle to each thread which means you are wasting memory by creating such threads in the pool that is not going to utilize by a thread pool. There is one more overhead for the thread pool to maintain scheduling for such extra threads. Therefore, the ideal pool size is available processors (AP) in your system or AP+1. Here is an example, how to get a number of processors available in your system using Java.

int poolSize  = Runtime.getRuntime().availableProcessors();

int poolSize  = Runtime.getRuntime().availableProcessors() + 1;

This is ideal pool size, if your multithreaded task is kind of computation, where threads are not getting block, wait on I/O or some combination.

But if your task also includes some kind of blocking or waiting time then there is one more way to determine pool size for such tasks. That is explained in the next section.

The behavior of Tasks: It is important to understand the behavior of your task what you want to perform inside a thread. If you have a different category of tasks with different behaviors then consider your thread pool size according to that behavior. If your task is simple computation and does not obtain any blocking or waiting then you can consider a number of available processors (AP)  or AP+1, as mentioned in the above section.

But for tasks that also include I/O or other blocking operations, you need a larger pool size that, since not all of the threads will be schedulable at all times, some will be in wait condition. In order to size the pool properly, you must estimate the ratio of waiting time to compute time for your tasks, this estimate need not be precise and can be obtained through profiling or instrumentation. Alternatively, the size of the thread pool can be tuned by running the application using several different pool sizes under a benchmark load and observing the level of CPU utilization.

The optimal pool size for keeping the processors at the desired utilization of CUP is:

N = Number of processors available in the system.

U = Target utilization of CUP; 0 >= U <= 1.

W/C = Ration of wait and computation time.

The number of threads for the thread pool can evaluate by this formula:

Number of Threads = N * U * (1 + W/C)

This is the way to estimating your suitable thread pool size.

Amdahl’s Law: In application development, there are lots of tasks that cannot be performed totally concurrently, there are few tasks that need to be performed sequentially. Therefore it is important to understand, how much proportion of tasks can be executed concurrently and how much speed you would get after making that portion of task concurrent. Therefore, Amdahl’s law is very useful to determine how much speedup you would get if you are breaking up your task into parallelism and sequential.

Click on link for Amdahl’s Law