ThreadPoolExecutor in Java: How Thread Pools Really Work

Learn ThreadPoolExecutor in Java step by step: core vs max pool size, bounded queues, keep-alive time, the 4 rejection policies and practical tuning tips.

ThreadPoolExecutor in Java is the class that actually runs your tasks when you use an ExecutorService. Most of us meet it through Executors.newFixedThreadPool() and never look inside. That works fine for a while. Then one day production slows down, memory keeps climbing, and nobody can explain why.

A very common story in backend teams goes like this. A downstream service becomes slow. Tasks start waiting in the pool’s queue. The queue has no limit, so it keeps growing quietly. After some time, the JVM throws an OutOfMemoryError. More hardware would not fix this. Understanding the pool would.

So in this article, we will open up ThreadPoolExecutor and look at every moving part. We will cover core and maximum pool size, the work queue, keep-alive time and the four rejection policies. After that, we will trace the exact path a task takes inside the pool. Finally, you will get simple tuning rules that you can use at work.

One quick note before we start. This article assumes you know ExecutorService basics like submit(), shutdown() and Future. If those feel new, read ExecutorService in Java first and then come back here.

1. Why Do We Need a Thread Pool?

1.1 The Problem With One Thread per Task

Imagine a web server that starts a brand new thread for every request. It looks simple. However, it breaks down fast under real traffic.

• Creating threads costs time. Every regular Java thread (a platform thread) maps to an operating system thread. The OS has to set it up and reserve stack memory for it.
• Nothing limits the count. A traffic spike can create thousands of threads. Each one eats memory, and the CPU wastes time switching between them.
• Nobody reuses anything. A thread does one small job and dies. Then the next request pays the full setup cost again.

1.2 What a Thread Pool Does Differently

A thread pool fixes all three problems. It keeps a small set of worker threads alive. When a task arrives, an existing worker picks it up. When the task finishes, the worker does not die. Instead, it goes back and takes the next task.

I like to think of it as a restaurant kitchen. The cooks are the threads. Order slips hanging on the rail form the queue. And the kitchen manager decides who cooks what, and when to call extra help. In Java, that manager is ThreadPoolExecutor.

1.3 Where ThreadPoolExecutor Fits

Here is the family tree, from the top down:

• Executor has just one method, execute(Runnable).
• ExecutorService adds submit(), shutdown(), invokeAll() and friends.
• AbstractExecutorService gives default code for the submit() family.
• ThreadPoolExecutor does the real work: threads, queue, sizing and rejection.

The Executors factory methods are just shortcuts. Behind the scenes, most of them simply call the ThreadPoolExecutor constructor for you. Look at what the JDK does internally:

// What Executors.newFixedThreadPool(n) does inside the JDK
new ThreadPoolExecutor(n, n,
        0L, TimeUnit.MILLISECONDS,
        new LinkedBlockingQueue<Runnable>());
 
// What Executors.newCachedThreadPool() does inside the JDK
new ThreadPoolExecutor(0, Integer.MAX_VALUE,
        60L, TimeUnit.SECONDS,
        new SynchronousQueue<Runnable>());

Keep these two lines in mind. Both of them hide a risk, and we will come back to it in the tuning section.

2. Creating a ThreadPoolExecutor: The Seven Parameters

The full constructor takes seven arguments. At first it looks scary. But each argument answers one simple question about how the pool should behave.

ThreadPoolExecutor pool = new ThreadPoolExecutor(
        4,                                  // corePoolSize
        8,                                  // maximumPoolSize
        60, TimeUnit.SECONDS,               // keepAliveTime + unit
        new ArrayBlockingQueue<>(100),      // workQueue
        Executors.defaultThreadFactory(),   // threadFactory
        new ThreadPoolExecutor.AbortPolicy()// handler
);

Shorter constructors also exist. If you skip the last two arguments, the pool uses Executors.defaultThreadFactory() and AbortPolicy.

The constructor also checks your numbers. It throws IllegalArgumentException in these cases:

• corePoolSize is less than zero.
• maximumPoolSize is zero or less.
• maximumPoolSize is smaller than corePoolSize.
• keepAliveTime is negative.

Passing a null queue, factory or handler gives you a NullPointerException instead.

3. Core Pool Size vs Maximum Pool Size

These two numbers confuse more developers than anything else in this class. So let us go slowly.

3.1 corePoolSize: The Permanent Staff

The core threads are your regular cooks. By default, they stay alive even when there is no work. The pool does not kill them for being idle.

Here is one detail many people miss. While the pool has fewer threads than corePoolSize, every new task creates a new thread. This happens even if other core threads are sitting idle. The pool wants to reach its core size first.

3.2 maximumPoolSize: The Extra Help

maximumPoolSize is the ceiling. The pool will never run more threads than this. Threads above the core count are temporary helpers. They come in during a rush and leave when things calm down.

3.3 The Rule Most People Get Wrong

Most developers guess the order like this: fill core threads, then grow to max, then queue. That guess sounds logical. Sadly, it is wrong.

The real order is different:

• First, the pool fills up the core threads.
• Next, it puts new tasks in the queue.
• Only when the queue is full does it add threads up to the max.
• When even that is not enough, it rejects the task.

So extra threads are the last resort, not the first. The queue always comes before them.

3.4 Core Threads Start Lazily

A fresh pool starts with zero threads. It creates core threads one by one as tasks arrive. Most of the time that is fine.

Sometimes, though, you want the threads ready before the first request hits. For example, a low-latency service may not want the first few users to pay the startup cost. Two methods help here:

pool.prestartCoreThread();      // starts one core thread
pool.prestartAllCoreThreads();  // starts all, returns how many

3.5 Changing the Sizes at Runtime

You can change both sizes on a running pool with setCorePoolSize() and setMaximumPoolSize(). This helps when you read sizes from a config server.

Watch the order, though. In modern Java, setCorePoolSize() throws IllegalArgumentException if the new core is above the max. Java 8 did not check this. So when you grow the pool, raise the max first. When you shrink it, lower the core first.

// Growing from 4/8 to 10/20: max first, then core
pool.setMaximumPoolSize(20);
pool.setCorePoolSize(10);
 
// Shrinking from 10/20 to 2/4: core first, then max
pool.setCorePoolSize(2);
pool.setMaximumPoolSize(4);

4. The Work Queue: Bounded vs Unbounded

The work queue holds tasks that are waiting for a free thread. Your queue choice changes how the whole pool behaves under load. Honestly, this is the most important decision you make when you build a pool.

Any BlockingQueue<Runnable> works here. In practice, four types cover almost every case.

4.1 Unbounded Queue: LinkedBlockingQueue

When you create a LinkedBlockingQueue without a size, its capacity is Integer.MAX_VALUE. For all practical purposes, it never fills up. That has two big side effects:

• The pool never grows past the core size. maximumPoolSize becomes useless.
• The pool never rejects work. Tasks just keep piling up in memory.

Let us prove the first point with a small program. We set max to 10 and throw 100 slow tasks at the pool.

import java.util.concurrent.*;
 
public class UnboundedQueueDemo {
    public static void main(String[] args) {
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                2, 10,                    // max is 10, but watch this
                60, TimeUnit.SECONDS,
                new LinkedBlockingQueue<>()); // unbounded queue
 
        for (int i = 1; i <= 100; i++) {
            pool.execute(() -> sleep(1000));
        }
        System.out.println("Threads created : " + pool.getPoolSize());
        System.out.println("Tasks waiting   : " + pool.getQueue().size());
        pool.shutdown();
    }
 
    static void sleep(long ms) {
        try { Thread.sleep(ms); } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

Only two threads, even though we allowed ten. The other 98 tasks just wait. Now picture this with a slow database and a million tasks. That is exactly how the out-of-memory story from the intro happens.

4.2 Bounded Queue: ArrayBlockingQueue

A bounded queue has a fixed size. ArrayBlockingQueue is the usual pick. A LinkedBlockingQueue with a size, like new LinkedBlockingQueue<>(500), also works.

With a bounded queue, the pool behaves the way you expect. It fills the core threads, then the queue, then grows to max. After that, it rejects. This gives you backpressure. The system pushes back on callers instead of silently eating memory.

4.3 Direct Handoff: SynchronousQueue

A SynchronousQueue has no storage at all. A task can only enter it if a thread is waiting right then to take it. In the kitchen, it is like handing the order straight to a cook’s hand.

So if no idle thread exists, the pool creates a new one, up to the max. newCachedThreadPool() pairs this queue with a max of Integer.MAX_VALUE. Under a heavy burst, that can mean thousands of threads. Use this queue only with a sensible maximumPoolSize.

4.4 Priority Order: PriorityBlockingQueue

Sometimes some tasks matter more than others. A PriorityBlockingQueue lets workers pick the most important task first. Keep two things in mind:

• It is unbounded, so the max-size problem from 4.1 applies here too.
• Use execute(), not submit(). The submit() method wraps your task in a FutureTask. FutureTask is not Comparable, so the queue fails with a ClassCastException.

4.5 Quick Comparison

5. Keep-Alive Time

5.1 What Keep-Alive Actually Controls

Keep-alive time decides how long an extra thread can stay idle. Whenever the pool has more than corePoolSize threads, any thread idle longer than this exits. As a result, the pool shrinks back to its core size after a rush.

One small detail here. Java does not tag threads as “core” or “extra”. It only counts them. So any idle thread can be the one that leaves, as long as the count stays above the core size.

Here is a small demo. Core size is 1, max is 3 and keep-alive is two seconds.

import java.util.concurrent.*;
 
public class KeepAliveDemo {
    public static void main(String[] args) throws InterruptedException {
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                1, 3,
                2, TimeUnit.SECONDS,      // extra threads live 2s idle
                new ArrayBlockingQueue<>(1));
 
        for (int i = 0; i < 4; i++) {
            pool.execute(() -> sleep(500));
        }
        System.out.println("During the burst : "
                + pool.getPoolSize() + " threads");
 
        Thread.sleep(5000);               // idle longer than keep-alive
        System.out.println("After idle time  : "
                + pool.getPoolSize() + " threads");
        pool.shutdown();
    }
 
    static void sleep(long ms) {
        try { Thread.sleep(ms); } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

During the burst, the pool grew to three threads. Five seconds later, the two helpers had timed out. Only the single core thread remained.

5.2 Letting Core Threads Time Out Too

By default, keep-alive only touches the extra threads. You can change that with one call:

pool.allowCoreThreadTimeOut(true);   // core threads can now time out

This suits pools that stay quiet most of the day. A nightly report job is a good example. Why keep threads alive for 23 hours doing nothing?

One catch here. The keep-alive time must be more than zero when you turn this on. Otherwise, the call throws IllegalArgumentException.

5.3 Picking a Sensible Value

• Too short: the pool keeps killing threads and creating new ones. You pay the setup cost again and again.
• Too long: idle threads hold memory for no reason after a spike ends.
• Good start: 30 to 60 seconds suits most services. The cached pool in the JDK uses 60 seconds.

6. ThreadFactory: Give Your Threads Proper Names

The default factory names threads like pool-1-thread-3. That tells you nothing when you read a thread dump at 2 AM. A custom ThreadFactory fixes this in a few lines.

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
 
public class NamedPoolDemo {
    public static void main(String[] args) {
        ThreadFactory factory = new ThreadFactory() {
            private final AtomicInteger count = new AtomicInteger(1);
 
            @Override
            public Thread newThread(Runnable r) {
                String name = "order-worker-" + count.getAndIncrement();
                return new Thread(r, name);
            }
        };
 
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                2, 4, 30, TimeUnit.SECONDS,
                new ArrayBlockingQueue<>(100),
                factory,
                new ThreadPoolExecutor.CallerRunsPolicy());
 
        Runnable printName =
                () -> System.out.println(Thread.currentThread().getName());
        pool.execute(printName);
        pool.execute(printName);
        pool.shutdown();
    }
}

Besides names, a factory can also mark threads as daemon threads or attach an uncaught exception handler. Named threads alone will save you hours during debugging.

7. Rejection Policies: The Four Built-in Handlers

7.1 When Does a Pool Reject a Task?

The pool rejects a task in two situations:

• Saturation. The queue is full, and every thread up to the max is busy.
• Shutdown. After shutdown(), the pool refuses every new task.

In both cases, the pool hands the task to its RejectedExecutionHandler. Java gives you four ready-made handlers as nested classes inside ThreadPoolExecutor.

7.2 AbortPolicy (the Default)

AbortPolicy throws a RejectedExecutionException back to the caller. The task never runs. This is loud and clear, which is a good thing.

It works well when the caller can do something useful with the error. A REST API, for instance, can catch it and return HTTP 503 so the client retries later.

7.3 CallerRunsPolicy

CallerRunsPolicy does not drop anything. Instead, the thread that called execute() runs the task itself. While it is busy with that task, it cannot submit more. So the producer slows down on its own.

import java.util.concurrent.*;
 
public class CallerRunsDemo {
    public static void main(String[] args) {
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                1, 1,
                0, TimeUnit.SECONDS,
                new ArrayBlockingQueue<>(1),
                new ThreadPoolExecutor.CallerRunsPolicy());
 
        for (int i = 1; i <= 3; i++) {
            int taskId = i;
            pool.execute(() -> {
                System.out.println("Task " + taskId + " ran on "
                        + Thread.currentThread().getName());
                sleep(1000);
            });
        }
        pool.shutdown();
    }
 
    static void sleep(long ms) {
        try { Thread.sleep(ms); } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

Task 1 took the only thread and task 2 took the only queue slot. Task 3 had nowhere to go, so the main thread ran it. The exact print order can differ on your machine.

This natural backpressure makes CallerRunsPolicy a favourite for batch jobs. Still, be careful where you use it:

• If the caller is a request thread, that request suddenly becomes slow.
• If the caller is an event-loop thread, it can freeze your whole server.
• After shutdown, this policy silently drops the task instead of running it.

7.4 DiscardPolicy

DiscardPolicy quietly throws the task away. No exception, no log, nothing. The caller never finds out.

Use it only for work that truly does not matter, like sending optional metrics. For anything else, silent loss is a nightmare to debug.

7.5 DiscardOldestPolicy

DiscardOldestPolicy removes the task at the head of the queue and then retries the new one. The idea is that the newest data is the most useful.

This can fit something like live price updates, where an old update has no value anymore. But think carefully with a priority queue. There, the head holds the most important task, so this policy drops exactly the wrong one. The Javadoc also warns about it. It calls this policy rarely useful when other code waits on task results or must record failures.

7.6 Writing Your Own Handler

Often none of the four fits perfectly. Luckily, RejectedExecutionHandler has just one method, so a custom one is easy. Here is a handler that logs the rejection and then fails loudly:

RejectedExecutionHandler logAndAbort = (task, executor) -> {
    System.err.println("Rejected task. Active=" + executor.getActiveCount()
            + ", queued=" + executor.getQueue().size());
    // You could also push a metric or save the task to a DB here
    throw new RejectedExecutionException("Pool is full, try again later");
};

7.7 Summary of the Policies

8. How a Submitted Task Flows Through the Pool

Now let us put every piece together. Whether you call execute() or submit(), the task ends up in the same place. The submit() method simply wraps your task in a FutureTask and then calls execute().

8.1 The Four Checks Inside execute()

Every time a task arrives, the pool asks these questions in this exact order:

1. Are fewer than core threads running? Then start a new thread and give it this task directly.
2. Can the queue accept the task? Then add it to the queue. A free worker will pick it up later.
3. Queue full, but fewer than max threads? Then start an extra thread and give it this task.
4. None of the above? Then send the task to the rejection handler.

8.2 Walking Through a Real Example

Let us watch this happen. We set core size to 2, max to 4 and a queue with room for 2 tasks. Then we submit 7 tasks, and each one sleeps for two seconds.

import java.util.concurrent.*;
 
public class TaskFlowDemo {
    public static void main(String[] args) {
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                2,                              // core pool size
                4,                              // maximum pool size
                30, TimeUnit.SECONDS,           // keep-alive time
                new ArrayBlockingQueue<>(2),    // bounded queue, capacity 2
                new ThreadPoolExecutor.AbortPolicy());
 
        for (int i = 1; i <= 7; i++) {
            int taskId = i;
            try {
                pool.execute(() -> {
                    System.out.println("Task " + taskId + " running on "
                            + Thread.currentThread().getName());
                    sleep(2000);
                });
                System.out.println("Submitted task " + taskId
                        + " -> threads=" + pool.getPoolSize()
                        + ", queued=" + pool.getQueue().size());
            } catch (RejectedExecutionException e) {
                System.out.println("Task " + taskId + " REJECTED");
            }
        }
        pool.shutdown();
    }
 
    static void sleep(long ms) {
        try { Thread.sleep(ms); } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
}

Here is what happened to each task:

Why Did Tasks 5 and 6 Run Before 3 and 4?

Look at the output again. Tasks 5 and 6 started right away, while 3 and 4 waited in the queue. A new thread always starts with the task that caused it, not with the oldest queued task.

So a ThreadPoolExecutor does not promise first-in, first-out order across all tasks. If strict ordering matters to you, a single-thread executor is the safer choice.

8.3 The Hidden Double-Check

There is one more subtle step inside check 2. After adding a task to the queue, the pool looks again. If the pool shut down meanwhile, it removes the task and rejects it. And if no threads are running at all, it starts one.

That second rule explains a strange case. Try core size 0, max size 10 and an unbounded queue. You might expect up to ten threads. In reality, you get exactly one. The queue never fills, so the pool never grows past that single rescue thread.

9. Watching Your Pool at Runtime

You cannot tune what you cannot see. Thankfully, ThreadPoolExecutor exposes several useful getters:

• getPoolSize() returns how many threads exist right now.
• getActiveCount() gives roughly how many threads are busy with tasks.
• getQueue().size() shows how many tasks are waiting.
• getLargestPoolSize() tells the highest thread count ever reached.
• getCompletedTaskCount() counts roughly how many tasks have finished.

Log these numbers every few seconds, or export them to your metrics tool. A queue that keeps growing is the earliest warning sign you will get.

ScheduledExecutorService monitor =
        Executors.newSingleThreadScheduledExecutor();
monitor.scheduleAtFixedRate(() ->
        System.out.printf("threads=%d active=%d queued=%d done=%d%n",
                pool.getPoolSize(),
                pool.getActiveCount(),
                pool.getQueue().size(),
                pool.getCompletedTaskCount()),
        0, 5, TimeUnit.SECONDS);

For deeper tracking, you can extend ThreadPoolExecutor and override its hooks. beforeExecute() runs before each task and afterExecute() runs after it. Both are handy for timing tasks or logging failures.

10. Sensible Tuning Guidance

No magic number works for every pool. But a few rules will get you close, and then testing does the rest.

10.1 First, Know Your Workload

CPU-Bound Tasks

These tasks keep the CPU busy the whole time. Think of image resizing, encryption or heavy calculations. Here, more threads than cores only adds switching overhead.

int cores = Runtime.getRuntime().availableProcessors();
int poolSize = cores + 1;   // the extra one covers the odd pause

I/O-Bound Tasks

These tasks spend most of their time waiting on a database, a file or a remote API. While one thread waits, another can use the CPU. So you need more threads than cores.

The book Java Concurrency in Practice gives a handy formula for this:

threads = cores × (1 + wait time / compute time)

Say you have 8 cores. Each task waits 90 ms for an API and computes for 10 ms. That gives 8 × (1 + 90/10) = 80 threads. Treat this as a starting point, not a final answer. Then load test it.

The book’s full formula also multiplies by a target CPU usage between 0 and 1. Here we assume you want the CPUs fully busy, so that factor is simply 1.

10.2 Always Bound the Queue

Pick a queue size that matches how much delay you can accept. If each task takes 50 ms and you have 10 threads, a queue of 1,000 means roughly 5 seconds of waiting. Is that fine for your users? That question should decide the number.

10.3 Choose a Rejection Policy on Purpose

Never leave the handler as an accident. Decide what should happen when the pool is full. Then write that decision down in the code with a comment.

10.4 Use Separate Pools for Separate Jobs

Do not run payments and PDF reports on the same pool. If reports get slow, they will block payments too. Separate pools keep one slow job from dragging down the rest. People call this the bulkhead pattern, named after the walls inside a ship.

10.5 Skip the Executors Shortcuts in Production

Remember the two factory methods from section 1.3? Now their risks should be clear:

• newFixedThreadPool() has an unbounded queue, so memory can grow without limit.
• newCachedThreadPool() has no real thread limit, so a burst can create thousands of threads.

For this reason, many teams forbid these shortcuts in production code. The Alibaba Java Coding Guidelines, for example, ask developers to build ThreadPoolExecutor directly.

10.6 A Production-Ready Starting Point

Here is a pool I would feel comfortable starting with for a typical I/O-heavy service:

int cores = Runtime.getRuntime().availableProcessors();
 
ThreadPoolExecutor pool = new ThreadPoolExecutor(
        cores * 2,                          // core: steady load
        cores * 4,                          // max: room for spikes
        60, TimeUnit.SECONDS,               // extras leave after 1 min idle
        new ArrayBlockingQueue<>(500),      // bounded: no silent growth
        namedFactory("api-worker"),         // readable thread dumps
        new ThreadPoolExecutor.CallerRunsPolicy()); // backpressure, no loss

Here, namedFactory() is just a helper that builds a factory like the one in section 6. Adjust every number after you watch real traffic and metrics.

10.7 A Quick Word on Virtual Threads

Java 21 brought virtual threads. They are cheap enough to create one per task, so you do not pool them. For heavy blocking I/O, Executors.newVirtualThreadPerTaskExecutor() is often simpler than a tuned pool.

Still, ThreadPoolExecutor is far from dead. It remains the right tool for CPU-bound work. It is also the one you will find in almost every existing codebase and interview.

11. Common Mistakes to Avoid

• Expecting max size to work with an unbounded queue. It never kicks in, as we saw in section 4.1.
• Forgetting to call shutdown(). Pool threads are non-daemon by default, so the JVM may never exit.
• Losing exceptions with submit(). The Future stores the exception. If you never call get(), you never see it.
• Tasks waiting on other tasks in the same pool. With a small pool, all threads can end up waiting on each other. Nothing moves forward. This trap has a name: thread starvation deadlock.
• Using DiscardPolicy without any logging. Work disappears and nobody knows.
• One giant shared pool for everything. One slow job then hurts every other job.

12. FAQ’s on threadpoolexecutor in java

13. Conclusion

ThreadPoolExecutor looks complex because of its seven parameters. But once you see the decision path, everything clicks. Core threads come first, then the queue, then extra threads, and finally the rejection handler.

Here are the key points to take away:

• Extra threads beyond the core only appear when the queue is full.
• An unbounded queue makes maximumPoolSize useless and hides overload.
• Keep-alive time trims idle extra threads, and optionally core threads too.
• Pick a rejection policy on purpose. CallerRunsPolicy is a safe default for many jobs.
• Size the pool from your workload, then measure and adjust.

Build your next pool by hand instead of calling a factory method. You will understand your system better, and production will thank you.

14. Further Reading

• ExecutorService in Java
• Callable and Future in Java
• ScheduledExecutorService in Java
• BlockingQueue in Java: Producer-Consumer Pattern
• ForkJoinPool in Java
• ThreadPoolExecutor – Oracle Java SE 21 Javadoc