LLVM Discussion Forums

LIT default number of used CPU cores

Hi all,
it’s the first time I use LLVM (at least knowing it) and I noticed that the LLVM Integrated Tester (LIT) used 4 cores CPU cores on my laptop having an Intel i7-4600U CPU with only 2 physical cores (which gets to 4 counting the virtual cores created by hyperthreading).

Usually, for processes where the CPU is the bottleneck the faster option is to use a number of threads no more and no less than the number of physical cores (not the total one including the virtual cores).

The default number of cores for the tests gets calculated here:

https://reviews.llvm.org/source/llvm-github/browse/master/llvm/utils/lit/lit/util.py;6a887d22a17e0b18bb5b6b98278bcd0de902bb1d$112-137

The best way I could find for obtaining the number of physical cores using Python is:

import psutil
psutil.cpu_count(logical=False)

Documented on https://psutil.readthedocs.io/en/latest/#psutil.cpu_count

For which python-psutil should be added to the dependencies list.

psutil is multiplatform, which means that all the indicated code block can be replaced by its usage.

Opinions?

Did you benchmark the effect of this for the LLVM tests on your laptop?

I was speaking generically, without testing. Now I tested with make check-cxx and it takes less time with more workers. My assumption about the tests being a pure CPU load was wrong or the observation I made with other software cannot be generalized (and have to be tested on a per-case basis), sorry for that.

With 5 workers:
Testing Time: 2777.47s
With 4 workers:
Testing Time: 2775.59s
With 3 workers;
Testing Time: 2890.70s
With 2 workers:
Testing Time: 3098.65s