Is the memory block allocated by a thread is on the same affinity as the thread itself until the thread exit?
Multi tool use
Is the memory block allocated by a thread is on the same affinity as the thread itself until the thread exit?
This is a question about NUMA.
For example, in the code below, is the buffer allocated at the local memory of the thread/process throughout its life?
for (int th = 0; th < maxThreads; th++)
{
threads[th] = std::thread([&, th] {
int* buffer = new int[1000];
// do something
delete buffer;
}
}
Update: to make the question more straightforward, let me ask in this way. If I have 10 simultaneous threads (say, t0 to t9) launched, and within the scope of each threads, it allocates a memory block(say, m0 to m9). Would thread t_n always operates on m_n (n from 0 to 9) before the thread exit, or thread 0 might migrate and operate on memory 9? The memory block in my situation is not very large, usually only a couple of mega-bytes.
related: stackoverflow.com/questions/2142107/…
– vu1p3n0x
Jul 2 at 2:39
Mostly I use visual studio compiler or intel compiler (bundled with visual studio)
– photoelectric
Jul 2 at 14:34
Given you're using Visual Studio, you probably want to start here: docs.microsoft.com/en-us/windows/desktop/procthread/…
– Andrew Henle
Jul 2 at 19:01
thanks @AndrewHenle I am not sure how to understand this sentence bout the VirtualAllocExNuma at the end of the first section: "If the preferred node runs out of pages, the memory manager will use pages from other nodes. If the memory is paged out, the same process is used when it is brought back in." Does this means that if the memory is allocated from other node, the process uses the memory is also on that "other" node?
– photoelectric
Jul 3 at 15:25
2 Answers
2
I think this behavior is related to implementation of OS, but I believe with Linux which configured and built with CONFIG_NUMA option, the answer is YES.
NUMA on Linux
If you don't call some function to tell it otherwise, most implementations will "stripe" all memory, ameliorating the cost of wrong-node memory.
For any given small allocation, this means the node is basically random.
See "what every programmer should know about memory, part 4"
So are you saying small memory block is allocated randomly across the nodes, even it is allocated inside the thread? This seems contradicts what I learned before. For example, this says "Modern OS (at least Linux, others probably too) do a good job thus far, the memory is, by default, allocated (if available) from the same domain of the CPU where the thread is running" stackoverflow.com/questions/24645880/…
– photoelectric
Jul 2 at 15:40
There's definitely conflicting information out there ... I admit that mine is an older source. But in the absence of setting the affinity, striped memory is the optimal solution, and I've verified that processes do migrate frequently. And pinning is hard when you have to worry about other processes on the system.
– o11c
Jul 2 at 16:59
I agree that processes do migrate frequently since that's exactly what set affinity aims to address. The question I am asking here is a little different. I am asking in multi-threading, inside the thread, would the thread/process also migrate even if the memory block is allocated inside that thread. That is, if I have 10 simultaneous threads (say, t0 to t9), within the scope of each threads, it allocates a memory block(say, m0 to m9). Would thread t_n always operates on m_n (n from 0 to 9) before the thread exit, or as you said, thread 0 might migrate and operate on memory 9?
– photoelectric
Jul 2 at 18:25
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
This will be implementation specific; what is your compiler?
– vu1p3n0x
Jul 2 at 2:25