Understanding AddressSanitizer Heap Profile

Runtimes Sanitizers
1

Hi all,

I am trying to diagnose why ASan is exhibiting a gigantic (137x) increase in max RSS in a program I’m working with. I am looking at the output from ASAN_OPTIONS=heap_profile=1 and I see lines like this:

HEAP PROFILE at RSS 898Mb
Live Heap Allocations: 155292034 bytes in 1767102 chunks; quarantined: 239376065 bytes in 2424031 chunks; 2224650 other chunks; total chunks: 6415783; showing top 90% (at most 20 unique contexts)

When I add up the numbers, it seems like a huge amount of memory is unaccounted for:

>>> 155292034 / 1e6 + 239376065 / 1e6
394.668099
>>> 155292034 / 1e6 + 239376065 / 1e6 - 898 
-503.331901

Where is this 500 megabyte difference between the RSS (898Mb) and the live allocations + quarantine (394Mb) coming from? From what I understand, shadow data is stack + heap + redzone + globals / 8, so that seems unlikely. Could there be 500Mb redzone with only 394Mb of live + quarantined allocs?

Any pointers to where I should look next would be greatly appreciated!

2

Well, I figured it out, and it turns out I didn’t truly understand what “RSS” means! When allocations are in the quarantine, they are still alive from the POV of the OS, meaning that the page they reside on is still mapped for the process, and can’t be released. Using the quarantine keeps so many of these pages alive that the memory fragmentation just explodes in my case, with huge numbers of pages of memory being held open by just a few allocations each.