Re: Use PROT_NONE for unused virtual stack memory

[Bill Allombert <Bill.Allombert@math.u-bordeaux.fr>]

On Mon, Oct 03, 2016 at 05:04:53PM +0200, Jeroen Demeyer wrote:
> On 2016-09-29 17:46, Bill Allombert wrote:
> >Do you have some framework to test this ?
> 
> Not really because it depends a lot on the OS. In order to properly test
> this, you need a Linux system with vm.overcommit = 2. So you need
> 
> # echo 2 >/proc/sys/vm/overcommit_memory

What linux kernel version did you test ?  And how did you test it ?

> >>The idea is to use PROT_NONE for the unused part of the PARI stack (between
> >>vbot and bot). Memory mmap()ed with PROT_NONE is not committed: it does not
> >>count against the available physical/swap memory available for the system.
> >
> >Do you have some relevant documentation for this behaviour ?
> 
> No. As far as I know, this is undocumented. 

This is problematic... I wrote this code following the POSIX standard.
The code is not meant to be linux specific.

However I have no issue with using map(,PROT_NONE) and remapping to
map(,PROT_WRITE|PROT_READ) if it helps since this is POSIX compliant.

Maybe pari_mainstack_malloc could take both size in arguments
and do both mmap at once.

Did you try madvise(, MADV_DONTNEED) ?

> But the trick is known and used
> in the wild, see for example the discussion
> 
> https://github.com/jemalloc/jemalloc/issues/255

which links to 
<https://github.com/jemalloc/jemalloc/issues/193>
which says:

"
  The overcommit heuristic is clearly not sensible when using jemalloc, so
  I think it would make sense to detect it and set MAP_NORESERVE on all of
  the mappings. Rather than catching obvious errors, it's resulting in an
  inability to use all of (well, more than half of) the system's
  resources. Alternatively, MAP_NORESERVE could just be set by default.
  It's a no-op for the other 2 overcommit states (it's not documented
  correctly in the man pages project).
"

But your patch remove MAP_NORESERVE. Why ?

Cheers,
Bill.