In modern programming languages on a modern operating system, you will either get a stack overflow , or they malloc()will fail when you try to increment the heap. But not all software is modern, so let's look at the failure modes:sbrk()mmap()
If the stack grows and hits the heap, the typical C compiler will start silently overwriting the heap's data structures. On modern operating systems there will be one or more virtual memory save pages that will prevent the stack from growing indefinitely. As long as the amount of guard page memory is at least as large as the size of the procedure's activation record that grows, the OS will guarantee you a segmentation fault ( segfault or segmentation fault ). If you are running your program on DOS or a machine without an MMU... then you have serious problems.
If the heap grows and meets the stack, the operating system must be aware of the situation and some kind of system call will fail. The implementation of malloc()will definitely catch the failure and return NULL. What happens next is up to you.
I am always amazed at the willingness of compiler developers to expect the operating system to have safeguards to prevent stack overflow. Of course, that trick works fine until you start having thousands of threads, each with its own stack...
In modern programming languages on a modern operating system, you will either get a stack overflow , or they
malloc()will fail when you try to increment the heap. But not all software is modern, so let's look at the failure modes:sbrk()mmap()If the stack grows and hits the heap, the typical C compiler will start silently overwriting the heap's data structures. On modern operating systems there will be one or more virtual memory save pages that will prevent the stack from growing indefinitely. As long as the amount of guard page memory is at least as large as the size of the procedure's activation record that grows, the OS will guarantee you a segmentation fault ( segfault or segmentation fault ). If you are running your program on DOS or a machine without an MMU... then you have serious problems.
If the heap grows and meets the stack, the operating system must be aware of the situation and some kind of system call will fail. The implementation of
malloc()will definitely catch the failure and returnNULL. What happens next is up to you.I am always amazed at the willingness of compiler developers to expect the operating system to have safeguards to prevent stack overflow. Of course, that trick works fine until you start having thousands of threads, each with its own stack...