Alkaline Virtual Memory and Swap

Any operating system has a fixed amount of physical memory available. Usually, application need more than the physical memory installed on your system, for that purpose the operating system uses a swap mechanism: instead of storing data in physical memory, it uses a disk file.

On operating systems, such as Windows NT, Windows 2000 or UNIX, the memory is logically divided in pages. When the system needs a certain portion of memory which is currently in the swap (this is called a page fault) it will load all the corresponding pages into RAM. When a page is not accessed for a long time, it is saved back to disk and discarded.

If you look on the Windows NT Task Manager or the output from ps or top, Mem Usage is the working set size. It is the amount of physical memory which is directly (currently) allocated to the process. It can be accessed without causing a page fault. This includes pages shared with other processes. The Windows NT VM Size or the UNIX RSS/RES value is the total private virtual memory allocated to the process.

Alkaline has a built in swapping mechanism that uses memory mapped files to simulate virtual memory. Running Alkaline with --enableswap may save you up to 75% of physical memory but may as well significantly decrease indexing and searching performance. The --enabelswap command line option forces Alkaline to create a file, usually of a blib-pid-index format. On Windows NT, you will usually see one or more swap files of 32 MB each with a growing index number in the default temporary directory. On a UNIX system, only one single file is created and you can specify the filename. For example, use --enableswap=/tmp/alkaline.swp . This option is especially helpful because Alkaline does not get a chance to cleanup its swap file if the Alkaline process is killed on a UNIX system. The operating system handles automatic temporary file removal on Windows NT.

You can consider that Alkaline allocates the amount of memory shown under VM Size and RSS/RES. The excess of memory shown in the Mem Usage is all the swapped data that is currently resident because more physical memory is available or because that data is required by the program to run.

Try running Alkaline with --enableswap and notice the large differences in VM Size and RSS/RES values.