Changes between Version 2 and Version 3 of WikiStart

Jul 27, 2005, 10:08:57 AM (14 years ago)



  • WikiStart

    v2 v3  
     1= swapspace: a dynamic swap manager for GNU/Linux =
     3== What it does for you ==
     5This system daemon for the Linux kernel aims to do away with the need for large,
     6fixed swap partitions or swap files.
     8When installing a Linux-based system (invariably GNU/Linux) with ''swapspace'', the usual swap partition can be omitted, or it can be kept quite small.  Whenever ''swapspace'' finds during normal system usage that more virtual memory is needed, it will automatically claim space from the hard disk.  Conversely, swap space that is no longer needed is freed up again for regular use by the filesystem.
     10This means that with ''swapspace'' installed, sizing the system's available swap
     11space during installation is no longer a life-or-death choice.  It now becomes
     12practical to run GNU/Linux off just a single, big partition--with no disk space
     13lost to regrettable installation choices.  The system should also be able to
     14handle the occasional memory-intensive task that takes much more swap space than
     15was originally foreseen, without leaving the same swap space unused and unusable
     16during normal operation as is normally the case.
     18''Swapspace'' is [ free software], made available to you under the [ GNU] [ General Public License] (GPL).
     21== How it compares ==
     23Unlike similar programs such as ''dynswapd'' and the older (and more portable) ''swapd'', ''swapspace'' also adapts the sizes of the swap files it creates to meet demand.  This means it is less dependent on limits that the kernel may impose on the total number of swapfiles, while reducing the need for manual configuration.  If the daemon finds that more and more swap files are needed, it will start creating larger ones to anticipate demand.  While demand for swap files is modest, it will stick to smaller ones that can be initialized more quickly and so respond more fluently to present requirements.
     25Robustness and user-friendliness are the first priorities in developing this
     26program.  For example, all alternatives we looked at perversely needed to
     27allocate multiple chunks of memory in dealing with low-memory situations;
     28allocation failure would typically crash these programs.  It turned out that
     29none of these allocations were really necessary, and ''swapspace'' manages to avoid them categorically.  This kills two birds with one stone when it comes to
     30reliability: (i) the program doesn't ask for memory just when the least memory
     31is available and (ii) it eliminates one of the most important causes of
     32programming bugs as a risk factor.
     34User-friendliness primarily means that no silly questions are asked of the user.
     35The daemon tries to be sensible and figure out what is needed at runtime, by
     36itself, and without user intervention.  You should not have any need to learn
     37about the configuration parameters, or tweak them.  They exist mostly for
     38development purposes.
     40The swapspace daemon has been in production use for several months on various
     4132-bit architectures before it was first released to the public, and has been
     42tested with swapfiles larger than can be addressed in 32 bits.  Some statistics
     43for one test on a 32-bit PC with 1 gigabyte of memory:
     45        Number of swapfiles     25
     46        Largest swapfile        5.6 GB
     47        Total swap allocated    44 GB
     49In other words, applications on this 1 GB desktop computer took up a whopping
     50(by 2005 standards) 44 GB of memory space without falling over.  They weren't
     51very fast, of course, having to swap so much; but they did keep running.
     53After this point, when we tried to use up even more memory, the hard disk filled
     54up with swap files so no more could be allocated.  The program has provisions to
     55deal with this case robustly and fairly gracefully.  It does not like to leave
     56you with a 100% full disk with no room to save your data.  After we closed some
     57of the running applications, swapspace started steadily decommissioning swap
     58files before settling in a more realistic state.
     60Swapspace itself is a small program: about 50 kilobytes on my system--or even
     61less in a special version that only accepts the most basic configuration options
     62and ignores its configuration file.  On top of that it allocates no memory at
     63runtime (although the system will allocate some, of course) and does not use
     64much stack space.
     67== When not to use it ==
     69In its current form, swapspace is probably not a good choice for systems that
     70need to remain responsive at all times; depending on the system and the
     71cicrumstances, the creation a large new swapfile can take as long as half a
     72minute and occupy quite a lot of the system's attention.  The program minimizes
     73the number of times swapfiles are created, but it wouldn't be very useful if it
     74never created any swapfiles at all!
     76We are hoping to bring further improvements in the future.  Since the problem
     77appears to be caused mostly by system code, however, it's hard to be sure that
     78this is really possible.  It may turn out to be possible for the kernel, with
     79some modifications, to extend an existing swapfile while it is already in use.
     80That would probably help a great deal, but we don't know at the moment how much
     81work it would take.
     84== Where to start ==
     86The program is available both as a source archive and as a Debian package built
     87from that same source archive.
     89To build and install from source, enter the main swapspace source directory and
     90run {{{make}}}.  Some editing of the Makefile may, but generally shouldn't, be
     91required as long as gcc is used as the C compiler.  The program code is written
     92in standard C99 (the 1999 edition of the C standard), plus one POSIX extension.
     93The easiest mode of installation is by running {{{make install}}} with root
     96See the provided manpage for details on how to run swapspace for troubleshooting
     97or debugging purposes.  A sample configuration file is also provided, but the
     98average user should not need to take an interest.  An init script is provided to
     99start and stop swapspace as a regular system service.
     101The "make install" procedure installs the init script in ''/etc/init.d'', but does not currently ensure that swapspace is run on system startup.
     104== Technical details ==
     106=== Installation ===
     108The installation procedure creates a directory ''/var/lib/swapspace'', which must be accessible to the system's superuser (root) __only__; granting any kind of access for this directory to an untrusted user is likely to constitute a serious security hole.
     110According to the Filesystem Hierarchy Standard, other appropriate places for
     111this kind of file might be ''/var/tmp'' or ''/var/run''.  The former was not deemed appropriate because it is accessible to all users, and the latter because most system administrators would probably expect it to occupy very little space and may confine it to a partition that isn't large enough to hold useful swap space.
     113Also, files in ''/var/lib'' survive reboots whereas those in ''/var/tmp'' and ''/var/run'' need not.  Obviously any data in swap files can be safely erased on reboot (it's even tempting to think that that would be safer, though I don't think it really is).  But consider a system consistently short on physical memory: during boot, swapspace will see the swapfiles left by the previous session, and reinstate them immediately at very little cost, ready for use when they are needed.  If they had been erased during reboot, swapspace would have to allocate new ones on disk when it recognized the need for more virtual memory, which takes much more time and resources.
     115=== Algorithm ===
     117Choices of allocation and deallocation are driven by a "finite state machine"
     118consisting of four states: steady, hungry, overfed, and diet.  The program will
     119alternate through these states as circumstances dictate, applying different
     120policies depending on current state.  This was done to achieve a good tradeoff
     121between willingness to free up unused swap space on the one hand, and avoidance
     122of "thrashing" between deallocation and re-allocation of swapfiles on the other.
     124For those interested, here is a quick description of these states and their
     125associated policies:
     127Steady - normal operation; no additional swap space is needed, nor do we have
     128more than is needed.  However, the program can go into the hungry or overfed
     129states at the drop of a hat.
     131Hungry - more swap space was recently allocated.  The program is willing to
     132allocate even more if needed, but it will not consider dropping unneeded swap
     133files.  After a certain timeout period, the program reverts to the steady state.
     135Overfed - significantly more virtual memory is available than is needed.  If
     136this situation persists for a certain timeout period, a swapfile is deallocated
     137and the program returns to the steady state.
     139Diet - a recent allocation attempt has run into resource limits, e.g. because
     140the filesystem used for swap files was full.  No more swapspace can be allocated
     141but excess files can be freed up very rapidly.  Afer timeout, reverts to steady.
     143State information can be queried by sending the program the SIGUSR1 signal (see
     144"man 7 signal" to get the number for your architecture) which will cause it to
     145log debug information to the system's daemon log and/or standard output, as
     146appropriate.  The program can also be made to log state transitions by starting
     147it with the ''-v'' or ''--verbose'' option.
    1150= This site runs on Trac 0.8.4 =