Changeset 92
- Timestamp:
- 07/03/05 04:53:35 (4 years ago)
- Files:
Legend:
- Unmodified
- Added
- Removed
- Modified
- Copied
- Moved
README
r90 r92 11 11 swapspace finds, during normal system usage, that more virtual memory is needed, 12 12 it will automatically claim space from the hard disk. Conversely, swap space 13 that is no longer needed is freed up for regular use by the filesystem.13 that is no longer needed is freed up again for regular use by the filesystem. 14 14 15 15 This means that with swapspace installed, sizing the system's available swap 16 16 space during installation is no longer a life-or-death choice. It now becomes 17 practical to run GNU/Linux off just a single, big partition 17 practical to run GNU/Linux off just a single, big partition--with no disk space 18 18 lost to regrettable installation decisions. The system should also be able to 19 19 handle the occasional memory-intensive task that takes much more swap space than 20 20 was originally foreseen, without leaving the same swap space unused and unusable 21 21 during normal operation. 22 23 Swapspace is made available for use under the GNU General Public License (GPL). 24 See the file COPYING for details. 22 25 23 26 … … 32 35 example, all alternatives we have found will unnecessarily allocate several 33 36 chunks of memory in dealing with low-memory situations. Allocation failure will 34 typically crash these programs, whereas swapspace avoids all dynamic memory35 allocation except as is done internally by the system in setting up swap files.37 typically crash these programs, whereas swapspace categorically avoids all 38 reliance on dynamically allocated memory. 36 39 37 40 User-friendliness primarily means that no silly questions are asked of the user. 38 41 The daemon tries to be sensible and figure out what is needed at runtime, by 39 itself, and without user intervention. It is by no means perfect in this 40 regard, but it is a priority for future improvement. 42 itself, and without user intervention. 41 43 42 44 The swapspace daemon has been in production use for several months on various 43 45 32-bit architectures before it was first released to the public, and has been 44 46 tested with swapfiles larger than can be addressed in 32 bits. Some statistics 45 for one test on a 32-bit PC with onegigabyte of memory:47 for one test on a 32-bit PC with 1 gigabyte of memory: 46 48 47 49 Number of swapfiles 25 48 Largest swapfile >5GB50 Largest swapfile 5.6 GB 49 51 Total swap allocated 44 GB 52 53 After this point the hard disk filled up with swap files, making it impossible 54 to allocate more. The program has provisions to deal with this case robustly 55 and fairly gracefully. After most of the virtual memory was freed up again, it 56 started steadily deallocating swap files before settling in a more realistic 57 state. 50 58 51 59 … … 53 61 54 62 In its current form, swapspace is probably not a good choice for systems that 55 need to remain responsive at all times; the creation a large new swapfile can 56 take as long as half a minute and occupy quite a lot of the system's attention. 63 need to remain responsive at all times; depending on the system and the 64 cicrumstances, the creation a large new swapfile can take as long as half a 65 minute and occupy quite a lot of the system's attention. The program attempts 66 to minimize the frequency of allocations, but cannot avoid them altogether while 67 still being useful. 57 68 58 We are hoping to address this shortcoming in the future , but since the problem59 appears to be caused mostly by system code, it's hard to be sure that this is60 really possible.69 We are hoping to address this shortcoming in the future. Since the problem 70 appears to be caused mostly by system code, however, it's hard to be sure that 71 this is really possible. 61 72 62 73 63 74 Where to start 64 75 65 To build and install from source, enter the main swapspace directory and run 66 "make". Some editing of the Makefile may, but generally shouldn't, be required 67 as long as gcc is used as the C compiler; the program code is written in 68 standard C99. The easiest mode of installation is by running "make install" 69 with root privileges. 76 The program is available both as a source archive and as a Debian package built 77 from that same source archive. 78 79 To build and install from source, enter the main swapspace source directory and 80 run "make". Some editing of the Makefile may, but generally shouldn't, be 81 required as long as gcc is used as the C compiler. The program code is written 82 in standard C99 (the 1999 edition of the C standard), plus one POSIX extension. 83 The easiest mode of installation is by running "make install" with root 84 privileges. 70 85 71 86 See the provided manpage for details on how to run swapspace for troubleshooting … … 77 92 not currently ensure that swapspace is run on system startup. 78 93 79 Swapspace is made available for use under the GNU General Public License (GPL).80 See the file COPYING for details.81 94 95 Technical details: Installation 96 97 The installation procedure creates a directory /var/lib/swapspace, which must be 98 accessible to the system's superuser (root) only; granting any kind of access 99 for this directory to an untrusted user is likely to constitute a serious 100 security hole. 101 102 According to the Filesystem Hierarchy Standard, other appropriate places for 103 this kind of file might be /var/tmp or /var/run. The former was not deemed 104 appropriate because it is accessible to all users, and the latter because most 105 system administrators would probably expect it to occupy very little space and 106 may confine it to a partition that isn't large enough to hold useful swap space. 107 108 Also, files in /var/lib survive reboots whereas those in /var/tmp and /var/run 109 need not. Obviously any data in swap files can be safely erased on reboot (it's 110 even tempting to think that that would be safer, though I don't think it really 111 is). But consider a system consistently short on physical memory: during boot, 112 swapspace will see the swapfiles left by the previous session, and reinstate 113 them immediately at very little cost, ready for use when they are needed. If 114 they had been erased during reboot, swapspace would have to allocate new ones on 115 disk when it recognized the need for more virtual memory, which takes much more 116 time and resources. 117 118 Technical details: Algorithm 119 120 Choices of allocation and deallocation are driven by a "finite state machine" 121 consisting of four states: steady, hungry, overfed, and diet. The program will 122 alternate through these states as circumstances dictate, applying different 123 policies depending on current state. This was done to achieve a good tradeoff 124 between willingness to free up unused swap space on the one hand, and avoidance 125 of "thrashing" between deallocation and re-allocation of swapfiles on the other. 126 127 For those interested, here is a quick description of these states and their 128 associated policies: 129 130 Steady - normal operation; no additional swap space is needed, nor do we have 131 more than is needed. However, the program can go into the hungry or overfed 132 states at the drop of a hat. 133 134 Hungry - more swap space was recently allocated. The program is willing to 135 allocate even more if needed, but it will not consider dropping unneeded swap 136 files. After a certain timeout period, the program reverts to the steady state. 137 138 Overfed - significantly more virtual memory is available than is needed. If 139 this situation persists for a certain timeout period, a swapfile is deallocated 140 and the program returns to the steady state. 141 142 Diet - a recent allocation attempt has run into resource limits, e.g. because 143 the filesystem used for swap files was full. No more swapspace can be allocated 144 but excess files can be freed up very rapidly. Afer timeout, reverts to steady. 145 146 State information can be queried by sending the program the SIGUSR1 signal (see 147 "man 7 signal" to get the number for your architecture) which will cause it to 148 log debug information to the system's daemon log and/or standard output, as 149 appropriate. The program can also be made to log state transitions by starting 150 it with the -v or --verbose option. 151 TODO
r90 r92 1 For some reason it seems to be impossible to enter the "diet" state right now.1 More documentation; find better source format. 2 2 3 After allocating 44 GB in 25 swapfiles of up to 5.8 GB (all decimally rounded 4 from byte counts, not binary gigabytes) the program keeps attempting to allocate 5 the same swapfile. It logs a notice "Allocating swapfile" but the only thing 6 that follows is a transition to "hungry." 3 Work on "eviction policy." Delete swapfile with largest free space? 7 4 8 In one case an "Error writing swapfile" was logged, with the errno of "No space 9 left on device," but the result was the same. Except that until this happened, 10 the program seemed to loop in a state where it was unresponsive to a decrease in 11 memory requirements. It should have gone into "overfed" by then, I think. 5 Custom signals can interrupt creation/decommissioning of swapfiles. I'm not 6 sure this is a bad thing, but it might create a risk of leaking files. 12 7 8 Experiment with different ways of writing data to swapfiles--which is the least 9 cache-intrusive? 10 11 Try to detect situations where the next swapfile will fill up the disk, and 12 don't try to allocate. As a side effect, this may make swapspace respect the 13 filesystem's conservative measurement of free disk space, which seems proper. 14 See coreutils "df" source code for how to find this information. 15 16 Describe the algorithm's finite state machine in Dot. 17 18 Adaptive cooldown times: recognize pattern of consistent policy reversals too 19 close to timeout or too soon after decision is made. 20 21 Test handling of too many swapfiles. 22 23 Test rollover of swapfile numbering, and skipping of occupied slots. 24 25 Allow saving of behaviour model (initially, just cooldown time) so as to reduce 26 warmup time. 27 28 Rephrase "meekness" as "elasticity". 29 30 Document -v|--verbose and any other forgotten options. 31
