Changeset 77

Timestamp:

06/01/05 05:32:25 (4 years ago)

Author:

jtv

Message:

Revamped free-space accounting system

Files:

• TODO (modified) (2 diffs)
• debian/changelog (modified) (1 diff)
• src/memory.c (modified) (6 diffs)
• src/memory.h (modified) (1 diff)
• src/opts.c (modified) (3 diffs)
• src/swaps.c (modified) (2 diffs)

TODO

@@ -1 @@
-
-Unify logging/printing code: print-only or dynamic choice
+
+Print version number with help output; add --version option
- - Ensure minimal repetition of error/warning messages
-
-
-Policy Weaknesses
-
-Current policy is a bit weak when it comes to filesystem caching.  The problem
-is that cache populations grow and grow until they hit the limit of available
-physical memory--creating an arguably false impression of memory shortage.  The
-flexibility of the Linux virtual memory subsystem makes this, as far as I can
-see at this stage, impossible to diagnose directly.
-
-Most noticeable symptom: existing swapfiles are freed up sometime between system
-boot (when little data is cached and most physical memory is still free) and the
-moment when ever-growing caches push the percentage of free memory below our
-daemon's allocation threshold.  We end up allocating a new swapfile shortly
-after deallocating our last one, just as the user is getting into the swing of
-things and expects his system to respond smoothly.  While we're setting up that
-new swapfile, it doesn't.  This is hard to anticipate; starting up in bull mode
-may help as a practical matter by deferring the decision to deallocate, but
-only if the associated cooldown period is long enough for filesystem caches to
-fill up again and bring the system into a "steady state."
-
-There are other cases that can trigger this behaviour, though.  Consider this
-situation: some application X eats up huge amounts of virtual memory, most of
-which is in active use and therefore resides in physical memory.  Swap is full
-of rarely-used virtual memory pages.  Little or no filesystem caching occurs.
-Next, X finishes and suddenly we have lots of free physical memory--although
-swap remains densely populated.  I'll need to do some math (probably quite
-simple, once I have the right formula) to figure out what can and cannot happen
-here.
-
-Scenario A: if most of the swapped-out data is touched again before we go into
-bear mode and consider deallocating, we're fine.  We'll end up freeing largely
-unused swap space.
-
-Scenario B: if swap is deallocated at this stage, any remaining data it held
-will have to be read back in a hurry--plus, continued system operation may be
-suboptimal because some of that data is "colder" than filesystem data that could
-have been cached in the physical memory that X left free in its wake.
-
-Scenario C: if the "data vacuum" left by X is filled up by filesystem caches, we
-may end up thinking (once again) that we're short on memory--and allocating swap
-space that we won't be needing at all.  Bull mode will be prolonged when a
-return to bear mode is in order.
-
-One consolation: since filesystem caches are never swapped to disk, they cannot
-exceed physical memory size.  This means that the algorithm, for any acceptable
-set of parameters, can "overshoot" by at most a single swapfile.
-
-We could try to distinguish "false shortages" from real memory needs by
-ensuring that there is some "tripwire swap"--perhaps an existing conventional
-swap partition, or a special swapfile numbered zero.  An increase in the
-population of this swap would indicate true memory shortage.  The only problem
-is that the system may not resort to using the tripwire swap until it is already
-desperate!  Can we set a positive priority to make the tripwire slightly more
-sensitive?  We'd also have to clean this file up from time to time, because once
-it fills up the mechanism breaks down.
-
-
@@ -66 +10 @@
-inbetween.  Maybe this isn't the best way to do it.
-
-
-
-
+
+
-

debian/changelog

@@ -1 +1 @@
-swapspace (1.3) unstable; urgency=low
-
-
+
+
-  * Unified logging code--it's not pretty, but it's prettier
-  * No longer bother with "negotiable" minimum swapfile sizes
-  * Clarified startup message ("require n extra bytes" to "would prefer []")
-  * Documented existing policy weaknesses, possible solutions
-  * Systematic rounding of swap sizes to page size
+  * Pad help output to align long options names vertically
+  * Fix: "smaller than expected" informational message was printed too often
-
- -- Jeroen T. Vermeulen <jtv@sipa.or.th>  Thu, 26 May 2005 17:00:00 +0700

src/memory.c

@@ -31 +31 @@
-
-
-left unused
-5
-left unused
-8
-
-for percentage of fre
+kept available
+10
+kept available
+6
+
+percentage of availabl
-static int freetarget=25;
-
+/// Configuration item: what percentage of buffer space do we consider "free"?
+static int buffer_meekness=20;
+
-// TODO: Make this adaptive based on actual cache usage, to avoid thrashing?
+// TODO: Any way of detecting actual cache flexibility or minimum size?
-/// Configuration item: what percentage of cache space do we consider "free"?
-5
+8
-
-char *set_freetarget(long long pct)
@@ -58 +62 @@
-{
-  upper_freelimit = (int)pct;
+  return NULL;
+}
+
+char *set_buffer_meekness(long long pct)
+{
+  buffer_meekness = (int)pct;
-  return NULL;
-}
@@ -158 +168 @@
-  memsize_t MemTotal,
-            MemFree,
+            Buffers,
-            Cached,
-            Dirty,
@@ -171 +182 @@
-  switch (inf->entry[0])
-  {
+  case 'B':
+    if (strcmp(inf->entry, "Buffers:")==0)         st->Buffers = inf->value;
+    break;
-  case 'C':
-    if (strcmp(inf->entry, "Cached:")==0)          st->Cached = inf->value;
@@ -227 +241 @@
-  }
-  return init_req != MEMSIZE_ERROR;
+}
+
+
+/// How much buffer space can we expect the system to free up?
+static inline memsize_t buffers_free(const struct memstate *st)
+{
+  return (st->Buffers/100) * buffer_meekness;
-}
-
@@ -265 +286 @@
-   * considered in-use.
-   */
-
+
+
+
+
+
-}
-

src/memory.h

@@ -40 +40 @@
-char *set_upper_freelimit(long long pct);
-char *set_freetarget(long long pct);
+char *set_buffer_meekness(long long pct);
-char *set_cache_meekness(long long pct);
-

src/opts.c

@@ -61 +61 @@
-static const struct option options[] =
-{
-
-
+
+
+
+
-  { "configfile",       'c', at_str,  1, PATH_MAX, set_configfile,
-  "Use configuration file s" },
@@ -70 +72 @@
-  "Run quietly in background" },
-  { "freetarget",       'f', at_num,  2, 99, set_freetarget,
-fre
+availabl
-  { "help",             'h', at_none, 0, 0, set_help,
-  "Display usage information" },
-  { "lower_freelimit",  'l', at_num,  0, 99, set_lower_freelimit,
-space fre
+ availabl
-  { "max_swapsize",     'M', at_num, PAGE_SIZE, LLONG_MAX, set_max_swapsize,
-  "Restrict swapfiles to n bytes" },
@@ -163 +165 @@
-  for (int i=0; i < sizeof(options)/sizeof(*options); ++i)
-  {
+    static const char pad[] = "   ";
-    const char *const pt = argproto(i, true);
-
+
+
+
+
+
+
-        options[i].shortopt,
-        pt,
+        pad+ptlen,
-        options[i].name,
-        argproto(i, false),

src/swaps.c

@@ -504 +504 @@
-              expected,
-              found);
- < (expected + PAGE_SIZE)
++2*PAGE_SIZE < expected
-          log_discrep(LOG_INFO,
-              "smaller than expected",
@@ -566 +566 @@
-  char namebuf[30];
-  snprintf(namebuf, sizeof(namebuf), "%d", file);
-str(LOG_NOTICE, "Retiring swapfile", namebuf
+int(LOG_NOTICE, "Retiring swapfile", file
-  if (unlikely(swapoff(namebuf) == -1)) return false;
-  swapfiles[file].size = 0;