des [Wed, 27 Jun 2007 12:56:04 +0000 (12:56 +0000)]
Mostly finish the LRU code and its integration:
- Wrap the storage code so we don't need to duplicate the "toss out some old
crap and try again" logic everywhere. This will also help when / if we
decide to add support for multiple concurrent storage arenas.
- While I'm at it, implement sma_trim().
- Rework the interaction between the LRU and expiry code. Instead of placing
objects retired by the LRU on death row, immediately terminate them.
- Give the LRU code its own fake session and worker so we don't have to pass
it a session pointer.
- Rework the LRU API, and add LRU_DiscardOne() which discards a single
object. This is what the stevedore code uses.
Known or suspected issues:
- The LRU and expiry code should use the same mutex, and / or the possiblity
for races between them should be examined closely.
- LRU_Init() needs to be looked at and possibly moved.
- LRU_DiscardSpace() and LRU_DiscardTime() are unused and quite possibly useless.
- Logging and statistics related to the LRU need more attention.
phk [Wed, 27 Jun 2007 12:37:34 +0000 (12:37 +0000)]
Make sure to reset the workspaces of the backend connection between
requests, otherwise we end up filling it up with Content-Length: headers
if we manage to keep the backend busy.
Also make the snapshot of the preferred workspace-size volatile to ensure
that we are consistent.
des [Tue, 26 Jun 2007 11:52:25 +0000 (11:52 +0000)]
Further refine -n handling. The argument to -n is now either an absolute
path, or a path relative to $localstatedir/varnish. By default, it is set
to the host name, which results in $localstatedir/varnish/$hostname.
This logic is centralized in instance.c, which is compiled into both
varnishd and libvarnishapi, with prototypes in varnishapi.h and heritage.h.
phk [Mon, 25 Jun 2007 21:12:17 +0000 (21:12 +0000)]
Redo the -n argument code, it had too many problems:
We need to process -P and -f arguments before we change directory.
(ticket 120)
(XXX: what about storage and hash arguments ??)
The daemon(3) call should not change our directory subsequently.
(ticket 121)
There is no need to enforce a hostname style format on the argument,
a directory nam makes much more sense, since that is what we need.
Defaulting to /tmp instead of our hostname makes more sense (ticket 119).
This also allows the admin to use a different directory if /tmp is
mounted noexec (ticket 111)
Put the directoryname used in the proctitle (via heritage)
XXX: for docs: vcl.load CLI commands will work relative to the -n directory.
des [Mon, 25 Jun 2007 17:04:09 +0000 (17:04 +0000)]
First step in implementing early retirement of objects when the cache fills
up: implement a "sloppy" LRU list. An object is placed on the list (or moved
to the head of the list if it's already on it and hasn't moved recently) by
calling LRU_Enter(), and removed by calling LRU_Remove(). LRU_DiscardSpace()
will iterate through the LRU list, starting at the back, and retire objects
(by adding them to the deathrow list) until the sum of the length of the
retired objects reaches a certain number. Similarly, LRU_DiscardTime() will
retire objects which haven't moved since a specified cutoff date. In both
cases, vcl_discard() will be given a chance to inspect the object and veto
its retirement.
Currently, LRU_Enter() and LRU_Remove() are called from HSH_Lookup() and
HSH_Deref() respectively. There may be better alternatives.
Neither LRU_DiscardSpace() nor LRU_DiscardTime() is currently called from
anywhere. There are a number of issues to consider: for instance, even if
LRU_DiscardSpace() is called when a high-water mark is reached, there is
still a possibility that the cache might fill up before it has had a chance
to finish and the hangman has had a chance to process the deathrow list.
phk [Mon, 25 Jun 2007 08:17:25 +0000 (08:17 +0000)]
Make the VCL compiler complain about attempts to access variables outside
their scope. One example of this is the "req.hash" variable which only
exists in the vcl_hash method.
phk [Mon, 25 Jun 2007 06:46:34 +0000 (06:46 +0000)]
Also tun the compiled VCL through cc(1) and try to load it into mgt process
when -C is specified, this makes it easier to do completeness test on the
VRT interface.
des [Mon, 18 Jun 2007 07:31:50 +0000 (07:31 +0000)]
Further tweak_name() improvements: restructure to reduce indentation; simplify
error handling; use a regexp to check the name syntax; check CLI errors after
the getopt() loop.
cecilihf [Fri, 15 Jun 2007 09:18:06 +0000 (09:18 +0000)]
Added the -n option for specifying a name for varnishd. All files are now stored under /tmp/<name> where
<name> is either a specified name or the hostname. All the varnish tools have also been updated to let the user
specify the name of the varnish instance to use. The name must conform to the hostname standard, but a test
for this is not yet implemented.
knutroy [Thu, 14 Jun 2007 12:08:15 +0000 (12:08 +0000)]
Miscellaneous improvements to regression test framework, most notably
changes regarding how events are processed. We now state what events
we are waiting for when calling run_loop. The central event dispatcher
monitors this list and decides when to pause the loop. Return value
from run_loop is the event name and whatever arguments were sent by
the event creator, if the event triggered no explicit handler, or
otherwise, the return value(s) from the event handler. If subsequent
events occur between a loop-pausing event and the time the loop
actually pauses, such events are queued and eligible candidates for
return value of the next call to run_loop. This way, events will not
be lost accidentally, which might happen in previous revisions.
des [Wed, 13 Jun 2007 11:26:09 +0000 (11:26 +0000)]
Remove the "magic" StartChild, StopChild and LoadVCL test cases. Instead,
Case.pm now has init() and fini() (which can of course be overloaded by
individual test classes) which start / stop the child, and load a VCL
script if the particular test class being run provides one. In addition,
fini() will revert to the initial VCL script ("boot") if a custom script
was loaded by init().
phk [Sun, 10 Jun 2007 08:49:43 +0000 (08:49 +0000)]
Take a shot at light-weight "Vary:" processing.
When we cache an object with a "Vary:" header, we generate
a "vary matching string" which can be used to efficiently
check for compliance when doing a cache lookup.
Only very lightly tested (ie: cnn.com).
For a full description of the reasoning, please see
http://varnish.projects.linpro.no/wiki/ArchitectureVary
des [Wed, 6 Jun 2007 11:24:06 +0000 (11:24 +0000)]
Move parts of tcp.c out into libvarnish. Rename the API from "TCP" to "VSS"
(Varnish Stream Sockets) as I intend to eventually add support for AF_UNIX
sockets.
This also moves the accept filter code out from VSS_listen() (previously
TCP_open()) and into a separate function in tcp.c