phk [Mon, 25 Jun 2007 21:12:17 +0000 (21:12 +0000)]
Redo the -n argument code, it had too many problems:
We need to process -P and -f arguments before we change directory.
(ticket 120)
(XXX: what about storage and hash arguments ??)
The daemon(3) call should not change our directory subsequently.
(ticket 121)
There is no need to enforce a hostname style format on the argument,
a directory nam makes much more sense, since that is what we need.
Defaulting to /tmp instead of our hostname makes more sense (ticket 119).
This also allows the admin to use a different directory if /tmp is
mounted noexec (ticket 111)
Put the directoryname used in the proctitle (via heritage)
XXX: for docs: vcl.load CLI commands will work relative to the -n directory.
des [Mon, 25 Jun 2007 17:04:09 +0000 (17:04 +0000)]
First step in implementing early retirement of objects when the cache fills
up: implement a "sloppy" LRU list. An object is placed on the list (or moved
to the head of the list if it's already on it and hasn't moved recently) by
calling LRU_Enter(), and removed by calling LRU_Remove(). LRU_DiscardSpace()
will iterate through the LRU list, starting at the back, and retire objects
(by adding them to the deathrow list) until the sum of the length of the
retired objects reaches a certain number. Similarly, LRU_DiscardTime() will
retire objects which haven't moved since a specified cutoff date. In both
cases, vcl_discard() will be given a chance to inspect the object and veto
its retirement.
Currently, LRU_Enter() and LRU_Remove() are called from HSH_Lookup() and
HSH_Deref() respectively. There may be better alternatives.
Neither LRU_DiscardSpace() nor LRU_DiscardTime() is currently called from
anywhere. There are a number of issues to consider: for instance, even if
LRU_DiscardSpace() is called when a high-water mark is reached, there is
still a possibility that the cache might fill up before it has had a chance
to finish and the hangman has had a chance to process the deathrow list.
phk [Mon, 25 Jun 2007 08:17:25 +0000 (08:17 +0000)]
Make the VCL compiler complain about attempts to access variables outside
their scope. One example of this is the "req.hash" variable which only
exists in the vcl_hash method.
phk [Mon, 25 Jun 2007 06:46:34 +0000 (06:46 +0000)]
Also tun the compiled VCL through cc(1) and try to load it into mgt process
when -C is specified, this makes it easier to do completeness test on the
VRT interface.
des [Mon, 18 Jun 2007 07:31:50 +0000 (07:31 +0000)]
Further tweak_name() improvements: restructure to reduce indentation; simplify
error handling; use a regexp to check the name syntax; check CLI errors after
the getopt() loop.
cecilihf [Fri, 15 Jun 2007 09:18:06 +0000 (09:18 +0000)]
Added the -n option for specifying a name for varnishd. All files are now stored under /tmp/<name> where
<name> is either a specified name or the hostname. All the varnish tools have also been updated to let the user
specify the name of the varnish instance to use. The name must conform to the hostname standard, but a test
for this is not yet implemented.
knutroy [Thu, 14 Jun 2007 12:08:15 +0000 (12:08 +0000)]
Miscellaneous improvements to regression test framework, most notably
changes regarding how events are processed. We now state what events
we are waiting for when calling run_loop. The central event dispatcher
monitors this list and decides when to pause the loop. Return value
from run_loop is the event name and whatever arguments were sent by
the event creator, if the event triggered no explicit handler, or
otherwise, the return value(s) from the event handler. If subsequent
events occur between a loop-pausing event and the time the loop
actually pauses, such events are queued and eligible candidates for
return value of the next call to run_loop. This way, events will not
be lost accidentally, which might happen in previous revisions.
des [Wed, 13 Jun 2007 11:26:09 +0000 (11:26 +0000)]
Remove the "magic" StartChild, StopChild and LoadVCL test cases. Instead,
Case.pm now has init() and fini() (which can of course be overloaded by
individual test classes) which start / stop the child, and load a VCL
script if the particular test class being run provides one. In addition,
fini() will revert to the initial VCL script ("boot") if a custom script
was loaded by init().
phk [Sun, 10 Jun 2007 08:49:43 +0000 (08:49 +0000)]
Take a shot at light-weight "Vary:" processing.
When we cache an object with a "Vary:" header, we generate
a "vary matching string" which can be used to efficiently
check for compliance when doing a cache lookup.
Only very lightly tested (ie: cnn.com).
For a full description of the reasoning, please see
http://varnish.projects.linpro.no/wiki/ArchitectureVary
des [Wed, 6 Jun 2007 11:24:06 +0000 (11:24 +0000)]
Move parts of tcp.c out into libvarnish. Rename the API from "TCP" to "VSS"
(Varnish Stream Sockets) as I intend to eventually add support for AF_UNIX
sockets.
This also moves the accept filter code out from VSS_listen() (previously
TCP_open()) and into a separate function in tcp.c
des [Fri, 1 Jun 2007 22:18:55 +0000 (22:18 +0000)]
Keep a master copy of the parameter block, to which all changes are applied,
and which is copied to the shared parameter block every time a parameter
changes as well as immediately before forking off a child. This prevents a
hypothetical compromised child from changing the parent's idea of run-time
parameters (which would, for example, allow it to trick the the parent into
starting a new, hypothetically exploitable child with the attacker's choice
of uid / gid).
While I'm here, correct the use of the "volatile" qualifier - it is the
parmeter block itself which can change unpredictably, not the pointer.
des [Thu, 31 May 2007 12:57:30 +0000 (12:57 +0000)]
Add two run-time parameters, "user" and "group", which specify an unprivileged
user and group to which the child process will switch immediately after fork()
returns, before it starts accepting connections. The default values are
"nobody" and "nogroup" (they should probably be tweakable at compile time...)
Note that this does not provide full privilege separation, as there are still
channels between the parent and child processes which need to be monitored,
but it is an improvement on the previous situation.
cecilihf [Wed, 30 May 2007 09:53:49 +0000 (09:53 +0000)]
varnishncsa is now able to produce backend logs. A couple of problems still remain: missing hostname and timestamp for backend communication. This could be solved with some extra tags serving the same purpose as ReqStart and ReqEnd does for client communication, providing this information.
des [Fri, 25 May 2007 10:00:38 +0000 (10:00 +0000)]
Add an API for synthetic objects, and use it to implement negative
caching of backend issues.
Brief summary:
- moved http_msg array from cache_response.c to cache_http.c,
introduced http_StatusMessage() lookup function
- introduced http_Put{Protocol,Status,Response} to complement
http_PrintfHeader().
- introduced SYN_ErrorPage() in a new file, cache_synthetic.c.
SYN_ErrorPage() populates the session's current object with the
specified error code and a corresponding HTML error page; it is the
caller's responsibility to ensure that the session has a suitable
object (i.e. one that doesn't already have headers or a body)
- rewrote RES_Error() to simply call SYN_ErrorPage() (with ttl = 0) and
RES_WriteObj().
- rewrote cnt_fetch() to use SYN_ErrorPage() to create a 503 page with
a TTL of 30 seconds when Fetch() fails.
- removed the call to RES_Error() in cache_backend.c; the error
trickles back up to cnt_fetch() anyway.
Comments from review:
- Memory allocation and pointer gymnastics for the header and body
are duplicated all over the place (in new and pre-existing code)
and should be centralized and hidden behind a suitable API.
- The http_*() API needs refactoring, we shouldn't need four
different functions to manipulate four different entries in the
same array.
projects / varnish / log