]> err.no Git - varnish/log
varnish
16 years agoSome source files (especially in libraries) have embedded test
des [Wed, 13 Feb 2008 13:05:36 +0000 (13:05 +0000)]
Some source files (especially in libraries) have embedded test
programs.  Add a configure option and a corresponding automake
conditional to enable these tests.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2453 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFix a bug relating to vbe_conn statistics counter.
phk [Wed, 13 Feb 2008 11:35:10 +0000 (11:35 +0000)]
Fix a bug relating to vbe_conn statistics counter.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2452 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoGo over the backend code and privatize a lot of stuff
phk [Wed, 13 Feb 2008 10:59:25 +0000 (10:59 +0000)]
Go over the backend code and privatize a lot of stuff
that now belongs only in cache_backend.c

Control the caching of bereq's via the parameter.

Found no bugs, so I think I will release -trunk for
adventurous testers again.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2451 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd two new stat fields for counting bereq's and backends.
phk [Wed, 13 Feb 2008 10:57:17 +0000 (10:57 +0000)]
Add two new stat fields for counting bereq's and backends.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2450 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd a parameter for controlling caching of vbe_conn structures.
phk [Wed, 13 Feb 2008 10:56:36 +0000 (10:56 +0000)]
Add a parameter for controlling caching of vbe_conn structures.

(the code that it controls follows in a minute)

I'm very interested to hear if there is any measurable effect
of this variable on heavily loaded systems.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2449 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFix example to new backend syntax
phk [Wed, 13 Feb 2008 10:55:03 +0000 (10:55 +0000)]
Fix example to new backend syntax

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2448 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd information to \"-a\" about how to listen on multiple hosts or ports
ssm [Tue, 12 Feb 2008 06:52:33 +0000 (06:52 +0000)]
Add information to \"-a\" about how to listen on multiple hosts or ports

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2447 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoUpdate backend declaration syntax. Note that a) vcl.7 needs a partial
des [Mon, 11 Feb 2008 10:46:09 +0000 (10:46 +0000)]
Update backend declaration syntax.  Note that a) vcl.7 needs a partial
rewrite to track this change, and b) there have been other changes which
also need to be merged in.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2446 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoSeparate the handling of backend and client requests, greatly simplifying
des [Mon, 11 Feb 2008 02:58:41 +0000 (02:58 +0000)]
Separate the handling of backend and client requests, greatly simplifying
both.

Skip piped requests, and requests that are interrupted by an error.  This
should eliminate or at least greatly reduce the (null)s seen in logs.

Fix a conditional that blocked backend requests from being printed.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2445 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAlphabetize command-line options.
des [Sun, 10 Feb 2008 18:00:33 +0000 (18:00 +0000)]
Alphabetize command-line options.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2444 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years ago<sys/types.h> required by <sys/mman.h>
phk [Sat, 9 Feb 2008 15:59:21 +0000 (15:59 +0000)]
<sys/types.h> required by <sys/mman.h>

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2443 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake the random director work, and remove the old code.
phk [Thu, 7 Feb 2008 09:54:42 +0000 (09:54 +0000)]
Make the random director work, and remove the old code.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2442 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoEmit the vcl_name from the backend host.
phk [Thu, 7 Feb 2008 09:53:28 +0000 (09:53 +0000)]
Emit the vcl_name from the backend host.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2441 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoRemove ident string from directors, they are not recycled.
phk [Thu, 7 Feb 2008 09:52:26 +0000 (09:52 +0000)]
Remove ident string from directors, they are not recycled.

Add vcl_name to backend hosts.  Simple backends get the obvious
name.  Directors inlined backend hosts gets the directors name
with an array suffix, for instance "b1[1]".

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2440 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoUse the correct ident (nuke the other one) and do only one dns lookup.
phk [Thu, 7 Feb 2008 09:33:30 +0000 (09:33 +0000)]
Use the correct ident (nuke the other one) and do only one dns lookup.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2439 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years ago_islocked_np() was renamed to isowned_np() in FreeBSD.
des [Thu, 7 Feb 2008 01:40:15 +0000 (01:40 +0000)]
_islocked_np() was renamed to isowned_np() in FreeBSD.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2438 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFirst part of major backend overhaul.
phk [Wed, 6 Feb 2008 15:19:49 +0000 (15:19 +0000)]
First part of major backend overhaul.

*** Please do not use -trunk in production until I say so again ***

I have not entirely decided in the precise terminology, so the following
may sound a lot more complicated than it really is:

In VCL we can now have "backends" and "directors" both of which we
treat as a "backend".

When we define backends and directors in VCL, they refer to "backend
hosts" which is just another way to say "hostname+portname" but later
these will grow other parameters (max connections etc).

A director is a piece of code that selects a "backend host" somehow,
"random" and "round-robin" are the first algorithms.  A backend
can still be specified directly of course, that's the "simple director"
that always return the same "backend host".

This is probably where an example is in order:

/* A backend as we know it */
backend b1 {
.host = "fs";
.port = "80";
}

/* A director */
director b2 random {
{
/* We can refer to named backends */
.backend        = b1;
.weight         = 7;
}
{
/* Or define them inline */
.backend        = {
.host = "fs2";
}
.weight         = 3;
}
}

sub vcl_recv {
if (req.url ~ "\[[a-z]]") {
set req.backend = b2;
} else {
set req.backend = b1;
}
}

This results in quite a lot of changes in the C code, VRT API and
VCL compiler, the major thrust being:

Directors like "simple" and "random" will not have to think about
the actual connections to the backends, but just concentrate on
selecting which backend should be used.

When a new VCL is loaded, it will instantiate all directors, but
try to reuse any preexisting "backend hosts" (which we still
call "backend" in the C code).

This is simple for a backend like "b1" in the example above, but
sligthly more complex for the backend inlined in b2.  The VCL
compiler solves this, by qualifying the ident string for the inlined
backend host with the prefix "b2 random :: 2 :: ", so that a reload
of the same director with the same (unchanged) inline backend host
will match, but none other will.

One implication of instantiating all directors for every VCL load,
is that private statistics cannot be reused, but stats on the
backend host can.  This is likely a very fine point of no consequence.

Once the backend is selected by the director, the generic code in
cache_backend.c will cope with reusing the connection pool,
establishing connections and all that, moving most of the nastyness
out of directors, leaving cache_dir_simple.c with only 96 lines of
code, of which the license is a large fraction.

Until now, we have done automatic DNS re-lookups, but they seem to
cause more grief than advantage (I suspect some of the DNS lookups
to be resposible for long timeouts), so that will be dropped, and
instead we might add an explicit CLI command for this later.

The code as here committed can handle a couple of simple requests,
but there are a large number of INCOMPL()'s that need to be resolved
before this is ready for prime time again.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2437 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoVBE_AddBackend() needs to always return the backend, so make it a pointer arg
phk [Wed, 6 Feb 2008 09:47:24 +0000 (09:47 +0000)]
VBE_AddBackend() needs to always return the backend, so make it a pointer arg
and use the return int to tell if it was a reuse.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2436 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoI forgot to include the vcl_name of the random director in the structure.
phk [Wed, 6 Feb 2008 09:43:22 +0000 (09:43 +0000)]
I forgot to include the vcl_name of the random director in the structure.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2435 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd more settings to vcl-mode.el, and make them customizable
ssm [Wed, 6 Feb 2008 07:26:28 +0000 (07:26 +0000)]
Add more settings to vcl-mode.el, and make them customizable

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2434 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd vcl_fetch to list of subroutines
ssm [Wed, 6 Feb 2008 06:36:58 +0000 (06:36 +0000)]
Add vcl_fetch to list of subroutines

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2433 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoPut vcl-mode.el nearer to working order
ssm [Tue, 5 Feb 2008 19:41:11 +0000 (19:41 +0000)]
Put vcl-mode.el nearer to working order
* Clean up unused functions
* Set default indent-level to 8
* Actually provide 'vcl-mode (quite useful)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2432 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd the missing bits to actually allow discards of VCL code.
phk [Tue, 5 Feb 2008 11:19:22 +0000 (11:19 +0000)]
Add the missing bits to actually allow discards of VCL code.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2431 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd a discard flag to the VCL_conf.
phk [Tue, 5 Feb 2008 10:50:33 +0000 (10:50 +0000)]
Add a discard flag to the VCL_conf.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2430 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoRename VBE_NewBackend() to VBE_AddBackend() and make it responsible for
phk [Tue, 5 Feb 2008 09:57:46 +0000 (09:57 +0000)]
Rename VBE_NewBackend() to VBE_AddBackend() and make it responsible for
the identity check for reusing backends between VCL.

Disable the round-robin code for now, I'm trying to get the random code
working first.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2429 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoWe assume that all CLI processing in the child process happens in the
phk [Tue, 5 Feb 2008 09:46:30 +0000 (09:46 +0000)]
We assume that all CLI processing in the child process happens in the
same thread (the main thread in fact).

Add an assert facility to mark the affected functions, just to make sure
nobody (else) gets an anciety attack over missing locking.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2428 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agocli_out() already has NULL check.
phk [Tue, 5 Feb 2008 09:31:00 +0000 (09:31 +0000)]
cli_out() already has NULL check.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2427 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd a VCL major mode for emacs
ssm [Tue, 5 Feb 2008 08:26:31 +0000 (08:26 +0000)]
Add a VCL major mode for emacs

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2426 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoBackend fini functions are methods of the class, call the generic one.
phk [Mon, 4 Feb 2008 09:58:55 +0000 (09:58 +0000)]
Backend fini functions are methods of the class, call the generic one.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2425 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd X-ref definition of random directors.
phk [Mon, 4 Feb 2008 09:55:53 +0000 (09:55 +0000)]
Add X-ref definition of random directors.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2424 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoComplete the VCC part of random directors, runtime code next.
phk [Mon, 4 Feb 2008 09:51:23 +0000 (09:51 +0000)]
Complete the VCC part of random directors, runtime code next.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2423 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoTypo patrol
ssm [Mon, 4 Feb 2008 09:35:53 +0000 (09:35 +0000)]
Typo patrol

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2422 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd documentation for req.hash
ssm [Mon, 4 Feb 2008 09:32:23 +0000 (09:32 +0000)]
Add documentation for req.hash

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2421 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoGC support code for the old backend specification code.
phk [Mon, 4 Feb 2008 09:26:36 +0000 (09:26 +0000)]
GC support code for the old backend specification code.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2420 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoClean up checks for non-portable pthread extensions, and add a check for
des [Sun, 3 Feb 2008 22:27:15 +0000 (22:27 +0000)]
Clean up checks for non-portable pthread extensions, and add a check for
pthread_mutex_islocked_np() (not present on any platform I know of, but I
am testing a FreeBSD patch)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2419 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoCentralize <pthread_np.h> include, and use pthread_mutex_islocked_np() for
des [Sun, 3 Feb 2008 22:26:25 +0000 (22:26 +0000)]
Centralize <pthread_np.h> include, and use pthread_mutex_islocked_np() for
ALOCKED() if available.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2418 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoLook for the new SF_SYNC facility in FreeBSDs sendfile(2), and if we
phk [Sun, 3 Feb 2008 15:59:01 +0000 (15:59 +0000)]
Look for the new SF_SYNC facility in FreeBSDs sendfile(2), and if we
find it, allow its use, but still default to off via sendfile_threshold
paramter.

SF_SYNC is only available in FreeBSD-current as of a few seconds
ago, and is unlikely to appear in any release before FreeBSD-8.0
for intricate reasons of ABI compliance.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2417 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd an ALOCKED() macro which asserts that a mutex is locked. Unfortunately,
des [Sat, 2 Feb 2008 10:58:05 +0000 (10:58 +0000)]
Add an ALOCKED() macro which asserts that a mutex is locked.  Unfortunately,
there is no portable way to do this, so we have to fake it by trying to lock
the mutex and assert that it fails.  This can be very expensive, so we only
do it when built with --enable-diagnostics.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2416 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoUpdate copyright; also convert a couple of files from ISO-8859-1 to UTF-8.
des [Thu, 31 Jan 2008 11:57:51 +0000 (11:57 +0000)]
Update copyright; also convert a couple of files from ISO-8859-1 to UTF-8.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2415 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMention that varnishd(1) has a list of commands and parameters, and
des [Thu, 31 Jan 2008 11:41:32 +0000 (11:41 +0000)]
Mention that varnishd(1) has a list of commands and parameters, and
also how to obtain them from varnishd itself.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2414 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd flexelint files
phk [Wed, 30 Jan 2008 10:33:17 +0000 (10:33 +0000)]
Add flexelint files

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2413 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoSilence FlexeLint
phk [Wed, 30 Jan 2008 10:32:52 +0000 (10:32 +0000)]
Silence FlexeLint

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2412 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoSilence FlexeLint: use a union for sockaddr sizing, mark ignored returns.
phk [Wed, 30 Jan 2008 10:30:39 +0000 (10:30 +0000)]
Silence FlexeLint: use a union for sockaddr sizing, mark ignored returns.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2411 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFlexelint silencing: a (), a couple of const and cast.
phk [Wed, 30 Jan 2008 10:24:54 +0000 (10:24 +0000)]
Flexelint silencing: a (), a couple of const and cast.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2410 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoSilence FlexeLint: ignore returns, a constification and a couple of asserts.
phk [Wed, 30 Jan 2008 10:20:54 +0000 (10:20 +0000)]
Silence FlexeLint: ignore returns, a constification and a couple of asserts.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2409 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoSilence Flexelint
phk [Wed, 30 Jan 2008 10:15:07 +0000 (10:15 +0000)]
Silence Flexelint

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2408 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFlexelint silencing
phk [Wed, 30 Jan 2008 10:12:58 +0000 (10:12 +0000)]
Flexelint silencing

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2407 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFlexelint silencing: constification, mark ignored returnvalues etc.
phk [Wed, 30 Jan 2008 10:08:42 +0000 (10:08 +0000)]
Flexelint silencing: constification, mark ignored returnvalues etc.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2406 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAssert non-NULL pointer return.
phk [Wed, 30 Jan 2008 10:03:51 +0000 (10:03 +0000)]
Assert non-NULL pointer return.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2405 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake sure we get the right assert() macro.
phk [Wed, 30 Jan 2008 10:02:19 +0000 (10:02 +0000)]
Make sure we get the right assert() macro.

Constification.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2404 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoRemove unused #include
phk [Wed, 30 Jan 2008 10:01:13 +0000 (10:01 +0000)]
Remove unused #include

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2403 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoCheck various return values (or not) to reduce Flexelint noise.
phk [Wed, 30 Jan 2008 09:50:26 +0000 (09:50 +0000)]
Check various return values (or not) to reduce Flexelint noise.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2402 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoI am not sure if this is a/the race some users are seeing, or if it
phk [Tue, 29 Jan 2008 16:05:54 +0000 (16:05 +0000)]
I am not sure if this is a/the race some users are seeing, or if it
even can have any effect, but this will close it at a cost of one
extra kevent(2) every 100ms timer tick.

The (perceived) problem is that we have pending kqueue changes we
have not yet told the kernel about, then close a number of expired
FD's which might be instantly be recycled by the accept(2) over in
the other thread before we tell the kernel about the pending changes.

In that case, the kernel has no way of knowing that our changes
referred to the previous instance of the fd and not the new one.

The solution is to push the changes to the kernel before servicing
the timer.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2401 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMore FlexeLint inspired cleanup
phk [Tue, 29 Jan 2008 11:53:41 +0000 (11:53 +0000)]
More FlexeLint inspired cleanup

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2400 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFlexelint inspired polishing
phk [Tue, 29 Jan 2008 11:39:27 +0000 (11:39 +0000)]
Flexelint inspired polishing

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2399 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agomissing #ifdef HAVE_SENDFILE
phk [Tue, 29 Jan 2008 11:17:37 +0000 (11:17 +0000)]
missing #ifdef HAVE_SENDFILE

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2398 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoSpecifically mark return values we do not wish to check.
phk [Tue, 29 Jan 2008 11:14:22 +0000 (11:14 +0000)]
Specifically mark return values we do not wish to check.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2397 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoClose another possible NULL dereference
phk [Tue, 29 Jan 2008 10:59:01 +0000 (10:59 +0000)]
Close another possible NULL dereference

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2396 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFix a possible NULL dereference.
phk [Tue, 29 Jan 2008 10:57:30 +0000 (10:57 +0000)]
Fix a possible NULL dereference.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2395 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoUse getdtablesize() to size bitmap if we have it.
phk [Tue, 29 Jan 2008 08:45:40 +0000 (08:45 +0000)]
Use getdtablesize() to size bitmap if we have it.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2394 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd a check to ensure that the event handler is never called by foreign
phk [Tue, 29 Jan 2008 08:14:49 +0000 (08:14 +0000)]
Add a check to ensure that the event handler is never called by foreign
threads.

Inspired by, but not expected to have any effect on #199

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2393 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFinish (the "easy") part of degraded mode:
phk [Mon, 28 Jan 2008 10:28:24 +0000 (10:28 +0000)]
Finish (the "easy") part of degraded mode:

Add req.grace timer:  We only serve degraded mode objects if both the
request and the object's grace timers are satisfied.

Sort expiry list on obj.ttl + obj.grace and fiddle list if either changes.

In the hash lookup: record if any objects still in grace by obj.grace which
match our Vary: criteria (if any).

If no in-ttl object was found AND we have a graced object AND it is also
graced by req.grace AND it is being fetched: serve the graced object.

Otherwise, mark us as successor to the graced object while we fetch to
give others the chance.

When we unbusy the object, clean the magic pointers between the two
objects again.

To play with this you need at least:

sub vcl_recv {
set req.grace = 2m;
}
sub vcl_fetch {
set obj.grace = 2m;
}

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2392 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoIntroduce obj.grace variable:
phk [Mon, 28 Jan 2008 09:28:47 +0000 (09:28 +0000)]
Introduce obj.grace variable:

The amount of time after obj.ttl this object can be served, provided
we are already trying to fetch a new copy.

Nothing inspects this variable yet.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2391 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoInstead of sleeping as soon as we see a busy object, traverse the rest
phk [Mon, 28 Jan 2008 09:09:12 +0000 (09:09 +0000)]
Instead of sleeping as soon as we see a busy object, traverse the rest
of the objects on the objecthead to see if there is anything we can use.

This unpessimizes Vary: processing, where we previously might go to sleep
on a busy object despite the fact that we have a good and valid object
with the Vary: we desire.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2390 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoWith Vary, Prefetch and degraded mode, a session sleeps not on a
phk [Mon, 28 Jan 2008 09:01:38 +0000 (09:01 +0000)]
With Vary, Prefetch and degraded mode, a session sleeps not on a
particular object, because we cannot know beforehand if it will work out
for us, but sleeps on any one of potentially multiple busy objects becoming
ready for us to test against.

Therefore it makes sense to move the waiting list from the object to the
object head, as this both simplifies the code and eliminates a refhold on
busy objects.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2389 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoDeoptimize the central object matching loop in the hash code:
phk [Mon, 28 Jan 2008 08:46:15 +0000 (08:46 +0000)]
Deoptimize the central object matching loop in the hash code:

With the advent of prefetch and degraded mode, the invariants of
objectheads change so that more than one object can be busy at any
one time.

Thus we can no longer assume that the busy object or one subsequent to
it, is the one we eventually desire, and we must start our search from
the front of the list again.

As an amusing sidenote: this eliminates the only "goto" in all of varnishd.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2388 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoDon't print strerror(errno) unless errno != 0.
des [Fri, 25 Jan 2008 16:00:42 +0000 (16:00 +0000)]
Don't print strerror(errno) unless errno != 0.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2387 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoRoundup of old uncommitted changes: Getopt::Long cleanup, IO::Multiplex
des [Fri, 25 Jan 2008 15:38:18 +0000 (15:38 +0000)]
Roundup of old uncommitted changes: Getopt::Long cleanup, IO::Multiplex
cleanup, statistics.  Also improve banning, and avoid // which is only
available in very recent Perl versions.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2386 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoDebian packaging: Add an override to the lintian program to stop it from complaining...
ssm [Thu, 24 Jan 2008 06:24:28 +0000 (06:24 +0000)]
Debian packaging: Add an override to the lintian program to stop it from complaining about a false positive

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2383 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoImprove readability, such as it is.
des [Wed, 23 Jan 2008 20:23:20 +0000 (20:23 +0000)]
Improve readability, such as it is.

Allow the user to disable specific acceptor mechanisms (e.g. do not
use epoll even though it is available).

Don't look for kqueue on systems where we know it doesn't work.

Bail if no acceptor mechanism was found.

Bail if no curses or ncurses was found.

Warn the user if SO_{RCV,SND}TIMEO are non-functional.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2382 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoIt is possible for VSS_parse() to succeed and return a NULL addr but a
des [Wed, 23 Jan 2008 16:23:28 +0000 (16:23 +0000)]
It is possible for VSS_parse() to succeed and return a NULL addr but a
non-NULL port (e.g. ":80" which is a valid listening address).  In that
case, port should be free()d before returning.

Coverity Scan (CID:15)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2379 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoImplement URL banning. It is now possible to use the Apache http
des [Wed, 23 Jan 2008 16:11:31 +0000 (16:11 +0000)]
Implement URL banning.  It is now possible to use the Apache http
manual as a test data set.

Consistently parenthesize print().

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2378 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoDon't assume that res0 != NULL automatically means i == 0. I can't say for
des [Wed, 23 Jan 2008 15:45:03 +0000 (15:45 +0000)]
Don't assume that res0 != NULL automatically means i == 0.  I can't say for
sure (without more coffee) that the assumption is incorrect, but it makes
the code gratuitously non-transparent.

Coverity Scan (CID:8)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2375 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd -c and -r options:
des [Wed, 23 Jan 2008 13:45:48 +0000 (13:45 +0000)]
Add -c and -r options:

- If the former is specified, fetcher will go into a loop after having
  traversed the entire tree, and continuously re-fetch all known URLs.

- The latter is not yet implented, but the idea is to assign a random
  probability to each URL based on an inverse-exponential (or similar)
  distribution, and re-fetch URLs at random according to this frequency.
  This will help simulate a "short head long tail" scenario.

Some restructuring.

Add a comment about a possible improvement which will help work around
bugs in certain commonly used data sets (e.g. the Apache httpd manual)
which can result in an infinite set of URLs (which in reality map to
a fairly large but finite set of pages)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2374 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAlways dispose of the reason from mgt_cli_askchild() properly.
phk [Wed, 23 Jan 2008 10:20:12 +0000 (10:20 +0000)]
Always dispose of the reason from mgt_cli_askchild() properly.

Coverty Scan (CID: 12-14,16)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2373 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agomgt_cli_askchild() always returns a pointer.
phk [Wed, 23 Jan 2008 10:13:43 +0000 (10:13 +0000)]
mgt_cli_askchild() always returns a pointer.

Coverity Scan (CID:12)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2372 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake sure to always NULL the pointer argument to cli_readres().
phk [Wed, 23 Jan 2008 10:05:45 +0000 (10:05 +0000)]
Make sure to always NULL the pointer argument to cli_readres().

Implicated in Coverity Scan (CID:12-16)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2371 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoProtect all vsb's (sbufs) with an assert that they did not overflow.
phk [Wed, 23 Jan 2008 09:46:13 +0000 (09:46 +0000)]
Protect all vsb's (sbufs) with an assert that they did not overflow.

I don't think it is likely that they would, but some users are running
out of memory, so make it deterministic when it happens.

Noticed by Coverity Scan (CID: 4-6)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2370 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake sure WRK_Flush() always resets w->niov so WRK_Write() does not
phk [Wed, 23 Jan 2008 09:26:03 +0000 (09:26 +0000)]
Make sure WRK_Flush() always resets w->niov so WRK_Write() does not
overrun the w->iov.

Because niov is right after iov in struct worker, it is hard to predict
what the effect of hitting this bug, but "core dump" is almost a given.

I don't think it has been likely to happen a lot however, as it would
require a full complement of HTTP headers or a very fragmented object.

Coverity Scan (CID:7)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2369 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd helper functions to check and report errors when fields are defined
phk [Tue, 22 Jan 2008 11:39:58 +0000 (11:39 +0000)]
Add helper functions to check and report errors when fields are defined
too much, too little or just plain wrong.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2368 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoParse a random director into an appropriate data structure,
phk [Tue, 22 Jan 2008 10:48:49 +0000 (10:48 +0000)]
Parse a random director into an appropriate data structure,
still bits missing.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2367 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake it possible to refer to backend hosts (== simple backends) by name.
phk [Tue, 22 Jan 2008 09:55:36 +0000 (09:55 +0000)]
Make it possible to refer to backend hosts (== simple backends) by name.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2365 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoDebian packaging: update changelog
ssm [Tue, 22 Jan 2008 08:36:07 +0000 (08:36 +0000)]
Debian packaging: update changelog

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2361 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoBail if there are multiple .host or .port fields in one backend host.
phk [Tue, 22 Jan 2008 08:32:23 +0000 (08:32 +0000)]
Bail if there are multiple .host or .port fields in one backend host.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2360 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoDebian packaging: Only reload varnishlog if it is running.
ssm [Tue, 22 Jan 2008 08:28:18 +0000 (08:28 +0000)]
Debian packaging: Only reload varnishlog if it is running.
(Debian bug #462029)

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2359 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoDebian packaging: Add a debian revision to the version number
ssm [Mon, 21 Jan 2008 19:04:41 +0000 (19:04 +0000)]
Debian packaging: Add a debian revision to the version number

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2357 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoStart the long awaited overhaul of the backend code and syntax.
phk [Mon, 21 Jan 2008 13:09:17 +0000 (13:09 +0000)]
Start the long awaited overhaul of the backend code and syntax.

Off the bat, this affects only the simple backend case, and the
main thrust is a syntax change from a dynamic procedural assignment
style:

backend b1 {
set backend.host = "fs.freebsd.dk";
set backend..port = "80";
}

to a constant structural definition style:

backend b1 {
.host = "fs.freebsd.dk";
.port = "80";
}

A helpfull compile error will advice on this change.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2356 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake sure to terminate the movement of a request's body if we get an
phk [Fri, 18 Jan 2008 10:42:27 +0000 (10:42 +0000)]
Make sure to terminate the movement of a request's body if we get an
EOF from the client.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2355 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoUse 192.0.2.0/24 in our examples, this net is reserved for documentation in RFC3330
ssm [Fri, 18 Jan 2008 07:35:17 +0000 (07:35 +0000)]
Use 192.0.2.0/24 in our examples, this net is reserved for documentation in RFC3330

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2354 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoRework the handling of message-bodies in requests to be closer to
phk [Wed, 16 Jan 2008 15:15:39 +0000 (15:15 +0000)]
Rework the handling of message-bodies in requests to be closer to
RFC2616 and catch things we don't deal with better.

Move code to receive body from request into separate function,
FetchReqBody() so it can be invoked when center::STP_HIT.

Check read errors and make FetchReqBody() return an error code for
client/server side so we don't penalize backends for client trouble.

Make it a client error, for now, to send chunked encoding.

This also isolates the big stack buffer from the subsequent function
calls in Fetch() and will be saner if we ever add chunked-encoding
handling.

Only send the body to the backend on pass, otherwise discard it.

This is controlled via the sp->sendbody flag which is reset in at
STP_RECV and set it at STP_PASS.

In STP_HIT, on deliver, call FetchReqBody() to dispose of any body
the request might contain.

Filter Content-Length: headers out (c->b) on fetches, but not for pass.

Rewrite default vcl_recv{} to be take advantage of our new capabilities.

This should make PUT, DELETE, OPTIONS, and TRACE work as well as POST.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2353 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMissed an assignment in last commit.
phk [Wed, 16 Jan 2008 14:53:48 +0000 (14:53 +0000)]
Missed an assignment in last commit.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2352 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake the destination pointer optional to http_GetHdr(), we might just
phk [Wed, 16 Jan 2008 14:49:27 +0000 (14:49 +0000)]
Make the destination pointer optional to http_GetHdr(), we might just
want to know if the header is there at all.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2351 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMark return value as intentionally unused.
phk [Wed, 16 Jan 2008 11:44:30 +0000 (11:44 +0000)]
Mark return value as intentionally unused.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2350 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMake Flexelint happier about our private ctype.
phk [Wed, 16 Jan 2008 11:39:06 +0000 (11:39 +0000)]
Make Flexelint happier about our private ctype.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2349 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoIntroduce the TRUST_ME() macro for voilating the C compilers pointer
phk [Wed, 16 Jan 2008 11:34:28 +0000 (11:34 +0000)]
Introduce the TRUST_ME() macro for voilating the C compilers pointer
sensibilities.  If only they had done const right...

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2348 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoMove the enum out of the struct to cater for marginal pedantism on
phk [Wed, 16 Jan 2008 11:27:34 +0000 (11:27 +0000)]
Move the enum out of the struct to cater for marginal pedantism on
the part of FlexeLint.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2347 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agofix spelling mistake reported in #183
phk [Wed, 16 Jan 2008 10:15:27 +0000 (10:15 +0000)]
fix spelling mistake reported in #183

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2346 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoFix ticket #179: non-ascii reason-phrase.
phk [Wed, 16 Jan 2008 10:08:34 +0000 (10:08 +0000)]
Fix ticket #179: non-ascii reason-phrase.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2345 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoPave more road for prefetch:
phk [Mon, 14 Jan 2008 10:42:56 +0000 (10:42 +0000)]
Pave more road for prefetch:

Sanity-check values assigned to obj.prefetch from VCL: if before present
time or after ttl, SHM a VCL_info message and set obj.prefetch to zero
disabling prefetch.

Change objects/timer interaction:  We only keep track of the first timer for
each object (->timer_when) and add a field to remember what we intend to
do at that time (->timer_what).

Rename heap_idx to timer_idx to group the relevant fields (XXX: it should be
accessed through function outside cache_expire.c which should be called
cache_timer.c now).

Abolish the 30 second advance move to death-row.

When the prefetch timer expires, SHM a debug message and don't do anything.

Minor polishing and cleanup.

Add vcl_prefetch{} to default VCL and set prefetch 30 seconds before ttl.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2344 d4fa192b-c00b-0410-8231-f00ffab90ce4

16 years agoAdd SHM tag VCL_info for informative messages from VCL execution.
phk [Mon, 14 Jan 2008 10:30:47 +0000 (10:30 +0000)]
Add SHM tag VCL_info for informative messages from VCL execution.

git-svn-id: svn+ssh://projects.linpro.no/svn/varnish/trunk@2343 d4fa192b-c00b-0410-8231-f00ffab90ce4