des [Wed, 2 May 2007 14:37:42 +0000 (14:37 +0000)]
Don't try to lock the objhead mutex if we don't have an objhead. Doing so
caused the Varnish child to die immediately after sending its 503 response
if the backend didn't respond.
des [Wed, 2 May 2007 12:20:43 +0000 (12:20 +0000)]
Try fixing #95 again. The trick is that if we get a new SLT_VCL_call
while the F_INVCL flag is set, we need to insert a newline before the
entry for the new VCL_call.
des [Tue, 1 May 2007 18:21:53 +0000 (18:21 +0000)]
Add and document a ping_interval parameter which controls the interval at
which the parent pings the child. Also document pipe_timeout, which was
left out of the man page by accident.
des [Tue, 1 May 2007 17:55:31 +0000 (17:55 +0000)]
Two minor logging fixes:
- change the type of vsl_handler()'s tag argument from unsigned int to
enum shmlogtag to allow gcc to check switch statements and gdb to show
its value by name rather than by number.
- fix the "missing newline after VCL_call" bug in varnishlog (#95)
des [Tue, 24 Apr 2007 09:39:12 +0000 (09:39 +0000)]
Correctly detect the presence and location of all external library we use
(except for the C math library, which the C standard guarantees is always
available as -lm) and more importantly, use them only where needed.
des [Sat, 21 Apr 2007 21:48:56 +0000 (21:48 +0000)]
Major rewrite of the VSL handler for increased robustness and clarity:
- Treat all request fields in a similar manner.
- Mostly eliminate fixed-size buffers.
- Don't print or format anything until we see ReqEnd.
- If we saw a Host: header, use it to generate an absolute URI,
resulting in far more useful output when processing logs from a
server which handles multiple virtual hosts.
When we have some amount of a chunk header, but not all of it, we
need to read more from the fd. The semantics we _really_ want for
that read operation is "wait until at least one char is available,
then return as many as N to us".
This can be done with a combination of system calls, but it is likely
just as cheap to just read one char at a time, so we do that.
Move the function that pushes the compiled VCL programs C source through
the systems cc(1) from the VCL compiler library to the varnishd process.
This reduces the VCL-compiler library to a text-procesing functionality
and makes it easier to build other tools, including test-suites,
around the VCL-compiler.
It also moves the actual compiler invocation string into the varnishd
sources, where it can be handled appropriately, possibly as a paramter.
Improve error handling for unterminated /* ... */ comments.
Add undocumented and unsupported facility for inline C source code
in VCL programs. The syntax is "C{ getpid(); }C" and you are on
your own if you use this.
Split the parsing of actions into a separate file, this is the bit
I expect to grow the most in the near future and all actions have
a lot more in common, than they have with conditionals etc.
phk [Fri, 30 Mar 2007 21:11:15 +0000 (21:11 +0000)]
Overhaul compiler to get rid of memory leaks and other bogons.
Add a memlist to the tokenlist and a function which allocates
with malloc(3) and hangs the piece on the memlist of tokenlist.
At the end of compilation, we ditch everything on the list.
Handle vrt_obj.h like the other #includes, and stuff these
into a vsb instead of directly to a file.
Free decoded token string, if any.
Pull creation and destruction of tokenlist into separate functions
for code clarity. Remember to destry everything in the tokenlist.
Pull invocation of cc(1) into a separate function and change the
way we do it, so we get any cc(1) groans and whines back in the
vsb so a CLI user will see them. More errorchecks than before.
des [Thu, 29 Mar 2007 10:49:58 +0000 (10:49 +0000)]
The argv length calculation was not only off by one, but failed to take
into account the extra space required by expanded quotes, backslashes and
newlines. Instead of pre-allocating a (possibly too short) buffer, start
with a 64-byte buffer and double it every time we come close to filling
it up. Also, avoid appending a trailing space before the final newline.
This issue was uncovered by Kristoffer Gleditsch <kristoffer@linpro.no>,
who also helped test this patch.
Technically this results in a change to how we account for source
code references in the counter/profile table as well, and as a result
the entire source code of the VCL program is now compiled into the
shared library for easy reference.
phk [Thu, 8 Mar 2007 10:09:18 +0000 (10:09 +0000)]
Make Varnish able to accept connections from multiple sockets by
specifying a whitespace separated list of addresses to -a (or
param.set listen_address).
I'm not sure about the error handling, for instance, what is the
desirable behaviour if one of multiple sockets fail to open ?
phk [Wed, 7 Mar 2007 10:38:20 +0000 (10:38 +0000)]
Add a bit of garbage collection to yesterdays change: Passed objects need
to have their storage properly reclaimed, including the actual content
of a obj.pass=1 cache entry, once we have sent the content to the original
requestor.
phk [Tue, 6 Mar 2007 22:40:06 +0000 (22:40 +0000)]
Having thought long and hard about this, commit what I think is the
new and simpler flow for version2.
Pass is now handled like a miss where the object will not be cached.
The main result of this is that we drag the entire object, header
and body, from the backend before transmitting it to the client,
thus isolating the backend from slow clients.
From a software engineering point of view it is a big improvement,
because it eliminates the need for all of cache_pass.c and we therefore
end up with less HTTP protocol implementations.
A side effect of this is that ticket #56 should be fixed now.
If the object is pass'ed before vcl_fetch{} that is, in vcl_recv{},
vcl_hit{} or vcl_miss{}, no "pass this" object is inserted in the
cache. The confusion between "pass", "insert" and "insert_pass"
has been cleaned up, by the removal of the latter.
Pipe and Pass calls vcl_pipe{} and vcl_pass{} respectively, before
contacting the backend. I havn't quite decided if they should
operate on the request header from the client or the one to the
backend, or both.
One possible use is to inject a "Connection: close" header to limit
pipe to one transaction.
A new vcl_hash{} has been added, it will allow customization of
which fields we hash on, instead of the default "url + Host:" but
this is not yet implemented.
vcl_fetch{} is now called after both the headers and body have been
picked up from the backend. This will allow us to do more comprehensive
handling of backend errors later on.
A disadvantage to this is that if the object ends up as a "pass
this" object in the cache, we could possibly have released any
queued requests already after the headers were received. If this
is transpires as a real-world problem, we can add a vcl_fetchhdr{}
which can do an early release (ie: "pass").
phk [Mon, 29 Jan 2007 22:06:33 +0000 (22:06 +0000)]
Things you didn't know about C, #7212:
There is no sane way to get sscanf to tell you how many characters
were consumed, if you want to allow a variable number of arguments.
The special format %n is patently useless for this, because you
have to insert it at every conceiveable point in the string and
that presumes full explicit whitespace markup.
Parse -w argument "by hand", to catch illegal input like "1,INF,15"
Tripped over by: Stein Ove Rosseland <steinove@vg.no>
phk [Mon, 22 Jan 2007 12:31:52 +0000 (12:31 +0000)]
The getaddrinfo(3) API does not tell us the TTL value learned from DNS
so we have to add our own stuff for that.
Without some kind of TTL, we would hit the DNS server once per failed
attempt to connect to the backend.
If the backend were down, we could hit it a LOT.
In the VCL code:
backend foobar {
[...]
set backend.dnsttl = 20s;
}
will assign a TTL for DNS lookups of this backends hostname+port
combination, we will not hit the DNS server more often that this.
The default is set at 30 seconds, short enough to make things are
workable in a load-balancing-via-DNS setups, yet long enough to not
pound the DNS server flat in case of backend failures.
NOTE that as long as we succeed in connecting to the backend we
do not perform new DNS lookups. That will have to be revisited
along with possible load-balancing schemes for the backend(s).
des [Thu, 2 Nov 2006 12:57:11 +0000 (12:57 +0000)]
As per ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change,
remove the so-called advertising clause from these files' license,
making it compatible with the Debian Free Software Guidelines. As
these files were obtained from FreeBSD, this change was discussed
with and approved by the FreeBSD core team.
projects / varnish / log