phk [Fri, 30 Mar 2007 21:11:15 +0000 (21:11 +0000)]
Overhaul compiler to get rid of memory leaks and other bogons.
Add a memlist to the tokenlist and a function which allocates
with malloc(3) and hangs the piece on the memlist of tokenlist.
At the end of compilation, we ditch everything on the list.
Handle vrt_obj.h like the other #includes, and stuff these
into a vsb instead of directly to a file.
Free decoded token string, if any.
Pull creation and destruction of tokenlist into separate functions
for code clarity. Remember to destry everything in the tokenlist.
Pull invocation of cc(1) into a separate function and change the
way we do it, so we get any cc(1) groans and whines back in the
vsb so a CLI user will see them. More errorchecks than before.
des [Thu, 29 Mar 2007 10:49:58 +0000 (10:49 +0000)]
The argv length calculation was not only off by one, but failed to take
into account the extra space required by expanded quotes, backslashes and
newlines. Instead of pre-allocating a (possibly too short) buffer, start
with a 64-byte buffer and double it every time we come close to filling
it up. Also, avoid appending a trailing space before the final newline.
This issue was uncovered by Kristoffer Gleditsch <kristoffer@linpro.no>,
who also helped test this patch.
Technically this results in a change to how we account for source
code references in the counter/profile table as well, and as a result
the entire source code of the VCL program is now compiled into the
shared library for easy reference.
phk [Thu, 8 Mar 2007 10:09:18 +0000 (10:09 +0000)]
Make Varnish able to accept connections from multiple sockets by
specifying a whitespace separated list of addresses to -a (or
param.set listen_address).
I'm not sure about the error handling, for instance, what is the
desirable behaviour if one of multiple sockets fail to open ?
phk [Wed, 7 Mar 2007 10:38:20 +0000 (10:38 +0000)]
Add a bit of garbage collection to yesterdays change: Passed objects need
to have their storage properly reclaimed, including the actual content
of a obj.pass=1 cache entry, once we have sent the content to the original
requestor.
phk [Tue, 6 Mar 2007 22:40:06 +0000 (22:40 +0000)]
Having thought long and hard about this, commit what I think is the
new and simpler flow for version2.
Pass is now handled like a miss where the object will not be cached.
The main result of this is that we drag the entire object, header
and body, from the backend before transmitting it to the client,
thus isolating the backend from slow clients.
From a software engineering point of view it is a big improvement,
because it eliminates the need for all of cache_pass.c and we therefore
end up with less HTTP protocol implementations.
A side effect of this is that ticket #56 should be fixed now.
If the object is pass'ed before vcl_fetch{} that is, in vcl_recv{},
vcl_hit{} or vcl_miss{}, no "pass this" object is inserted in the
cache. The confusion between "pass", "insert" and "insert_pass"
has been cleaned up, by the removal of the latter.
Pipe and Pass calls vcl_pipe{} and vcl_pass{} respectively, before
contacting the backend. I havn't quite decided if they should
operate on the request header from the client or the one to the
backend, or both.
One possible use is to inject a "Connection: close" header to limit
pipe to one transaction.
A new vcl_hash{} has been added, it will allow customization of
which fields we hash on, instead of the default "url + Host:" but
this is not yet implemented.
vcl_fetch{} is now called after both the headers and body have been
picked up from the backend. This will allow us to do more comprehensive
handling of backend errors later on.
A disadvantage to this is that if the object ends up as a "pass
this" object in the cache, we could possibly have released any
queued requests already after the headers were received. If this
is transpires as a real-world problem, we can add a vcl_fetchhdr{}
which can do an early release (ie: "pass").
phk [Mon, 29 Jan 2007 22:06:33 +0000 (22:06 +0000)]
Things you didn't know about C, #7212:
There is no sane way to get sscanf to tell you how many characters
were consumed, if you want to allow a variable number of arguments.
The special format %n is patently useless for this, because you
have to insert it at every conceiveable point in the string and
that presumes full explicit whitespace markup.
Parse -w argument "by hand", to catch illegal input like "1,INF,15"
Tripped over by: Stein Ove Rosseland <steinove@vg.no>
phk [Mon, 22 Jan 2007 12:31:52 +0000 (12:31 +0000)]
The getaddrinfo(3) API does not tell us the TTL value learned from DNS
so we have to add our own stuff for that.
Without some kind of TTL, we would hit the DNS server once per failed
attempt to connect to the backend.
If the backend were down, we could hit it a LOT.
In the VCL code:
backend foobar {
[...]
set backend.dnsttl = 20s;
}
will assign a TTL for DNS lookups of this backends hostname+port
combination, we will not hit the DNS server more often that this.
The default is set at 30 seconds, short enough to make things are
workable in a load-balancing-via-DNS setups, yet long enough to not
pound the DNS server flat in case of backend failures.
NOTE that as long as we succeed in connecting to the backend we
do not perform new DNS lookups. That will have to be revisited
along with possible load-balancing schemes for the backend(s).
des [Thu, 2 Nov 2006 12:57:11 +0000 (12:57 +0000)]
As per ftp://ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change,
remove the so-called advertising clause from these files' license,
making it compatible with the Debian Free Software Guidelines. As
these files were obtained from FreeBSD, this change was discussed
with and approved by the FreeBSD core team.
des [Mon, 9 Oct 2006 09:58:58 +0000 (09:58 +0000)]
Fix build on MacOS X: add a fake clock_gettime() and fix some includes.
WARNING: varnish will build and run, but the lack of a monotonic clock
may lead to strange behaviour if the clock is stepped (rather than skewed)
while varnish is running.
Thanks to Niklas Saers for providing a test environment.
des [Fri, 6 Oct 2006 15:37:32 +0000 (15:37 +0000)]
On Linux 2.6.8 and older, epoll_ctl(EPOLL_CTL_DEL) requires a poll_event
even though it is not used, and will return EFAULT if one is not provided.
Also, instead of silently accepting EBADF from epoll_ctl(), check that
sp->fd != -1 before calling vca_del(). This can happen in some cases
where vca_pollsession() closes the session before returning. This way,
we will still get an assertion failure if epoll_ctl() fails for some
other (unexpected) reason.
Thanks to airmax for his assistance in tracking this down.
des [Fri, 6 Oct 2006 12:58:04 +0000 (12:58 +0000)]
The delivered document's age should be computed relative to the time of
the response, not the time of the request. If the document was not in
cache and the backend took a long time to respond, the document would
end up with a negative computed age, which when printed with %u would
appear as a large positive number (a few seconds short of either 2^32
or 2^64 depending on the size of time_t).
des [Thu, 5 Oct 2006 14:56:50 +0000 (14:56 +0000)]
Older versions of automake generate incorrect Makefiles, and automake's
own mechanism for specifying a required version doesn't seem to work.
Hack autogen.sh to check which version of automake is installed.
des [Thu, 5 Oct 2006 11:57:35 +0000 (11:57 +0000)]
RFC 2616 says "All 1xx (informational), 204 (no content), and 304 (not
modified) responses MUST NOT include a message-body," so
Content-Length: is not needed in these cases (and Apache does indeed
not include it). This causes PassBody() to call pass_straight() with
a NULL length argument, which waits until the connection is closed by
the server. PassBody() should not call pass_*() at all for responses
that are known to be bodyless.
Submitted by: Dagfinn Ilmari Mannsåker <ilmari@ping.uio.no>
des [Tue, 26 Sep 2006 13:10:36 +0000 (13:10 +0000)]
Ignore accept() failures if errno is EAGAIN - this is not likely to happen
on a production server, but it will on a test rig, and it may confuse and
alarm the admin.
projects / varnish / log