Fix dak update-db

Use UnpackedSource in queue.py.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'contents'

Improve exception handling in make-changelog.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Work with service mode

Signed-off-by: Mark Hymers <mhy@debian.org>

allow null versions

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Use UnpackedSource in make-changelog.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add new class UnpackedSource and a new test.

Signed-off-by: Torsten Werner <twerner@debian.org>

use  service name

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'pkgsrc'

Metadata: add cascading to mapper configuration.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Configure and test more mapper relationships.

no longer pgdatabase

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Configure and test more mapper relationships.

- DBBinary and DBSource have relationships to BinaryMetadata and SourceMetadata.
- Both have an associationproxy to hide the details.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add Metadata classes + tests.

+ some cleanup of dbconn.py

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'contents' of ftp-master.debian.org:public_html/dak into contents

Replace DBConn.reset() by something more robust.

Signed-off-by: Torsten Werner <twerner@debian.org>

Fix database tests.

Signed-off-by: Torsten Werner <twerner@debian.org>

gps

move stable/oldstable into gps too, so we only generate packages/sources
from there, ever

Signed-off-by: Joerg Jaspert <joerg@debian.org>

license

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Contents: replace multithreading by multiprocessing.

That avoids the global interpreter lock of CPython.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'contents'

Add more logging to contents generate.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Add Built-Using support

Signed-off-by: Mark Hymers <mhy@debian.org>

Contents: ignore errors when removing destination file.

Signed-off-by: Torsten Werner <twerner@debian.org>

Do no longer re-exec dak contents.

We just call DBConn().reset() in the child process now.

Signed-off-by: Torsten Werner <twerner@debian.org>

ries sync

no longer push or name anything merkel
sync ries differently depending on if its called with sync or pool
that way cron.hourly does a small sync while cron.dinstall a full one triggers

Signed-off-by: Joerg Jaspert <joerg@debian.org>

shutup

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Contents: use the new string_agg() function of Pg9.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Fix typo
  Improve db conf handling
  Commit uncommitted fix
  Patch in support for service= DSN syntax

add the syncscript for ries. also call it hourly

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Don't remove sources which are referenced by other binaries

Signed-off-by: Mark Hymers <mhy@debian.org>

Fix typo

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Improve db conf handling

Signed-off-by: Mark Hymers <mhy@debian.org>

Commit uncommitted fix

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Patch in support for service= DSN syntax

Signed-off-by: Mark Hymers <mhy@debian.org>

add ries

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Use PK at twerners suggestion

Signed-off-by: Mark Hymers <mhy@debian.org>

set db schema version

Signed-off-by: Mark Hymers <mhy@debian.org>

Add method reset() to class DBConn() and test it.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote branch 'ftpmaster/master'

First pass at P/S database schema

Signed-off-by: Mark Hymers <mhy@debian.org>

First pass at extra_src_references table

Signed-off-by: Mark Hymers <mhy@debian.org>

Add table src_contents.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Add logging to update-db including transaction id.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Really update to DB version #44.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' into dbtests

Conflicts:
dak/update_db.py

Signed-off-by: Torsten Werner <twerner@debian.org>

Rename DB upgrade #43 to #44.

bad, bad mhy...

Signed-off-by: Torsten Werner <twerner@debian.org>

make check_overrides not delete stuff thats not at least 14 days old

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Don't contradict ourselves within 2 lines

Signed-off-by: Mark Hymers <mhy@debian.org>

add mvo quote

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Prepare for pg 9.0

gps

change the way we run the pool once more, hopefully being less bad now. also, this tool is EOL somehwat, so who cares how bad it looks

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Prepare for pg 9.0

Signed-off-by: Mark Hymers <mhy@debian.org>

gps timeout/error handling

return the result of the a-f call.
additionally get it using a timeout.
when either a TimeoutError occurs or the result is non-zero, complain about it
and then exit non-zero, so dinstall could stop.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

mhy, das kruemelmonster

Signed-off-by: Joerg Jaspert <joerg@debian.org>

database backup foo

no longer dump the full database on each dinstall run. instead we just
save a transaction id. DSA is doing the backup dumps in whatever cycle,
and keeping the WAL stuff around, so we can tell them to get us to
whatever point in our db which we might need, ie. the specific transaction
id.

as those are just 9 bytes per file, and not 4gigabytes, we now also save
wth we have been before and after unchecked runs, which makes our backup
be nearer to actual filesystem state than ever.

Signed-off-by: Joerg Jaspert <joerg@debian.org>
Signed-off-by: Joerg Jaspert <joerg@debian.org>

remove pgdakdev

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Remove old contents code and tables.

Signed-off-by: Torsten Werner <twerner@debian.org>

parse scriptvars, not . vars

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Fix a comment in dbconn.py.

Signed-off-by: Torsten Werner <twerner@debian.org>

Fix copy-paste error.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Ignore more warnings from python-apt.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak:
  Add filename de-duplication to contents scanning.

updates

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'dbtests'

Add filename de-duplication to contents scanning.

Signed-off-by: Torsten Werner <twerner@debian.org>

byhand-di: allow YYYYMMDD+<suite>x version numbers

To: debian-dak@lists.debian.org
Cc: Julien Cristau <jcristau@debian.org>, Otavio Salvador <otavio@ossystems.com.br>
Date: Mon, 14 Mar 2011 14:11:39 +0100
X-Spam-Status: No, score=-1.775 tagged_above=-99 required=4.6 tests=[BAYES_00=-1.9, SARE_SUB_OBFU_OTHER=0.135, T_RP_MATCHES_RCVD=-0.01] autolearn=no
X-Mailer: git-send-email 1.7.2.5
Resent-From: debian-dak@lists.debian.org

stable updates need to have a higher version than binNMUs, but
YYYYMMDD<suite>x << YYYYMMDD+b1.

Signed-off-by: Julien Cristau <jcristau@debian.org>
Cc: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Joerg Jaspert <joerg@debian.org>

Break hardlink when writing Contents files.

Signed-off-by: Torsten Werner <twerner@debian.org>

we do want -n1, not -n16. we want as many parallel xz as we can

Signed-off-by: Joerg Jaspert <joerg@debian.org>

dinstall - mirror

exclude the temporary files from apt-ftparchive (*.new), and delete excluded
files too.
Should keep the mirrors happy, cos those files arent readable for others.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Handle empty packages in 'dak contents scan'.

Signed-off-by: Torsten Werner <twerner@debian.org>

allow squeeze-updates to be reset

Signed-off-by: Mark Hymers <mhy@debian.org>

Switch to 'dak contents generate'.

It replaces APT::FTPArchive::Contents.

Signed-off-by: Torsten Werner <twerner@debian.org>

Enable the new contents generation for experimental.

... as a first test

Signed-off-by: Torsten Werner <twerner@debian.org>

Write full section to Contents files.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' into dbtests

Add workaround for duplicate filenames in contents.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak:
  bugfixes
  Contents: support override suites.

dinstall - use xz for pg dumps, and use a | xargs instead of while

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

bugfixes

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://franck.debian.org/srv/ftp.debian.org/git/dak:
  Use 'dak contents generate_helper'.
  bugfixes
  Add a new subcommand 'generate_helper' to 'dak contents'.
  Switch to multiprocessing module and reap child gzips.

dinstall - dont recompress .xz files

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Contents: support override suites.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' into dbtests

Use 'dak contents generate_helper'.

Signed-off-by: Torsten Werner <twerner@debian.org>

bugfixes

Signed-off-by: Torsten Werner <twerner@debian.org>

Add a new subcommand 'generate_helper' to 'dak contents'.

Signed-off-by: Torsten Werner <twerner@debian.org>

Switch to multiprocessing module and reap child gzips.

Signed-off-by: Torsten Werner <twerner@debian.org>

And make the whitespace something useful

Signed-off-by: Joerg Jaspert <joerg@debian.org>