Merge remote-tracking branch 'origin/master' into version-checks

Conflicts:
dak/update_db.py

Merge remote branch 'origin/master'

More fixes for stupid suitesuffix stuff

Signed-off-by: Mark Hymers <mhy@debian.org>

More fixes for suite_suffix

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Fix up config file

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'maintainer'

Fix up output path for security

Signed-off-by: Mark Hymers <mhy@debian.org>

Extend debugging info for DBSource.

Signed-off-by: Torsten Werner <twerner@debian.org>

Replace SrcUploaders class by SQLA relationship.

Signed-off-by: Torsten Werner <twerner@debian.org>

Remove more debug code (sigh)

Signed-off-by: Mark Hymers <mhy@debian.org>

And don't strip the leading / anymore

Signed-off-by: Mark Hymers <mhy@debian.org>

Whoops - remove debug code

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'mhy/g-r' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

New g-r command

Signed-off-by: Mark Hymers <mhy@debian.org>

update-db: Increase required schema version to 52

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

control-suite: Correct typo and error message

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

update52: Add version_check table.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

admin: Add version-check subcommands.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

control-suite: Validate version_checks

Also test version_checks when using control-suite instead of only for
uploaded packages.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Move version_checks to the database

No longer use Suite::*::VersionCheck::* from the configuration file.
Instead move these settings to the version_checks table.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

bugfix

Signed-off-by: Torsten Werner <twerner@debian.org>

Don't assume arch 'all' has id 2 in daklib/lists.py.

Signed-off-by: Torsten Werner <twerner@debian.org>

Fix SIGPIPE errors in source contents scan.

Signed-off-by: Torsten Werner <twerner@debian.org>

Fix and improve cron.unchecked.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' into contents

Fix table src_contents.

- Add missing index.
- Add missing columns 'created' and 'modified'.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote branch 'ganneff/gr2' into g-r

incomplete, non-working, just to give over

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' into contents

Improve source contents logging a bit.

Signed-off-by: Torsten Werner <twerner@debian.org>

Start writing source contents.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add class SourceContentsWriter.

Signed-off-by: Torsten Werner <twerner@debian.org>

Fix reject message generation

Signed-off-by: Mark Hymers <mhy@debian.org>

deal with NULL source_acls

Signed-off-by: Mark Hymers <mhy@debian.org>

filter on active keyrings only

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Allow us to mark keyrings as no longer in use

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'mhy/master' into merge

Conflicts:
daklib/utils.py

Signed-off-by: Mark Hymers <mhy@debian.org>

Load keyring names from the database instead of the config file

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote-tracking branch 'ansgar/package-set' into merge

* ansgar/package-set:
  process-new: allow overrides to work for Package-Set
  Make use of Package-Set in determine_new
  queue.Upload: separate code to load .dsc from check_dsc
  utils.py: Add parser for Package-Set field in .dsc
  process-upload: Dinstall::LintianTags may not be set

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Refactor class ContentsWriter to prepare for source contents.

Signed-off-by: Torsten Werner <twerner@debian.org>

process-new: allow overrides to work for Package-Set

New packages from Package-Set do not always have files assocciated with
them. In that case edited overrides would be lost once we call
override_new again. We pass the old new dictionary so we can preserve
this information.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge branch 'contents' into pkgsrc

Merge branch 'pkgsrc' of ftp-master.debian.org:public_html/dak into pkgsrc

Make use of Package-Set in determine_new

Make use of the Package-Set field of an included .dsc file when looking
for NEW packages.  See http://bugs.debian.org/619131 for more
information.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge branch 'master' into contents

Conflicts:
daklib/dbconn.py

queue.Upload: separate code to load .dsc from check_dsc

Move the code to load the .dsc file into the self.pkg.dsc dictionary
into a separate function so it can be called without the side effects in
check_dsc.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

utils.py: Add parser for Package-Set field in .dsc

See http://bugs.debian.org/619131 for more information.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

process-upload: Dinstall::LintianTags may not be set

Handle the case when Dinstall::LintianTags is not set correctly.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Refactor functions getSource() and getBinaries().

Both functions are moved from dak/generate_filelist.py to
daklib/lists.py. The generator now yields a tuple with the id as the
first element.

Signed-off-by: Torsten Werner <twerner@debian.org>

mark doesnt like ; even though he gave me one

Signed-off-by: Joerg Jaspert <joerg@debian.org>

a shorter version for attribs/subattribs, thanks mark

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

ORMObject: make validation more robust.

Signed-off-by: Torsten Werner <twerner@debian.org>

start a new generate-releases

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'merge'

* merge:
  Fix dak update-db

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Proper fix for validate problem
  validate is somehow causing a double flush; comment for now

Merge remote-tracking branch 'ansgar/fix-update-db' into merge

* ansgar/fix-update-db:
  Fix dak update-db

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Proper fix for validate problem

Signed-off-by: Mark Hymers <mhy@debian.org>

validate is somehow causing a double flush; comment for now

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Fix up permissions
  metadata generation work
  Use UnpackedSource in queue.py.
  Improve exception handling in make-changelog.
  Work with service mode
  Use UnpackedSource in make-changelog.
  Add new class UnpackedSource and a new test.
  Start work on Packages import

Optimize file rename in dak generate.

Signed-off-by: Torsten Werner <twerner@debian.org>

Handle exceptions in dak contents scan-source.

Signed-off-by: Torsten Werner <twerner@debian.org>

Rename contents subcommands to be consistent with metadata.

Signed-off-by: Torsten Werner <twerner@debian.org>

bugfixes

Signed-off-by: Torsten Werner <twerner@debian.org>

Fix up permission

Signed-off-by: Mark Hymers <mhy@debian.org>

Fix up permissions

Signed-off-by: Mark Hymers <mhy@debian.org>

Add subcommand source-scan to 'dak contents'.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add class SourceContentsScanner.

Signed-off-by: Torsten Werner <twerner@debian.org>

move to service usage

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Fix dak update-db

Merge remote branch 'mhy/psimport' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

metadata generation work

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'ftpmaster/master' into psimport

Conflicts:
daklib/dbconn.py

Signed-off-by: Mark Hymers <mhy@debian.org>

Rename ContentsScanner to BinaryContentsScanner.

Signed-off-by: Torsten Werner <twerner@debian.org>

Use UnpackedSource in queue.py.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'contents'

Improve exception handling in make-changelog.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote branch 'mhy/master' into merge

Signed-off-by: Mark Hymers <mhy@debian.org>

Work with service mode

Signed-off-by: Mark Hymers <mhy@debian.org>

allow null versions

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Use UnpackedSource in make-changelog.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add new class UnpackedSource and a new test.

Signed-off-by: Torsten Werner <twerner@debian.org>

use  service name

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'pkgsrc'

Metadata: add cascading to mapper configuration.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Configure and test more mapper relationships.

no longer pgdatabase

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Configure and test more mapper relationships.

- DBBinary and DBSource have relationships to BinaryMetadata and SourceMetadata.
- Both have an associationproxy to hide the details.

Signed-off-by: Torsten Werner <twerner@debian.org>

Start work on Packages import

Signed-off-by: Mark Hymers <mhy@debian.org>

Add Metadata classes + tests.

+ some cleanup of dbconn.py

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'contents' of ftp-master.debian.org:public_html/dak into contents

Replace DBConn.reset() by something more robust.

Signed-off-by: Torsten Werner <twerner@debian.org>

Fix database tests.

Signed-off-by: Torsten Werner <twerner@debian.org>

gps

move stable/oldstable into gps too, so we only generate packages/sources
from there, ever

Signed-off-by: Joerg Jaspert <joerg@debian.org>

license

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Contents: replace multithreading by multiprocessing.

That avoids the global interpreter lock of CPython.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'contents'

Add more logging to contents generate.

Signed-off-by: Torsten Werner <twerner@debian.org>