generate-packages-sources2: Use new filewriter

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

dakmultiprocessing: Add a helpful comment

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge branch 'multiprocessing' into p-s-from-db

generate-packages-sources2: Use dakmultiprocessing

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Fix typo

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'mhy/per-suite-signing-keys'

Conflicts:
dak/update_db.py

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

Avoid using an SQL keyword as a column name

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Merge branch 'pkgsrc'

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Apply update 56

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'origin/master' into multiprocessing

Update pickle file for DB tests.

Signed-off-by: Torsten Werner <twerner@debian.org>

Remove bin_assoc_by_arch from dbconn.py.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote-tracking branch 'ansgar/p-s-from-db' into merge

* ansgar/p-s-from-db:
  Add order column to metadata_keys
  generate-packages-sources2: various bugs fixed
  sort generated Packages and Sources
  generate Packages/Sources directly from database

Signed-off-by: Joerg Jaspert <joerg@debian.org>

generate-filelist: Use dakmultiprocessing

Use dakmultiprocessing and no longer die in worker processes.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Drop an unused view.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add a simple multiprocessing wrapper to catch exceptions

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Add PackagesFileWriter and SourcesFileWriter.

Signed-off-by: Torsten Werner <twerner@debian.org>

Implement SourceContentsFileWriter.

Signed-off-by: Torsten Werner <twerner@debian.org>

run i-u-f-p inside c.h and not inside crontab

Signed-off-by: Joerg Jaspert <joerg@debian.org>

enable the buildd keyring stuff

Signed-off-by: Joerg Jaspert <joerg@debian.org>

various bugfixes

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Don't sync apt-ftparchive caches

only run when there is something to do. also only update the keyrings for dsa if there was a change

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Allow per-suite signing keys

Signed-off-by: Mark Hymers <mhy@debian.org>

Add the *FileWriter helper classes.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add order column to metadata_keys

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

generate-packages-sources2: various bugs fixed

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Don't sync apt-ftparchive caches

Signed-off-by: Mark Hymers <mhy@debian.org>

untabify

Signed-off-by: Joerg Jaspert <joerg@debian.org>

sort generated Packages and Sources

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

generate Packages/Sources directly from database

We can now write Packages/Sources outselves without using
apt-ftparchive.  Before this is usable we still need to compress
the output.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Do not sync lock/stages/ to ries.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Add commands to list keyrings

ddtp-i18n.sh

update the script with what Felipe sent me. This now generates the Index
file needed by the generate-release script.
advantage having it in here: it is only generated when needed and it
doesn't riddle gen-rel with entirely unrelated code

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add commands to list keyrings

Signed-off-by: Mark Hymers <mhy@debian.org>

add the new keyrings to the queueds

Signed-off-by: Joerg Jaspert <joerg@debian.org>

yuck, that was one debug line too much ending up in commit

Signed-off-by: Joerg Jaspert <joerg@debian.org>

its true damnit

Signed-off-by: Joerg Jaspert <joerg@debian.org>

less :

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Walk the right path when suite_suffix is in use

The weasel wants another path name

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

Walk the right path when suite_suffix is in use

Signed-off-by: Mark Hymers <mhy@debian.org>

not needing 027

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Fix generate_release to include the Architecture: line in subrelease files.

ignore gpg errors

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

use a special signing key instead

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge remote branch 'origin/master'

Fix generate_release to include the Architecture: line in subrelease files.

Also deal with the possiblity that files may have been hardlinked by
writing to a new file and renaming.

Signed-off-by: Mark Hymers <mhy@debian.org>

and use a gnupghome

Signed-off-by: Joerg Jaspert <joerg@debian.org>

G

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Stupid warning messages!

Signed-off-by: Torsten Werner <twerner@debian.org>

bugfix

Signed-off-by: Torsten Werner <twerner@debian.org>

Consolidate warning configuration even more.

Signed-off-by: Torsten Werner <twerner@debian.org>

Consolidate warnings to support dak rm, too.

Signed-off-by: Torsten Werner <twerner@debian.org>

Ignore yet another warning in cruft-report.

Signed-off-by: Torsten Werner <twerner@debian.org>

Let's hope nobody reads THAT commit!

It's too embarrassing.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Ignore more python-apt warnings.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'merge'

* merge:
  generate-filelist: Use multiprocessing

Merge remote-tracking branch 'ansgar/generate-filelist-multiprocessing' into merge

* ansgar/generate-filelist-multiprocessing:
  generate-filelist: Use multiprocessing

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'dbtests'

just one admin keyring

Signed-off-by: Joerg Jaspert <joerg@debian.org>

generate-filelist: Use multiprocessing

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Provide a way to set Pg's work_mem from dak.

Signed-off-by: Torsten Werner <twerner@debian.org>

Somehow it helps to actually use the admin keyrings

Signed-off-by: Joerg Jaspert <joerg@debian.org>

fixes

Signed-off-by: Joerg Jaspert <joerg@debian.org>

autosigning foo

add 3 scripts to handle the keystuff

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge remote-tracking branch 'ansgar/dak-admin-show-config' into merge

* ansgar/dak-admin-show-config:
  admin: Show options set in configuration database

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge remote-tracking branch 'ansgar/build-queues' into merge

* ansgar/build-queues:
  Add files from .dsc, but not in .changes to changes_pool_files
  Handle build queues for all policy queues
  Add send_to_build_queues to policy_queue table
  build queues: add support for files from policy queues
  Add table build_queue_policy_files.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' into merge

* master: (50 commits)
  Make getArchAll() multithread safe.
  Try building binary-all for buildd
  bugfix
  Write an additional arch all (only) file in generate-filelist.
  Ignore missing source file in process-new.
  Create temp directory at the preferred location.
  0 pad month and day
  Update DB pickle file for new table.
  Put changes files straight into the hashed queue/done dir
  Don't die on a single error
  Fix typo
  Update dinstall for new makemaintainers.
  Move all files in split_done; we're going to start writing them straight out that way anyways
  Add logging to make-maintainers.
  Add missing stuff to make-maintainers.
  Rewrite make-maintainers from scratch.
  More fixes for stupid suitesuffix stuff
  More fixes for suite_suffix
  Fix up config file
  Fix up output path for security
  ...

Merge remote-tracking branch 'origin/master' into build-queues

Merge remote-tracking branch 'ansgar/package-set' into merge

* ansgar/package-set:
  determine_new: Really make `new' an empty dict

Signed-off-by: Joerg Jaspert <joerg@debian.org>

admin: Show options set in configuration database

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Make getArchAll() multithread safe.

Signed-off-by: Torsten Werner <twerner@debian.org>

determine_new: Really make `new' an empty dict

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Add files from .dsc, but not in .changes to changes_pool_files

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

Try building binary-all for buildd

Signed-off-by: Mark Hymers <mhy@debian.org>

bugfix

Signed-off-by: Torsten Werner <twerner@debian.org>

Write an additional arch all (only) file in generate-filelist.

Signed-off-by: Torsten Werner <twerner@debian.org>

Handle build queues for all policy queues

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Add send_to_build_queues to policy_queue table

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

build queues: add support for files from policy queues

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Add table build_queue_policy_files.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Ignore missing source file in process-new.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Create temp directory at the preferred location.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

0 pad month and day

Signed-off-by: Mark Hymers <mhy@debian.org>

Update DB pickle file for new table.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Put changes files straight into the hashed queue/done dir

Signed-off-by: Mark Hymers <mhy@debian.org>

Don't die on a single error

Signed-off-by: Mark Hymers <mhy@debian.org>