Make cron.daily shell script more robust.

... and reindent it.

Signed-off-by: Torsten Werner <twerner@debian.org>

Excluding more directories from syncing.

Signed-off-by: Torsten Werner <twerner@debian.org>

sick

Signed-off-by: Joerg Jaspert <joerg@debian.org>

function pg_timestamp

Signed-off-by: Joerg Jaspert <joerg@debian.org>

updates

Signed-off-by: Joerg Jaspert <joerg@debian.org>

off you are

Signed-off-by: Joerg Jaspert <joerg@debian.org>

moved ddaccess push into locks

Signed-off-by: Joerg Jaspert <joerg@debian.org>

no need for extra step

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Close log so we log program end

Signed-off-by: Mark Hymers <mhy@debian.org>

Fix up function call

Signed-off-by: Mark Hymers <mhy@debian.org>

Tidy up logging

Signed-off-by: Mark Hymers <mhy@debian.org>

And the right arguments

Signed-off-by: Mark Hymers <mhy@debian.org>

Actually call the functions...

Signed-off-by: Mark Hymers <mhy@debian.org>

Our log function expects a single list argument

Signed-off-by: Mark Hymers <mhy@debian.org>

Run linearly until we can debug why the multiprocessing stuff hangs

Signed-off-by: Mark Hymers <mhy@debian.org>

Comment out results gathering to test for hang failure

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

close logger

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote-tracking branch 'ansgar/p-s-from-db' into merge

* ansgar/p-s-from-db:
  Add external-overrides to byhand scripts.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add external-overrides to byhand scripts.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

and one = is enough for the scripts

Signed-off-by: Joerg Jaspert <joerg@debian.org>

holy fuck goddamn lunatics

whoever had the fuck broken stupid idea of making dash the default
/bin/sh should be shot. fucking stupid damn lunatics, go die you
suckers. or use windows. whatever. LEAVE THE WORLD SANE. stupid bitches.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

no deferred queue

Signed-off-by: Joerg Jaspert <joerg@debian.org>

shutup

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge remote-tracking branch 'ansgar/p-s-from-db' into merge

* ansgar/p-s-from-db:
  generate-packages-sources2: respect overridesuite
  process-upload: Add counter for rejected packages

Signed-off-by: Joerg Jaspert <joerg@debian.org>

generate-packages-sources2: respect overridesuite

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

process-upload: Add counter for rejected packages

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

also call dak gps2

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add self. to rejects

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

Mark dsc as invalid if we find any reject reasons

Signed-off-by: Mark Hymers <mhy@debian.org>

Use the newdists dir as temporary output dir for g-p-s2.

It should be changed back after more testing.

Signed-off-by: Torsten Werner <twerner@debian.org>

Be more robust to missing suites

Signed-off-by: Mark Hymers <mhy@debian.org>

Map -proposed-updates properly

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote-tracking branch 'ansgar/p-s-from-db' into merge

* ansgar/p-s-from-db:
  daklib/changes.py: add logger parameter
  generate-packages-sources2: Add external overrides
  Add external-overrides.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

daklib/changes.py: add logger parameter

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

generate-packages-sources2: Add external overrides

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Add external-overrides.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Get function name right...

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

make sure we've applied distribution maps before trying to use the distribution

Signed-off-by: Mark Hymers <mhy@debian.org>

add a bit more verbose logging

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge remote-tracking branch 'ansgar/p-s-from-db' into merge

* ansgar/p-s-from-db:
  show-new: Use dakmultiprocessing
  dakmultiprocessing: close all session
  Add missing directories in filewriter.py.
  dont have GOODSIG in the output
  mention who submitted the key
  wtf?
  i hate gpg
  change to using the debian buildd host list
  Fix unicode issue in show-deferred.
  generate-packages-sources2: Use new filewriter
  dakmultiprocessing: Add a helpful comment
  generate-packages-sources2: Use dakmultiprocessing
  generate-filelist: Use dakmultiprocessing
  Add a simple multiprocessing wrapper to catch exceptions

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge remote branch 'origin/master' into p-s-from-db

show-new: Use dakmultiprocessing

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

dakmultiprocessing: close all session

Make sure to close all sessions so we do not run out of connections.
Before this exceptions in the workers might leave open transactions
around.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Add missing directories in filewriter.py.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of http://ftp-master.debian.org/git/dak

dont have GOODSIG in the output

Signed-off-by: Joerg Jaspert <joerg@debian.org>

mention who submitted the key

Signed-off-by: Joerg Jaspert <joerg@debian.org>

wtf?

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  change to using the debian buildd host list
  Fix unicode issue in show-deferred.

i hate gpg

Signed-off-by: Joerg Jaspert <joerg@debian.org>

change to using the debian buildd host list

Signed-off-by: Mark Hymers <mhy@debian.org>

Fix unicode issue in show-deferred.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote-tracking branch 'ansgar/build-queues' into merge

* ansgar/build-queues:
  daklib/dbconn.py: f.poolfile might be None

Signed-off-by: Joerg Jaspert <joerg@debian.org>

daklib/dbconn.py: f.poolfile might be None

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge remote-tracking branch 'ansgar/build-queues' into merge

* ansgar/build-queues:
  daklib/changes.py: handle Dinstall::SuiteSuffix

Signed-off-by: Joerg Jaspert <joerg@debian.org>

daklib/changes.py: handle Dinstall::SuiteSuffix

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Fix DB tests.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'contents'

Merge branch 'contents'

bugfix

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Fix very bad logic error
  Fix Release file hardlink handling
  Fix typo
  Fix database permissions
  Avoid using an SQL keyword as a column name
  Apply update 56
  Update pickle file for DB tests.
  Add PackagesFileWriter and SourcesFileWriter.
  Implement SourceContentsFileWriter.
  various bugfixes
  Allow per-suite signing keys
  Add the *FileWriter helper classes.

add repuser as known user

Signed-off-by: Joerg Jaspert <joerg@debian.org>

generate-packages-sources2: Use new filewriter

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Split binary contents into components.

Signed-off-by: Torsten Werner <twerner@debian.org>

dakmultiprocessing: Add a helpful comment

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Merge remote branch 'ftpmaster/master'

Fix very bad logic error

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'multiprocessing' into p-s-from-db

Fix Release file hardlink handling

Signed-off-by: Mark Hymers <mhy@debian.org>

generate-packages-sources2: Use dakmultiprocessing

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Fix typo

Signed-off-by: Mark Hymers <mhy@debian.org>

Fix database permissions

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'mhy/per-suite-signing-keys'

Conflicts:
dak/update_db.py

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

Avoid using an SQL keyword as a column name

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Merge branch 'pkgsrc'

Merge branch 'master' of ftp-master.debian.org:/srv/ftp.debian.org/git/dak

Apply update 56

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge remote branch 'origin/master' into multiprocessing

Update pickle file for DB tests.

Signed-off-by: Torsten Werner <twerner@debian.org>

Remove bin_assoc_by_arch from dbconn.py.

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge remote-tracking branch 'ansgar/p-s-from-db' into merge

* ansgar/p-s-from-db:
  Add order column to metadata_keys
  generate-packages-sources2: various bugs fixed
  sort generated Packages and Sources
  generate Packages/Sources directly from database

Signed-off-by: Joerg Jaspert <joerg@debian.org>

generate-filelist: Use dakmultiprocessing

Use dakmultiprocessing and no longer die in worker processes.

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Drop an unused view.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add a simple multiprocessing wrapper to catch exceptions

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

Add PackagesFileWriter and SourcesFileWriter.

Signed-off-by: Torsten Werner <twerner@debian.org>

Implement SourceContentsFileWriter.

Signed-off-by: Torsten Werner <twerner@debian.org>

run i-u-f-p inside c.h and not inside crontab

Signed-off-by: Joerg Jaspert <joerg@debian.org>

enable the buildd keyring stuff

Signed-off-by: Joerg Jaspert <joerg@debian.org>

various bugfixes

Signed-off-by: Torsten Werner <twerner@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  Don't sync apt-ftparchive caches

only run when there is something to do. also only update the keyrings for dsa if there was a change

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Allow per-suite signing keys

Signed-off-by: Mark Hymers <mhy@debian.org>

Add the *FileWriter helper classes.

Signed-off-by: Torsten Werner <twerner@debian.org>

Add order column to metadata_keys

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>

generate-packages-sources2: various bugs fixed

Signed-off-by: Ansgar Burchardt <ansgar@debian.org>