Merge branch 'merge'

* merge:
  lenny's python-apt does not support "#" comments.

Merge commit 'lamby/master' into merge

* commit 'lamby/master':
  lenny's python-apt does not support "#" comments.
  no i-u-f-p
  cron

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge branch 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak

* 'master' of ssh://ftp-master.debian.org/srv/ftp.debian.org/git/dak:
  fix up tbl handling

fix up tbl handling

Signed-off-by: Mark Hymers <mhy@debian.org>

lenny's python-apt does not support "#" comments.

Signed-off-by: Chris Lamb <lamby@debian.org>

Revert "no i-u-f-p"

This reverts commit 983f7ea9cec7265c4d5c59af2c355881bdec0f20.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

no i-u-f-p

Signed-off-by: Joerg Jaspert <joerg@debian.org>

cron

modify for new world order

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master': (23 commits)
  Add tests for process_gpgv_output
  Remove some of the Table boilerplate
  Heh, someone fixed these issues since 2001.
  Add dak.conf for tests as utilities parse it.
  Port old tests to unittest; currently 18 failures.
  Add a 'fixture' utility to generate a path to a fixture.
  Tidy daklib.utils.which_conf_file
  Use a custom dak.conf during tests - now we can import daklib.utils.
  Drop executable bits on libraries.
  Prefer the "Borg" pattern over the GoF singleton pattern.
  Remove empty docstring.
  Make daklib.lintian work on the named groups, not the "flat" groups
  Use named groups in lintian regex.
  Alter generate_reject_messages to return a generator.
  Tidy upload.check_lintian.
  Generate lintian reject messages in daklib.lintian + tests
  Add a docstring for parse_lintian_output
  An empty output is handled fine.
  Move lintian parsing to daklib.lintian and add tests.
  Rename warning/error in lintian autorejects to nonfatal/fatal.
  ...

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Add tests for process_gpgv_output

Signed-off-by: Chris Lamb <lamby@debian.org>

Remove some of the Table boilerplate

Signed-off-by: Chris Lamb <lamby@debian.org>

Heh, someone fixed these issues since 2001.

Signed-off-by: Chris Lamb <lamby@debian.org>

Add dak.conf for tests as utilities parse it.

Signed-off-by: Chris Lamb <lamby@debian.org>

Port old tests to unittest; currently 18 failures.

Signed-off-by: Chris Lamb <lamby@debian.org>

Add a 'fixture' utility to generate a path to a fixture.

Signed-off-by: Chris Lamb <lamby@debian.org>

Tidy daklib.utils.which_conf_file

Signed-off-by: Chris Lamb <lamby@debian.org>

Use a custom dak.conf during tests - now we can import daklib.utils.

Signed-off-by: Chris Lamb <lamby@debian.org>

Drop executable bits on libraries.

Signed-off-by: Chris Lamb <lamby@debian.org>

Prefer the "Borg" pattern over the GoF singleton pattern.

The only visible difference is the lack of object identity between
instances.

Signed-off-by: Chris Lamb <lamby@debian.org>

Remove empty docstring.

Signed-off-by: Chris Lamb <lamby@debian.org>

Make daklib.lintian work on the named groups, not the "flat" groups

Signed-off-by: Chris Lamb <lamby@debian.org>

Use named groups in lintian regex.

Signed-off-by: Chris Lamb <lamby@debian.org>

Alter generate_reject_messages to return a generator.

Signed-off-by: Chris Lamb <lamby@debian.org>

Tidy upload.check_lintian.

Signed-off-by: Chris Lamb <lamby@debian.org>

Generate lintian reject messages in daklib.lintian + tests

Signed-off-by: Chris Lamb <lamby@debian.org>

Add a docstring for parse_lintian_output

Signed-off-by: Chris Lamb <lamby@debian.org>

An empty output is handled fine.

Signed-off-by: Chris Lamb <lamby@debian.org>

Move lintian parsing to daklib.lintian and add tests.

Signed-off-by: Chris Lamb <lamby@debian.org>

Rename warning/error in lintian autorejects to nonfatal/fatal.

Signed-off-by: Chris Lamb <lamby@debian.org>

Make all tests use DakTestCase.

Signed-off-by: Chris Lamb <lamby@debian.org>

Add a base TestCase that we can extend later.

Signed-off-by: Chris Lamb <lamby@debian.org>

Simplify which_conf_file.s

Merge commit 'mhy/master' into merge

* commit 'mhy/master': (50 commits)
  hmm...
  fix name
  right variable name
  I'm never tidying up names again
  poolfile, you moron
  in_queue is the object, _id the id
  queue_id -> policy_queue_id
  fix variable name
  argh
  When changing Queue -> Build/PolicyQueue, it helps to do a full job
  argh, another typo
  typo
  well spotted, djpig
  Fix up hand off to policy and buildd queues
  sort out permission columns
  do not support multi-archive mode in process-upload
  check_files: Fix check for known changes files
  Small fixes
  Fix typo in foreign key constraint name
  initial policy queue implementation in process_upload
  ...

Signed-off-by: Joerg Jaspert <joerg@debian.org>

hmm...

Signed-off-by: Mark Hymers <mhy@debian.org>

fix name

Signed-off-by: Mark Hymers <mhy@debian.org>

right variable name

Signed-off-by: Mark Hymers <mhy@debian.org>

I'm never tidying up names again

Signed-off-by: Mark Hymers <mhy@debian.org>

poolfile, you moron

Signed-off-by: Mark Hymers <mhy@debian.org>

in_queue is the object, _id the id

Signed-off-by: Mark Hymers <mhy@debian.org>

queue_id -> policy_queue_id

Signed-off-by: Mark Hymers <mhy@debian.org>

fix variable name

Signed-off-by: Mark Hymers <mhy@debian.org>

argh

Signed-off-by: Mark Hymers <mhy@debian.org>

When changing Queue -> Build/PolicyQueue, it helps to do a full job

Signed-off-by: Mark Hymers <mhy@debian.org>

argh, another typo

Signed-off-by: Mark Hymers <mhy@debian.org>

typo

Signed-off-by: Mark Hymers <mhy@debian.org>

well spotted, djpig

Signed-off-by: Mark Hymers <mhy@debian.org>

Fix up hand off to policy and buildd queues

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge commit 'ftpmaster/master'

Merge commit 'djpig/process-upload'

sort out permission columns

Signed-off-by: Mark Hymers <mhy@debian.org>

do not support multi-archive mode in process-upload

We don't actually have any multi-archive machines at the moment
and removing it makes test-setups easier to do.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

check_files: Fix check for known changes files

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Small fixes

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Fix typo in foreign key constraint name

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge commit 'mhy/master' into process-upload

initial policy queue implementation in process_upload

Signed-off-by: Mark Hymers <mhy@debian.org>

Move one tag

Signed-off-by: Joerg Jaspert <joerg@debian.org>

fix up existing changes file handling

Signed-off-by: Mark Hymers <mhy@debian.org>

Multi-valued changes fields are space-separated

Not comma-separated.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

rename known_changes

Signed-off-by: Mark Hymers <mhy@debian.org>

Merge commit 'ftpmaster/master'

Conflicts:
daklib/dbconn.py

Signed-off-by: Mark Hymers <mhy@debian.org>

Try to sanitise changes and queue handling

Signed-off-by: Mark Hymers <mhy@debian.org>

Stop talking about old and even older and yet older versions / vcs information

Signed-off-by: Joerg Jaspert <joerg@debian.org>

remove some very old files

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Merge commit 'djpig/djpig' into merge

* commit 'djpig/djpig':
  clean_queues: Log which queue we remove from

Signed-off-by: Joerg Jaspert <joerg@debian.org>

clean_queues: Log which queue we remove from

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge commit 'djpig/djpig' into merge

* commit 'djpig/djpig':
  Fix error in SQLA mapper setup

Signed-off-by: Joerg Jaspert <joerg@debian.org>

Fix error in SQLA mapper setup

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Fix queue-report

Since we have no .dak files anymore, parse the changes file directly.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

remove manpages, not much sense to have them, noone uses them

Signed-off-by: Joerg Jaspert <joerg@debian.org>

remove a manpage. closes: 540170

remove a manpage

Signed-off-by: Joerg Jaspert <joerg@debian.org>

rm

another rm.
closes: #540170

Signed-off-by: Joerg Jaspert <joerg@debian.org>

readme

remove a pretty old documentation

Signed-off-by: Joerg Jaspert <joerg@debian.org>

update-db: General code cleanup

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge commit 'djpig/process-upload'

Also remove p-a and p-unchecked to avoid confusion

Conflicts:
daklib/dbconn.py
daklib/queue.py

Signed-off-by: Mark Hymers <mhy@debian.org>

check_files: check changes against known_changes

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Various fixes

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

changes: fix known_changes handling of parsed changes files

Some fields in the changes file are stored as dicts, so handle
them correctly here where we only want the string.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

changes: Remove .dak file handling

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

don't delete files in queue_files

Signed-off-by: Mark Hymers <mhy@debian.org>

check_dsc_against_db: fix a corner-case in .orig. handling

If the .orig.tar was contained in the .changes file but already
present in the pool, we did delete it from the file list, but
did not record the files id in the desc file list.

In the old code this did not cause any errors since process-accepted
called check_dsc_against_db with the file list loaded from the .dak
file which was already cleansed of the .orig. So it took the other
code branch (.orig. present in the pool but not in changes) which
did record the files id.

Now we only call check_dsc_against_db once in process-upload so
we need to record the files id in this case, too.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

process-upload: try to use the same session all over the code

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Fix various errors caused by moving stuff around

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

process-upload: make sure we clean up after ourself

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

changes: Use @session_wrapper

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge commit 'mhy/master' into merge

* commit 'mhy/master':
  add changes and queue handling

Signed-off-by: Joerg Jaspert <joerg@debian.org>

add changes and queue handling

Signed-off-by: Mark Hymers <mhy@debian.org>

sig verify

dont barf on policy_url in a sig, even if the usge of that is pretty
pointless for things uploaded to the archive.

Signed-off-by: Joerg Jaspert <joerg@debian.org>

queue: Add missing SQLA import

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Fix typo in function name

check_dm_source_upload -> check_dm_upload

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge process-accepted's install into Upload.install()

queue build support is still disabled since we will modify that
significantly.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Upload: remove cruft

* Remove unused reprocess member
* Remove CopyDotDak support

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Move helper functions for database insertion to dbconn

Copy
add_poolfile
add_dsc_to_db
add_deb_to_db
from process-accepted to dbconn so that we can use them
from the new process-upload.

Signed-off-by: Frank Lichtenheld <djpig@debian.org>

Merge commit 'private/master' into process-upload

Merge commit 'stew/knownchanges' into merge

* commit 'stew/knownchanges':
  split read and write locks

Signed-off-by: Joerg Jaspert <joerg@debian.org>

split read and write locks

Signed-off-by: Mike O'Connor <stew@vireo.org>

Merge commit 'lamby/master' into merge

* commit 'lamby/master': (23 commits)
  'as' is a reserved word in Python 2.6.
  Don't shadow 'file' builtin in daklib.queue.
  Use 4 spaces for indentation.
  Correct code to match documentation of Binary.check_utf8_package
  Remove unused 'time' import.
  Correct isinstance call; DictType is not an actual type.
  dak's "Dict" is a synonym of the 'dict' builtin
  Use foo.iteritems() to avoid reentrancy with foo[k].
  Don't shadow 'map' builtin in daklib.utils.TemplateSubst
  Update comment about our monkeypatching of commands.getstatusoutput.
  Remove duplicated 'commands' import.
  Don't use deprecated string module.
  Remove unused imports from daklib.queue.
  Remove shadowing builtins in daklib.queue
  Remove daklib's now-unused extension functionality.
  Remove shadowing builtins in daklib.dbconn.
  Don't shadow 'dir' builtin in daklib.utils.get_changes_files
  We don't use re_is_orig_source in daklib.utils anymore.
  Don't shadow "dir" builtin.
  Fix uid references in DM upload error codepaths.
  ...

Signed-off-by: Joerg Jaspert <joerg@debian.org>

'as' is a reserved word in Python 2.6.

Signed-off-by: Chris Lamb <lamby@debian.org>

Don't shadow 'file' builtin in daklib.queue.

Signed-off-by: Chris Lamb <lamby@debian.org>

Use 4 spaces for indentation.

Signed-off-by: Chris Lamb <lamby@debian.org>