Make distribution and retraction efficient

Description

Currently, a job object is created and made persistent in the database for each single file that is distributed or retracted. While this approach allows a conceptually very nice implementation, is suffers some severe operational problems:

a) database bloat
b) search index bloat

a) leads to high loads on the database server and large database size which is pain to backup
b) leads to long search index re-built times which is very undesirable

While there is a cleanup script to remove old job objects, it would be preferable to not create them in first place and therefore avoiding all the bloats at first hand instead of fighting them afterwards.

A possible solution could be to re-implement both distribution and retraction to use a single job for all files.

Activity

Show:
Stephen Marquard
July 20, 2016, 6:38 PM

I agree, conceptually and in implementation distribution should be a single job that distributes all the required files (for a target service).

Fixed and reviewed
Your pinned fields
Click on the next to a field label to start pinning.

Assignee

Sven Stauber

Reporter

Sven Stauber