Make distribution and retraction efficient
Currently, a job object is created and made persistent in the database for each single file that is distributed or retracted. While this approach allows a conceptually very nice implementation, is suffers some severe operational problems:
a) database bloat
b) search index bloat
a) leads to high loads on the database server and large database size which is pain to backup
b) leads to long search index re-built times which is very undesirable
While there is a cleanup script to remove old job objects, it would be preferable to not create them in first place and therefore avoiding all the bloats at first hand instead of fighting them afterwards.
A possible solution could be to re-implement both distribution and retraction to use a single job for all files.
I agree, conceptually and in implementation distribution should be a single job that distributes all the required files (for a target service).