Conditionally synchronize Archive Service's add mediapackge

Description

Currently, the archive add() method is synchronized in the archive service to protect "oc_latest_version" flag on a mediapackage. But, this protection is only needed for concurrent archives of the same mediapackge Id. Archive's for different mediapackage Id's have their own oc_latest_version flag, so they do not need to be synchronized. Also, it is very unlikely that two mediapackges of the same mediapackageId to be archiving at the same time.

Allowing more than one mediapackage to archive at the same time speeds up the archive step when multiple workflows are processing.

The proposal is to keep track of actively archiving mediapackages and conditionally synchronize when two mediapackages have the same Id.

https://bitbucket.org/opencast-community/matterhorn/src/24371827d47fcd8d2e7238319411b961ec3697d3/modules/matterhorn-archive-base/src/main/java/org/opencastproject/archive/base/ArchiveBase.java?at=r%2F2.1.x&fileviewer=file-view-default#ArchiveBase.java-172

Activity

Show:
Former user
March 23, 2016, 5:40 PM

The proposed change has the same conditional as the current synchronized method, which is that synchronize only protects processes running on that node. For instance, when one archive service impl is active in an all-in-one or on the admin node. The synchronize does not protect when multiple archive service impls are running on different workers nodes with the central solr server.

James Perrin
October 10, 2016, 2:58 PM

We are currently testing the attached patch. https://opencast.jira.com/secure/attachment/23800/archive_sync_per_mp.diff .Though this doesn't allow multiple archive instances it does provide synchronization on a per mediapackage basis. It uses a simple time expiring cache to manage synchronization lock objects stored by mediapackageId. Comments welcome.

James Perrin
October 19, 2016, 10:35 PM

We've now put this code into production.

Former user
October 20, 2016, 12:57 PM
Edited

Can you make a pull request for this? I will review it. Rute Santos made a mediapackage lock service for HUDCE so we can update mediapackages from different services without collision, in cases where Optimistic update are likely to fail. i.e. I already like the MP Lock strategy.

James Perrin
November 18, 2016, 4:08 PM

Backporting my patch to the Episode service in 1.6.3 is causing stuff to unravel, as the guava cache packages are in guava-osgi 11+ (currently 9) which means porting all modules to the newer version of guava. Watch this space.

Fixed and reviewed

Assignee

James Perrin

Reporter

Former user

Tags (folksonomy)

None

Components

Fix versions

Affects versions

Priority

Minor