Uploaded image for project: 'Opencast'
  1. MH-13600

AssetManagerDeleteWorkflowOperation not deleting correct

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects versions: 7.0
    • Fix versions: 7.0
    • Components: Backend Software
    • Labels:
      None
    • Severity:
      Data Loss/Corruption
    • Steps to reproduce:
      Hide
      using the new operation „asset-delete“ with keep-last-snapshot = true



      Using this operation as one of the last steps in a publish workflow (here: publish-after-cutting.xml) deletes all ACLs for that event in the adminUI and an unprivileged user does not see the event any longer.

      He/she is able to follow the workflow step by step and suddenly the event vanishes, when WOH asset-delete is running. Only the admin is still able to see the event.

      A user also can not apply the series ACLs again. It has no effect.



      When you examine the log file you see that the last created archive version is 8 (in this example) and version 0 -7 should be deleted. But the data is requested from asset version 6, which is obviously already deleted ???

      Perhaps version 8 is not yet ready ???



      019-05-30T10:24:00,111 | INFO | (OaiPmhPublicationServiceRemoteImpl:82) - Publishing media package bbcd18c7-4ca2-4c9d-946a-6713acc97a14 to OAI-PMH channel default using a remote publication service
      2019-05-30T10:24:31,708 | INFO | (AssetManagerSnapshotWorkflowOperationHandler:91) - Take snapshot of media package bbcd18c7-4ca2-4c9d-946a-6713acc97a14
      2019-05-30T10:24:31,723 | INFO | (AbstractAssetManager:242) - Creating new version 8 of media package bbcd18c7-4ca2-4c9d-946a-6713acc97a14
      2019-05-30T10:24:31,862 | INFO | (AssetManagerWithMessaging:211) - Send update message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 8 to ActiveMQ
      2019-05-30T10:24:36,963 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 0 to ActiveMQ
      2019-05-30T10:24:36,983 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 1 to ActiveMQ
      2019-05-30T10:24:37,018 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 2 to ActiveMQ
      2019-05-30T10:24:37,039 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 3 to ActiveMQ
      2019-05-30T10:24:37,055 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 4 to ActiveMQ
      2019-05-30T10:24:37,072 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 5 to ActiveMQ
      2019-05-30T10:24:37,111 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 6 to ActiveMQ
      2019-05-30T10:24:37,130 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 7 to ActiveMQ
      2019-05-30T10:24:37,132 | INFO | (AssetManagerDeleteWorkflowOperationHandler:76) - Deleting all but latest Snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14
      2019-05-30T10:24:37,132 | INFO | (AssetManagerDeleteWorkflowOperationHandler:85) - Successfully deleted {} version/s episode {} from the asset manager
      2019-05-30T10:24:37,419 | ERROR | (DublinCoreUtil:109) - Unable to load metadata from catalog 'https://opencast-dev-admin.uni-koeln.de/assets/assets/bbcd18c7-4ca2-4c9d-946a-6713acc97a14/2eccc964-0447-41ce-93c7-c8f412aa615c/6/dublincore.xml'
      org.opencastproject.util.NotFoundException: null
      at org.opencastproject.workspace.impl.WorkspaceImpl.downloadIfNecessary(WorkspaceImpl.java:493) ~[?:?]
      at org.opencastproject.workspace.impl.WorkspaceImpl.access$200(WorkspaceImpl.java:101) ~[?:?]
      at org.opencastproject.workspace.impl.WorkspaceImpl$3.xapply(WorkspaceImpl.java:518) ~[?:?]
      at org.opencastproject.workspace.impl.WorkspaceImpl$3.xapply(WorkspaceImpl.java:515) ~[?:?]
      at org.opencastproject.util.data.Function$X.apply(Function.java:111) ~[58:opencast-common:7.0.0.SNAPSHOT]
      at org.opencastproject.util.IoSupport.locked(IoSupport.java:651) ~[58:opencast-common:7.0.0.SNAPSHOT]
      at org.opencastproject.workspace.impl.WorkspaceImpl.get(WorkspaceImpl.java:359) ~[?:?]
      at org.opencastproject.workspace.impl.WorkspaceImpl.read(WorkspaceImpl.java:387) ~[?:?]
      at org.opencastproject.metadata.dublincore.DublinCoreUtil.loadDublinCore(DublinCoreUtil.java:106) [75:opencast-dublincore:7.0.0.SNAPSHOT]
      at org.opencastproject.metadata.dublincore.DublinCoreCatalogService.getMetadata(DublinCoreCatalogService.java:112) [75:opencast-dublincore:7.0.0.SNAPSHOT]
      at org.opencastproject.metadata.dublincore.DublinCoreCatalogService.getMetadata(DublinCoreCatalogService.java:58) [75:opencast-dublincore:7.0.0.SNAPSHOT]
      at org.opencastproject.workflow.impl.WorkflowServiceImpl.populateMediaPackageMetadata(WorkflowServiceImpl.java:1689) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT]
      at org.opencastproject.workflow.impl.WorkflowServiceImpl.update(WorkflowServiceImpl.java:1289) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT]
      at org.opencastproject.workflow.impl.WorkflowServiceImpl.runWorkflowOperation(WorkflowServiceImpl.java:924) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT]
      at org.opencastproject.workflow.impl.WorkflowServiceImpl.process(WorkflowServiceImpl.java:1867) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT]
      at org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:2295) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT]
      at org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:2261) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT]
      at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
      at java.lang.Thread.run(Thread.java:748) [?:?]
      2019-05-30T10:24:37,426 | WARN | (WorkflowServiceImpl:1305) - [>d75f2b3a] Metadata for mediapackage bbcd18c7-4ca2-4c9d-946a-6713acc97a14 could not be updated because it wasn't found
      2019-05-30T10:24:42,055 | ERROR | (DublinCoreUtil:109) - Unable to load metadata from catalog 'https://opencast-dev-admin.uni-koeln.de/assets/assets/bbcd18c7-4ca2-4c9d-946a-6713acc97a14/2eccc964-0447-41ce-93c7-c8f412aa615c/6/dublincore.xml'
      org.opencastproject.util.NotFoundException: null


      You can insert the „ingest-download“ WOH before „asset-delete" ( it is recommended in the documentation ) and doing so the error of missing ACLs can be avoided, because the needed data is downloaded from the asset version 6 before the delete operation. But the documentation only mentions that you need this WOH to avoid logging errors.



      „If the keep-last-snapshot parameter is used, it is advised to use the ingest-download workflow before asset-delete. Otherwise there will be logged a lot of errors for unreferenced snapshots“



      No word that you need it to preserve the ACLs !!!
      Show
      using the new operation „asset-delete“ with keep-last-snapshot = true Using this operation as one of the last steps in a publish workflow (here: publish-after-cutting.xml) deletes all ACLs for that event in the adminUI and an unprivileged user does not see the event any longer. He/she is able to follow the workflow step by step and suddenly the event vanishes, when WOH asset-delete is running. Only the admin is still able to see the event. A user also can not apply the series ACLs again. It has no effect. When you examine the log file you see that the last created archive version is 8 (in this example) and version 0 -7 should be deleted. But the data is requested from asset version 6, which is obviously already deleted ??? Perhaps version 8 is not yet ready ??? 019-05-30T10:24:00,111 | INFO | (OaiPmhPublicationServiceRemoteImpl:82) - Publishing media package bbcd18c7-4ca2-4c9d-946a-6713acc97a14 to OAI-PMH channel default using a remote publication service 2019-05-30T10:24:31,708 | INFO | (AssetManagerSnapshotWorkflowOperationHandler:91) - Take snapshot of media package bbcd18c7-4ca2-4c9d-946a-6713acc97a14 2019-05-30T10:24:31,723 | INFO | (AbstractAssetManager:242) - Creating new version 8 of media package bbcd18c7-4ca2-4c9d-946a-6713acc97a14 2019-05-30T10:24:31,862 | INFO | (AssetManagerWithMessaging:211) - Send update message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 8 to ActiveMQ 2019-05-30T10:24:36,963 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 0 to ActiveMQ 2019-05-30T10:24:36,983 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 1 to ActiveMQ 2019-05-30T10:24:37,018 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 2 to ActiveMQ 2019-05-30T10:24:37,039 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 3 to ActiveMQ 2019-05-30T10:24:37,055 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 4 to ActiveMQ 2019-05-30T10:24:37,072 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 5 to ActiveMQ 2019-05-30T10:24:37,111 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 6 to ActiveMQ 2019-05-30T10:24:37,130 | INFO | (AssetManagerWithMessaging:219) - Send delete message for snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14, 7 to ActiveMQ 2019-05-30T10:24:37,132 | INFO | (AssetManagerDeleteWorkflowOperationHandler:76) - Deleting all but latest Snapshot bbcd18c7-4ca2-4c9d-946a-6713acc97a14 2019-05-30T10:24:37,132 | INFO | (AssetManagerDeleteWorkflowOperationHandler:85) - Successfully deleted {} version/s episode {} from the asset manager 2019-05-30T10:24:37,419 | ERROR | (DublinCoreUtil:109) - Unable to load metadata from catalog ' https://opencast-dev-admin.uni-koeln.de/assets/assets/bbcd18c7-4ca2-4c9d-946a-6713acc97a14/2eccc964-0447-41ce-93c7-c8f412aa615c/6/dublincore.xml' org.opencastproject.util.NotFoundException: null at org.opencastproject.workspace.impl.WorkspaceImpl.downloadIfNecessary(WorkspaceImpl.java:493) ~[?:?] at org.opencastproject.workspace.impl.WorkspaceImpl.access$200(WorkspaceImpl.java:101) ~[?:?] at org.opencastproject.workspace.impl.WorkspaceImpl$3.xapply(WorkspaceImpl.java:518) ~[?:?] at org.opencastproject.workspace.impl.WorkspaceImpl$3.xapply(WorkspaceImpl.java:515) ~[?:?] at org.opencastproject.util.data.Function$X.apply(Function.java:111) ~[58:opencast-common:7.0.0.SNAPSHOT] at org.opencastproject.util.IoSupport.locked(IoSupport.java:651) ~[58:opencast-common:7.0.0.SNAPSHOT] at org.opencastproject.workspace.impl.WorkspaceImpl.get(WorkspaceImpl.java:359) ~[?:?] at org.opencastproject.workspace.impl.WorkspaceImpl.read(WorkspaceImpl.java:387) ~[?:?] at org.opencastproject.metadata.dublincore.DublinCoreUtil.loadDublinCore(DublinCoreUtil.java:106) [75:opencast-dublincore:7.0.0.SNAPSHOT] at org.opencastproject.metadata.dublincore.DublinCoreCatalogService.getMetadata(DublinCoreCatalogService.java:112) [75:opencast-dublincore:7.0.0.SNAPSHOT] at org.opencastproject.metadata.dublincore.DublinCoreCatalogService.getMetadata(DublinCoreCatalogService.java:58) [75:opencast-dublincore:7.0.0.SNAPSHOT] at org.opencastproject.workflow.impl.WorkflowServiceImpl.populateMediaPackageMetadata(WorkflowServiceImpl.java:1689) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT] at org.opencastproject.workflow.impl.WorkflowServiceImpl.update(WorkflowServiceImpl.java:1289) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT] at org.opencastproject.workflow.impl.WorkflowServiceImpl.runWorkflowOperation(WorkflowServiceImpl.java:924) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT] at org.opencastproject.workflow.impl.WorkflowServiceImpl.process(WorkflowServiceImpl.java:1867) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT] at org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:2295) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT] at org.opencastproject.workflow.impl.WorkflowServiceImpl$JobRunner.call(WorkflowServiceImpl.java:2261) [171:opencast-workflow-service-impl:7.0.0.SNAPSHOT] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?] at java.lang.Thread.run(Thread.java:748) [?:?] 2019-05-30T10:24:37,426 | WARN | (WorkflowServiceImpl:1305) - [>d75f2b3a] Metadata for mediapackage bbcd18c7-4ca2-4c9d-946a-6713acc97a14 could not be updated because it wasn't found 2019-05-30T10:24:42,055 | ERROR | (DublinCoreUtil:109) - Unable to load metadata from catalog ' https://opencast-dev-admin.uni-koeln.de/assets/assets/bbcd18c7-4ca2-4c9d-946a-6713acc97a14/2eccc964-0447-41ce-93c7-c8f412aa615c/6/dublincore.xml' org.opencastproject.util.NotFoundException: null You can insert the „ingest-download“ WOH before „asset-delete" ( it is recommended in the documentation ) and doing so the error of missing ACLs can be avoided, because the needed data is downloaded from the asset version 6 before the delete operation. But the documentation only mentions that you need this WOH to avoid logging errors. „If the keep-last-snapshot parameter is used, it is advised to use the ingest-download workflow before asset-delete. Otherwise there will be logged a lot of errors for unreferenced snapshots“ No word that you need it to preserve the ACLs !!!

      TestRail: Results

        Attachments

          Activity

            People

            • Assignee:
              greg_logan Greg Logan
              Reporter:
              langr Ruth Lang
            • Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                TestRail: Cases