Uploaded image for project: 'Opencast'
  1. MH-13388

Updating job load values for composer service on worker nodes causes running jobs to fail

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed and reviewed
    • Affects versions: 6.2
    • Fix versions: 6.3
    • Components: Backend Software
    • Labels:
      None
    • Severity:
      Incorrectly Functioning With Workaround
    • Steps to reproduce:
      Hide
      Steps to reproduce:
      1. Edit etc/org.opencastproject.composer.impl.ComposerServiceImpl.cfg on a worker node and change job load values while jobs are running.
       
       Actual Results:
       
       Running jobs fail.

       Expected Results:
       
      Running jobs should continue.
       
       Workaround (if any):
       
      Don't change this file live on a running worker node with active jobs.

      Service gets deactivated and reactivated rather than just updated:

      2019-02-21 16:00:59,466 | INFO | CM Event Dispatcher (Fire ConfigurationEvent: pid=org.opencastproject.composer.impl.ComposerServiceImpl) | (ComposerServiceImpl:255) - Deactivating composer service
      2019-02-21 16:00:59,530 | INFO | CM Event Dispatcher (Fire ConfigurationEvent: pid=org.opencastproject.composer.impl.ComposerServiceImpl) | (ComposerServiceImpl:248) - Activating composer service

      Jobs fail like this:

      org.opencastproject.composer.api.EncoderException: java.io.IOException: Stream closed
      at org.opencastproject.composer.impl.EncoderEngine.process(EncoderEngine.java:243)
      at org.opencastproject.composer.impl.ComposerServiceImpl.encode(ComposerServiceImpl.java:369)
      at org.opencastproject.composer.impl.ComposerServiceImpl.mux(ComposerServiceImpl.java:608)
      at org.opencastproject.composer.impl.ComposerServiceImpl.process(ComposerServiceImpl.java:1400)
      at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:314)
      at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:273)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Stream closed
      at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)
      at java.io.BufferedInputStream.read1(BufferedInputStream.java:283)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
      at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
      at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
      at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
      at java.io.InputStreamReader.read(InputStreamReader.java:184)
      at java.io.BufferedReader.fill(BufferedReader.java:161)
      at java.io.BufferedReader.readLine(BufferedReader.java:324)
      at java.io.BufferedReader.readLine(BufferedReader.java:389)
      at org.opencastproject.composer.impl.EncoderEngine.process(EncoderEngine.java:221)
      ... 9 more
      Show
      Steps to reproduce: 1. Edit etc/org.opencastproject.composer.impl.ComposerServiceImpl.cfg on a worker node and change job load values while jobs are running.    Actual Results:    Running jobs fail.  Expected Results:   Running jobs should continue.    Workaround (if any):   Don't change this file live on a running worker node with active jobs. Service gets deactivated and reactivated rather than just updated: 2019-02-21 16:00:59,466 | INFO | CM Event Dispatcher (Fire ConfigurationEvent: pid=org.opencastproject.composer.impl.ComposerServiceImpl) | (ComposerServiceImpl:255) - Deactivating composer service 2019-02-21 16:00:59,530 | INFO | CM Event Dispatcher (Fire ConfigurationEvent: pid=org.opencastproject.composer.impl.ComposerServiceImpl) | (ComposerServiceImpl:248) - Activating composer service Jobs fail like this: org.opencastproject.composer.api.EncoderException: java.io.IOException: Stream closed at org.opencastproject.composer.impl.EncoderEngine.process(EncoderEngine.java:243) at org.opencastproject.composer.impl.ComposerServiceImpl.encode(ComposerServiceImpl.java:369) at org.opencastproject.composer.impl.ComposerServiceImpl.mux(ComposerServiceImpl.java:608) at org.opencastproject.composer.impl.ComposerServiceImpl.process(ComposerServiceImpl.java:1400) at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:314) at org.opencastproject.job.api.AbstractJobProducer$JobRunner.call(AbstractJobProducer.java:273) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170) at java.io.BufferedInputStream.read1(BufferedInputStream.java:283) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) at java.io.BufferedReader.readLine(BufferedReader.java:389) at org.opencastproject.composer.impl.EncoderEngine.process(EncoderEngine.java:221) ... 9 more

      TestRail: Results

        Attachments

          Activity

            People

            • Assignee:
              smarquard Stephen Marquard
              Reporter:
              smarquard Stephen Marquard
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                TestRail: Cases