Uploaded image for project: 'Opencast'
  1. MH-13238

ServiceRegistry max.attempts ignored when handling related services

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects versions: 3.7, 4.5, 5.2
    • Fix versions: 4.6
    • Components: Backend Software
    • Labels:
      None
    • Severity:
      Incorrectly Functioning With Workaround
    • Steps to reproduce:
      Hide
      Steps to reproduce:
      set max.attempts = 10

      expect ingest jobs to fail 10 times before service is put into error state, however it occurs usual after 1 or 2 further failed jobs.

      It appears that related services are changed from WARNING straight to ERROR without checking max.attempts. This is further confused as the log message for related services is using the currentService details in some places.
       

      2018-10-30 13:56:34,390 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-01 11:56:04,503 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-05 14:56:24,762 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-06 10:58:17,078 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk
      2018-11-06 10:58:22,192 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-06 11:55:40,318 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk
      2018-11-06 11:55:41,241 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-06 11:58:09,170 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-06 12:00:06,374 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk
      2018-11-06 12:09:58,580 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-06 13:56:29,433 | INFO | (ServiceRegistryJpaImpl:2409) - State reset to NORMAL for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk
      2018-11-06 15:57:57,898 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-07 00:47:18,319 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk
      2018-11-07 00:49:33,139 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-07 11:57:25,012 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk
      2018-11-07 11:57:39,079 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-08 15:56:35,454 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-09 03:26:52,189 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload02.opencast.manchester.ac.uk
      2018-11-09 12:58:47,961 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-12 11:58:07,571 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-12 12:58:39,363 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-12 13:57:50,727 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-12 13:57:51,140 | INFO | (ServiceRegistryJpaImpl:2453) - State reset to NORMAL for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-12 16:57:08,374 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-13 12:56:28,163 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-13 14:56:26,662 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-13 14:58:23,902 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk
      2018-11-13 14:58:48,295 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-14 10:28:27,410 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-14 11:56:27,087 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-14 12:57:33,823 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-15 11:56:22,112 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-15 11:56:22,522 | INFO | (ServiceRegistryJpaImpl:2453) - State reset to NORMAL for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
      2018-11-15 12:56:29,994 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk



      Show
      Steps to reproduce: set max.attempts = 10 expect ingest jobs to fail 10 times before service is put into error state, however it occurs usual after 1 or 2 further failed jobs. It appears that related services are changed from WARNING straight to ERROR without checking max.attempts. This is further confused as the log message for related services is using the currentService details in some places.   2018-10-30 13:56:34,390 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-01 11:56:04,503 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-05 14:56:24,762 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-06 10:58:17,078 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk 2018-11-06 10:58:22,192 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-06 11:55:40,318 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk 2018-11-06 11:55:41,241 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-06 11:58:09,170 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-06 12:00:06,374 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk 2018-11-06 12:09:58,580 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-06 13:56:29,433 | INFO | (ServiceRegistryJpaImpl:2409) - State reset to NORMAL for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk 2018-11-06 15:57:57,898 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-07 00:47:18,319 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk 2018-11-07 00:49:33,139 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-07 11:57:25,012 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk 2018-11-07 11:57:39,079 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-08 15:56:35,454 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-09 03:26:52,189 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload02.opencast.manchester.ac.uk 2018-11-09 12:58:47,961 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-12 11:58:07,571 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-12 12:58:39,363 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-12 13:57:50,727 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-12 13:57:51,140 | INFO | (ServiceRegistryJpaImpl:2453) - State reset to NORMAL for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-12 16:57:08,374 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-13 12:56:28,163 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-13 14:56:26,662 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-13 14:58:23,902 | INFO | (ServiceRegistryJpaImpl:2416) - State reset to WARNING for related service org.opencastproject.ingest on host http://upload00.opencast.manchester.ac.uk 2018-11-13 14:58:48,295 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-14 10:28:27,410 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-14 11:56:27,087 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-14 12:57:33,823 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-15 11:56:22,112 | INFO | (ServiceRegistryJpaImpl:2431) - State set to WARNING for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-15 11:56:22,522 | INFO | (ServiceRegistryJpaImpl:2453) - State reset to NORMAL for current service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk 2018-11-15 12:56:29,994 | INFO | (ServiceRegistryJpaImpl:2464) - State set to ERROR for related service org.opencastproject.ingest on host http://upload03.opencast.manchester.ac.uk
    • Tags (folksonomy):

      TestRail: Results

        Attachments

          Activity

            People

            • Assignee:
              james.perrin James Perrin
              Reporter:
              james.perrin James Perrin
            • Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                TestRail: Cases