Job dispatching can be slowed down excessively by host loads query

Description

With a significant number of jobs (in history and/or running), job dispatching can become very slow, in excess of 3 minutes per dispatch cycle.

This is caused by the call:

SystemLoad systemLoad = getHostLoads(em, true);

for every job being dispatched. This is the query

https://bitbucket.org/cilt/opencast/src/dc68c37d2eea7adbd32edba371d5192c1351feed/modules/matterhorn-common-jpa-impl/src/main/java/org/opencastproject/serviceregistry/impl/jpa/ServiceRegistrationJpaImpl.java?at=r%2F3.x&fileviewer=file-view-default#ServiceRegistrationJpaImpl.java-64

On our 3.x production system, New Relic reports this query taking 94% of all database time. Response time can be up to 2s, and the query can be called hundreds of times per minute.

The job dispatching code needs refactoring to avoid this expensive query being called frequently.

Assignee

Greg Logan

Reporter

Stephen Marquard

Tags (folksonomy)

None

Components

Fix versions

Affects versions

Priority

Critical
Configure