Uploaded image for project: 'Opencast'
  1. MH-12025

Performance issues with very large groups

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects versions: 2.3.1
    • Fix versions: 4.6
    • Labels:
      None
    • Severity:
      Performance
    • Steps to reproduce:
      Hide
      Steps to reproduce:
      1. Create 5000 internal users
      2. Add all 5000 users to a group (at creation time or later)
      3. In the Admin UI, access Organization / Groups
       
       Actual Results:
       
      The REST endpoint for groups is extremely slow, as getting a list of groups also loads all of the group members.

       Expected Results:
       
      Displaying a list of groups should be fast.

      In the storage implementation, the group members should be lazy-loaded, i.e. it should be possible to get a list of groups without also loading all members for all groups.
       
      In one case, it took 5.5min for the REST call

      /admin-ng/groups/groups.json?limit=10&offset=0&sort=name:ASC

      to complete. Adding these keys to the database helped slightly (reduced load time to around 2 minutes):

      alter table mh_group_member add unique key mh_group_member_i (group_id, member);
      alter table mh_group_member add key mh_group_member_member_i ( member);

      but the real solution is the groups endpoint should not load the membership unless required to do so.

       Workaround (if any):
       
      Don't create groups with large membership.




      Show
      Steps to reproduce: 1. Create 5000 internal users 2. Add all 5000 users to a group (at creation time or later) 3. In the Admin UI, access Organization / Groups    Actual Results:   The REST endpoint for groups is extremely slow, as getting a list of groups also loads all of the group members.  Expected Results:   Displaying a list of groups should be fast. In the storage implementation, the group members should be lazy-loaded, i.e. it should be possible to get a list of groups without also loading all members for all groups.   In one case, it took 5.5min for the REST call /admin-ng/groups/groups.json?limit=10&offset=0&sort=name:ASC to complete. Adding these keys to the database helped slightly (reduced load time to around 2 minutes): alter table mh_group_member add unique key mh_group_member_i (group_id, member); alter table mh_group_member add key mh_group_member_member_i ( member); but the real solution is the groups endpoint should not load the membership unless required to do so.  Workaround (if any):   Don't create groups with large membership.

      TestRail: Results

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              smarquard Stephen Marquard
            • Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                TestRail: Cases