Performance issues with very large groups

Steps to reproduce

Steps to reproduce:
1. Create 5000 internal users
2. Add all 5000 users to a group (at creation time or later)
3. In the Admin UI, access Organization / Groups

Actual Results:

The REST endpoint for groups is extremely slow, as getting a list of groups also loads all of the group members.

Expected Results:

Displaying a list of groups should be fast.

In the storage implementation, the group members should be lazy-loaded, i.e. it should be possible to get a list of groups without also loading all members for all groups.

In one case, it took 5.5min for the REST call


to complete. Adding these keys to the database helped slightly (reduced load time to around 2 minutes):

alter table mh_group_member add unique key mh_group_member_i (group_id, member);
alter table mh_group_member add key mh_group_member_member_i ( member);

but the real solution is the groups endpoint should not load the membership unless required to do so.

Workaround (if any):

Don't create groups with large membership.


Sven Stauber
April 13, 2017, 9:33 PM

Note that this issue smells similar to

Stephen Marquard
January 31, 2017, 10:13 AM

There may also be some issue with the External API index getting out of sync when you create a very large group, where you get the correct results from:




stops updating.

Stephen Marquard
January 31, 2017, 8:51 AM

Note that there is a whole parallel storage and retrieval mechanism for groups in the index service, which is used by the External API:





Stephen Marquard