Search index rebuild results in missing dcCreated

Steps to reproduce

Steps to reproduce:
1. Delete search index (on a non empty database)
2. Restart matterhorn; wait for index to rebuild
3. On completion go to the media module - all dates appear as n.a.

Actual Results:

Media module dates appear as n.a.
/search/episode.json endopint shows no dcCreated entries
Expected Results:

The dates should appear as they did before i.e. the index should rebuild successfully

Workaround (if any):

rebuild the index by deleting it and restarting the search service implementation via the system/console

Activity

Show:
Former user
May 12, 2017, 6:46 PM

FYI - The delay in does not fix issue. The delay in retry populateIndex is only if error occured when build the index immediately on load. The issue of this ticket is that the first time would succeed but contain missing metadata.

3x The SearchServiceImpl's active creates the SolrIndexManager and passes the current set of StaticMetadataService that are set dynamically, only one is needed for SearchServiceImpl to be satisfied, and two are needed normally to extract required metadata.

https://bitbucket.org/opencast-community/matterhorn/src/14174860df6161b65217385956ff4530af6253ca/modules/matterhorn-search-service-impl/src/main/java/org/opencastproject/search/impl/SearchServiceImpl.java?at=r%2F3.x&fileviewer=file-view-default#SearchServiceImpl.java-232

I have not yet had luck adjusting the karaf bundle load levels to load the SearchServiceImpl after the two StaticMetadataServices. This would be the simplest resolution for 3x.

Former user
May 23, 2016, 11:57 AM

Adding a delay before starting to build of the search index on load, see MH-11584, will alleviate this issue.

Former user
January 28, 2016, 2:46 PM
Edited

I'm uncomfortable with the factory fix solution for the non-karaf versions of Matterhorn because it adds complexity and constraints that prevents mediapackage parsers from being added and removed easily from the system. The post-karaf versions of Matterhorn has the ability of module load order, which allows underlying services, like parser services, to be loaded before the modules that consume them. This allows parsers to be added and removed simply by removing them from the load config.

Any site that uses load profiles, instead of hot loading, can implement this fix by loading the search service on a level after the main matterhorn modules within the load profile config.

Former user
October 22, 2015, 10:54 PM

I'm going to verify if the 2.x karaf-maven-plugin automatically does the right thing* when assigning the Search service's bundle load level when constructing the startup.properties.

  • assign a lower bundle load level to bundles that implement StaticMetadataServices than it does to the Search service.

Former user
October 22, 2015, 6:20 PM
Edited

Below is a more detailed explanation of the Aggregator class by Christoph D.
In summary, the matterhorn-search-service-impl would no longer require 0..n StaticMetadataService before begins rebuilding its index. It requires exactly one StaticMetadataServiceAggregator implementation. The StaticMetadataServiceAggregator explicitly requires two services: one StaticMetadataServiceDublinCoreImpl and one StaticMetadataServiceMediaPackageImpl.

My reservation against the Aggregator is that it is less flexible. A new StaticMetadataService can't just be added to the mix (and rebuild the index with it). Our site resolved the issue by not hot-loading all the Matterhorn jars at the same felix load level, but to load the Search service after all the StaticMetadataService implementations have been loaded.

===============================================
Aggregator class by Christoph D.

the aggregator could decouple dependencies like this: (-> means "depends on")

[1]
Bundle "matterhorn-search-service-impl"

SearchServiceImpl ->(static, 1..1) Aggregator

Aggregator
getStaticMetadataServices(): List<StaticMetadataService>

[2]
Bundle "matterhorn-search-service-aggregator-default"

Contains an implementation of Aggregator with dynamic dependencies to StaticMetadataServices, the same way as the current SearchServiceImpl does.

Aggregator
->(dynamic, 0..n) StaticMetadataService

[3]
Bundle "harvard-search-service-aggregator"

Contains an implementation of Aggregator with static dependencies to the required StaticMetadataServices

Aggregator
->(static, 1..1) StaticMetadataServiceA
->(static, 1..1) StaticMetadataServiceB
...
===============================================

Fixed and reviewed

Assignee

Former user

Reporter

Edmore Moyo

Severity

Incorrectly Functioning Without Workaround