Series cannot be created

Steps to reproduce

Currently series cannot be created under the develop branch (see screenshot of develop.opencast.org). The error appears when trying to update the solr index:

java.lang.NoSuchMethodError: org.tartarus.snowball.ext.EnglishStemmer.eq_s(ILjava/lang/StringZ
at org.tartarus.snowball.ext.EnglishStemmer.r_prelude(EnglishStemmer.java:188)
at org.tartarus.snowball.ext.EnglishStemmer.stem(EnglishStemmer.java:1196)
at org.apache.solr.analysis.SnowballPorterFilter.incrementToken(SnowballPorterFilterFactory.java:126)
at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:406)
at org.apache.solr.analysis.BufferedTokenStream.read(BufferedTokenStream.java:97)
at org.apache.solr.analysis.BufferedTokenStream.next(BufferedTokenStream.java:83)
at org.apache.lucene.analysis.TokenStream.incrementToken(TokenStream.java:321)
at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:138)
at org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocFieldProcessorPerThread.java:244)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:828)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:809)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:2683)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:2655)
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:241)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:61)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:139)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:139)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:64)
at org.opencastproject.series.impl.solr.SeriesServiceSolrIndex.updateIndex(SeriesServiceSolrIndex.java:308)
at org.opencastproject.series.impl.SeriesServiceImpl.updateSeries(SeriesServiceImpl.java:211)
at org.opencastproject.index.service.impl.IndexServiceImpl.createSeries(IndexServiceImpl.java:1854)
at org.opencastproject.adminui.endpoint.SeriesEndpoint.createNewSeries(SeriesEndpoint.java:534)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_151]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_151]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_151]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_151]
at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:180)
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
... 89 more

Apparently this problem was introduced by PR #83 which contained the update to Java8, since reverting the commit results in the error disappearing.

Activity

Show:
Julian Kniephoff
February 16, 2018, 5:03 PM
Edited

I think I know what the problem is, but I don't know what the idiomatic solution is, yet. FWIW:

Opencasts `solr` bundle embeds Lucene, which has a copy of the `org.tartarus.snowball` package. This package is also exported from `solr` via the `-exportcontents` directive. This last part also makes the bundle plugin add that package to `solr`-s `Import-Package` directive, so that it can be picked up from another bundle, should it already be loaded. This is good practice to avoid incompatibilities between bundles and also to keep the number of duplicate packages small. So far so good.

Opencasts `search` module *also* embeds Lucene, but a much newer version, which again contains a copy of `org.tartaruss.snowball`. This is again added to `-exportcontents`, yada yada. The `eq_s` method of this newer version apparently has a different signature, not taking a `String`, but a `CharSequence`. (Or was it the other way around? I don't remember.)

The problem now arises when `search` is loaded before `solr`, because then `solr` imports `org.tartarus.snowball` from the newer Lucene, but the Lucene included in `solr` still calls `eq_r` with a `String`. Now you might wonder why `solr` doesn't also use the Lucene from `search`? Well for one it of course shouldn't because of a vastly different version, but also because `solr` explicitly excludes `org.apache.lucene.*` from its `Import-Package`-es, so it is guaranteed to use its embedded version of Lucene.

Why did this happen only now? Maybe the bundle plugin changed some arbitrary factor that determines the order in which bundles are loaded or used for resolving imports. This does not really matter, IMO, though, since we should not depend on that order, anyway.

Now I have no idea whether the configuration of these two bundles is any good to begin with or whether it maybe should be rethought from scratch, anyway, but if we just wanted to fix it, there are a few possibilities:

  • Do not export `snowball` from `search`, but that is actually against OSGi best practices as far as I understood them (see above, we want to share packages if possible)

  • Do not import `snowball` in `solr`, but that has the same drawback, actually. However it would be consistent with bundles already excluding the Lucene packages from their imports

  • Correctly version the import and export packages. I have no idea how to do this at the moment, unfortunately, but would be willing (eager, even) to learn about this. Currently the Lucene and snowball packages are just assigned the version of Opencast, strangely.

Kind of orthogonal to all of that: Updating the Lucene version of `solr` would probably also "fix" (as in "hide until the next divergence) the problem and should probably also be done at some point, but not as *the* solution to this problem, IMO.

Do with this what you will. If no one has fixed this by Monday, I might claim it.

Julian Kniephoff
February 19, 2018, 1:06 PM

Assignee

Julian Kniephoff

Reporter

Katrin Ihler

Severity

Incorrectly Functioning Without Workaround

Tags (folksonomy)

None

Components

Fix versions

Affects versions

Priority

Blocker
Configure