Inbox has a 2GB video file size limit: larger files are truncated

Steps to reproduce

Steps to reproduce:
you can use an all-in-one or a distributed matterhorn this happens on both kinds of systems. The all in one does not uses nfs or any other kind of network storage.
1. If I copy a video > 2GB in the inbox it will be copied correctly to files/collection/inbox and is still > 2 GB
2. After I ingested the video with the upload UI and the option "designated inbox" the file seems to be copied to the workspace. Unexpectedly this is done with a http-download instead of the simple file copy.
2011-05-03 15:37:19 INFO (WorkspaceImpl:245) - Downloading http://opencast.virtuos.uos.de:8080/files/mediapackage/31754773-b3f2-44d0-8a90-3f8965e43765/source-track/449.mpg to /data/matterhorn/content/workspace/mediapackage/31754773-b3f2-44d0-8a90-3f8965e43765/source-track/449.mpg

In the matterhorn config file the file.repo.url is not commented:
org.opencastproject.file.repo.url=${org.opencastproject.admin.ui.url}

3. after the "Downloading" the file is only 2GB large.

4. If ingest the file as a ZIP package over the inbox it keeps the correct length! Here I don't see a "Downloading" entry in the logs

Actual Results:

Expected Results:

Workaround (if any):

Activity

Show:
Christopher Brooks
May 12, 2011, 8:28 PM

Dropping to critical, work around is not to use the inbox (which is lame), but to use the REST endpoints.

Rüdiger Rolf
May 17, 2011, 12:12 PM

I commited a patch to the 1.1 branch. Please review it

Tobias Wunden
May 17, 2011, 3:29 PM

Rüdiger, I don't see how this fixes the problem. Could you please help me understand?

Rüdiger Rolf
May 17, 2011, 3:48 PM

The if (f.isFile) has permited that a file not already in the workspace can be copied to the workspace with a a simple file copy or even linking. If the file did not already exist in the workspace f did not exist and f.isFile failed.

The the file was copied over the http-connection and that this sometimes causes shorter files is a know issue ().

So this solution works for all-in-one and distributed with workspace-impl (not workspace stub) as far as I have tested it.

Rüdiger Rolf
July 27, 2011, 12:34 PM

Merged to other branch and trunk

Assignee

Rüdiger Rolf

Reporter

Rüdiger Rolf

Severity

Data Loss/Corruption

Tags (folksonomy)

None

Components

Fix versions

Affects versions

Priority

Critical
Configure