http file transfer via Matterhorn is very inefficient

Steps to reproduce

Steps to reproduce:

1. Benchmark download of a file from an ingest server using curl, e.g.

curl --digest -u matterhorn_system_accountIGEST_PWD -H "X-Requested-Auth: Digest" -H "X-Opencast-Matterhorn-Authorization: true" --verbose -o file.bin http://servername:8080/files/mediapackage/37a77334-8850-4ec9-8c41-ca62e1f8236d/track-0/presenter.mpeg

2. Install an apache httpd 2.2 or similar server on port 80 mapped to the same file path as the above. Benchmark transfer speed using apache e.g.

curl --verbose -o file.bin http://servername/files/mediapackage/37a77334-8850-4ec9-8c41-ca62e1f8236d/track-0/presenter.mpeg

3. Compare transfer times fetching the file using method 1 (Matterhorn) vs method 2 (apache)

Actual Results:

Transfer times of the file served by Jetty+Matterhorn are approximately 10x the duration when the file is served by Apache.

Sample file:

7760793365 bytes = 7.3G

Transfer time between two VMs with high-speed networking (10 Gbps logical speed):

Jetty+MH: 9min 50s
Apache2: 55s
scp: 2min 10s

These results are reproducible, i.e. multiple transfers show the same speeds. Tests are the same on MH 1.4.x and MH 1.6.x.

Expected Results:

Jetty+Matterhorn should be almost as efficient as Apache.

Workaround (if any):

Status

Assignee

Unassigned

Reporter

Stephen Marquard

Severity

Performance

Tags (folksonomy)

None

Components

Affects versions

Priority

Major
Configure