Large Uploads
From Xibo
 |
Languages: |
English |
User Input
Dan Do you have an opinion on a standalone application (maybe in .NET, maybe in Java) that ships with the Xibo Client specifically for uploading large files to Xibo? Its an idea I have been toying with for some time...
ks
I think there might be better solutions than a standalone large-file-uploader. If you were building a complete standalone admin client, that would be different and of course should handle large files.
On a trivial point, we are actually deploying the xibo client on "standalone" displays that aren't otherwise used for administration (the Asus Eee boxes are a very nice platform for that). I personally use Linux, so a .NET client would be an inconvenience (though I'm just setting this up, the folks who will use it day to day are running windows).
But since administration is web-based already, it seems that keeping it contained in the web browser would be the most convenient. On a reasonably fast network connection (relative to the size of the upload) there shouldn't be a problem on the client end with just using the browser (eg. javascript HttpRequest) to do the upload. The problem we're hitting is on the server, specifically in php, so a server-side fix would probably be more appropriate.
So .. could do a custom java (or whatever) server to catch the upload, but that's probably just 'recreating the wheel' when there are quite a few 'wheels' around to choose from already. There are a lot of web-based file managers you could consider (I'd imagine you'd find something that could ship and integrate right with the xibo server), and there's also standard WebDAV which is designed exactly for http server file access. (You'll find more searching, but one library that might work for DAV is "davclient.js" (http://debris.demon.nl/projects/davclient.js/doc/README.html).)
Alex H
- FancyUpload Available under MIT license
- Flash Uploader OS license. Webpage down when I tried to look.
- JumpLoader free not Free, but I think license wise OK.
- JUpload GPL
Main issue with the Java uploaders is that they require a Java-based server to talk to. If possible I think we should avoid having to get users to setup both a PHP server environment and a Java-enabled server too. I think one possible solution here it to take JUpload and extend it to support posting chunks to XMDS via SOAP.
Possible protocol:
- Client says to server "I want to upload new file with <Name>, <size>, <checksum>, <tags> as <user> with <password>".
- Server gives client a one-time token to identify the file in transfer.
- Server adds database record for the new file, but hides it from the admin interface (schema needs extending for that? or mark it as retired?)
- Client posts numbered chunks to the server using the one-time key. If chunk fails, retry until user quits or chunk succeeds.
- On completion, server checksums file and compares to client checksum and if all went OK marks the media available for use.
Problems:
- How does a completely disconnected session restart (ie client or server quits)? Server is stateless, but need to ensure that for a given input file, the one-time session key is identical so that the download can resume. Also server needs to tell client which chunk number or a size offset to start at if file is partially uploaded.
- Part-uploaded files are left intact, so a background process is needed on the server to go around and periodically delete un-resumed uploads.
