Long delay prior to initiation of upload on large files
November 18, 2008 - 3:37pm — nohawknomiss
I've just begun to integrate SWFUpload into my RoR application for the purpose of uploading video files from client (MacBook) to server (Linux).
It works fine, but on larger files (say, 100MB), my entire machine locks up while -- I presume -- the file is enqueued for upload. This is not a happy-user experience. I imagine my laptop is busy swapping.
I could diagnose this further, but I imagine I'd be rediscovering something already known:
Is this behavior expected?
Are files queued in RAM?
Might there be some configuration or integration step that I missed?
Is this covered already somewhere else in the forum? (I did try searching... but maybe didn't hit the right keywords)
Thanks!
Loading in to memory
We don't have the resources (physical, not temporal) to test on Mac. But I've had reports the in the Mac Flash Player that the entire file is loaded into memory.
This issue hasn't been noted in the documentation. We've had some reports but nothing concrete.
I have a few questions that would be helpful:
Can you observer the browser memory usage increase when files are queued? Is the increase proportional to the file size?
Does the issue occur when the file is queued or after startUpload is called and the file is uploaded?
Thanks for your quick
Thanks for your quick response!
Yes, I can observe the browser mem usage go up approximately proportionately: at least, RSIZE goes up by about 400MB when uploading a single 500MB file.
As for whether or not the issue occurs prior to the call to startUpload: I put an alert in at this point in the swfupload.js code:
// Public: startUpload starts uploading the first file in the queue unless
// the optional parameter 'fileID' specifies the ID
SWFUpload.prototype.startUpload = function (fileID) {
alert ("Hi!!");
this.callFlash("StartUpload", [fileID]);
};
The long delay occurs AFTER the alert (which is raised almost immediately)
I take it this means the behavior is internal to flash on Mac -- so... I could break the file up on the client side & send it in 64kb chunks -- is that something that could be done reasonably easily within the SWFUpload source itself?
What do you suggest?
Thanks!
Flash Player 10
Supposedly some of this is possible in Flash Player 10. They've added the ability to load a file and get its contents. However, I suspect this will cause the same problem. It looks like it loads the entire file into memory.
Furthermore, there is then no way to upload the loaded bytes, or stream them or chunk them once they are loaded. It seems the only options are:
1) Prompt the user to save a new file with the bytes, then prompt them to reselect the new file and upload it. That doesn't solve anything in this case.
2) Use a socket and build our own HTTP client for connecting to the server, sending a properly formatted HTTP request and properly handling the server's responses. That is well outside the scope of anything we want to do (although it would fix 99% of the Flash Player bugs that hound SWFUpload). But we'd probably still have the memory issue because all the bytes from the 100 MB file have been loaded up. But at least it'd affect all platforms now
I was surprised that many of these bugs weren't fixed in Flash Player 10. The only changes were those relating to what seems to be Adobe's core interests for Flash: graphic manipulation and video playback. The more I work with Flash the more surprised I am that it even supports uploading files. The whole process is extremely buggy and I haven't seen much effort go into fixing those bugs. It's frustrating for us but understandable from a business point of view. They make oodles of money on those features but haven't seen a penny from SWFUpload.
Avatar upload?
It has to work well enough to upload a custom avatar, is all!
I guess the threat of Silverlight supporting file upload efficiently on Macs wouldn't be enough to scare them, huh?
Do you guys talk with Adobe directly and ask them to fix their problems? This doesn't seem like it would be hard to fix: needs the same kind of double buffering that every file transfer mechanism already uses.
For now, I guess I'm inclined to use this for image uploads, but stick to the direct file upload mechanism for video for now -- access to source is a big plus.
Thanks again!