Azure Storage: BlobClient keeps resetting itself while uploading large files

Issue

I'm trying to upload a 200 MB video file from a mobile phone that's connected to a fast Wifi connection. I'm using v12 of Azure Storage SDK for .NET, but the below code keeps resetting itself after about 30% upload progress. When the reset occurs, progress begins from 0 and there is no exception thrown.

await blobClient.UploadAsync(stream, progressHandler: new Progress<long>(progress =>
{
     // show progress bar
}), cancellationToken: cancellationToken);

Does v12 support uploading large files? If I remember correctly, the old APIs had the ability to upload individual blocks. I was under the impression that the above upload method would handle chunking implicitly.

How can I upload large files using the latest sdk?

P.S. I tried passing a StorageTransferOptions with very high concurrency and 1 MB transfer size, but it made no difference.

EDIT: After waiting for a long time, I was able to get an exception thrown. I see that multiple tasks get canceled because of

Cannot access a disposed object.
Object name: 'MobileAuthenticatedStream'.

  at Mono.Net.Security.MobileAuthenticatedStream.StartOperation (Mono.Net.Security.MobileAuthenticatedStream+OperationType type, Mono.Net.Security.AsyncProtocolRequest asyncRequest, System.Threading.CancellationToken cancellationToken) [0x00245] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:410 
  at System.Net.Http.HttpConnection.WriteAsync (System.ReadOnlyMemory`1[T] source) [0x00118] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:1008 
  at System.IO.Stream.CopyToAsyncInternal (System.IO.Stream destination, System.Int32 bufferSize, System.Threading.CancellationToken cancellationToken) [0x000e7] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/shared/System/IO/Stream.cs:152 
  at Azure.Core.RequestContent+StreamContent.WriteToAsync (System.IO.Stream stream, System.Threading.CancellationToken cancellation) [0x00094] in <7b1dc95b0b4841539beb48023c1128d3>:0 
  at Azure.Core.Pipeline.HttpClientTransport+PipelineRequest+PipelineContentAdapter.SerializeToStreamAsync (System.IO.Stream stream, System.Net.TransportContext context) [0x0007c] in <7b1dc95b0b4841539beb48023c1128d3>:0 
  at System.Net.Http.HttpContent.CopyToAsyncCore (System.Threading.Tasks.ValueTask copyTask) [0x00022] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/HttpContent.cs:361 

So it seems there is parallelization built into blobClient, but it's failing still. Additionally, I cannot use https://www.nuget.org/packages/Microsoft.Azure.Storage.DataMovement because it doesn't work with Xamarin Forms.

Solution

If you want to upload file to Azure blob storage in chunk, please refer to the following code

public async Task upload(Stream stream){
            string connectionString = "";
            string containerName = "upload";
            string blobName = "";
            BlockBlobClient blobClient = new BlockBlobClient(connectionString, containerName, blobName);
           
            List<string> blockList = new List<string>();

                while (true) {
                    byte[] b = new byte[1024 * 1024];
                    var n = await stream.ReadAsync(b, 0, 1024 * 1024);
                    if (n == 0) break;
                    string blockId = Guid.NewGuid().ToString();
                    string base64BlockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockId));
                    await blobClient.StageBlockAsync(base64BlockId, new MemoryStream(b, true));
                    blockList.Add(base64BlockId);
                }

                await blobClient.CommitBlockListAsync(blockList);

   
}

---

Update

If you want to use sas token, please refer to the following code

var uri = new Uri($"https://{storageAccountName}.blob.core.windows.net/{containerName}/{blobName}?{sasToken}");
BlockBlobClient blobClient = new BlockBlobClient(uri);

Answered By - Jim Xu