WebRequest and Binary Data

Someone sent me an email today describing a problem when downloading a binary file with the WebRequest class. There are plenty of articles (including some of mine) with screen scraping code like the following:

WebRequest request = WebRequest.Create(someUrl);        
using(WebResponse response = request.GetResponse())
{
   using(StreamReader reader = new StreamReader(response.GetResponseStream()))
   {
      string result = reader.ReadToEnd();
   }         
}

Unfortunately, StreamReader is only good for reading text. When it comes to binary data the result has a good chance of being incomplete. The approach for binary data is to stick to the basic Stream type and read raw bytes.

byte[] result;
byte[] buffer = new byte[4096];
 
WebRequest wr = WebRequest.Create(someUrl);
 
using(WebResponse response = wr.GetResponse())
{
   using(Stream responseStream = response.GetResponseStream())
   {
      using(MemoryStream memoryStream = new MemoryStream())
      {
         int count = 0;
         do
         {
            count = responseStream.Read(buffer, 0, buffer.Length);
            memoryStream.Write(buffer, 0, count);
 
         } while(count != 0);
 
         result = memoryStream.ToArray();
 
      }
   }
}

P.S. IDisposable lurks everywhere!. It’s a shame some classes use an explicit interface implementation and hide the Dispose method from Intellisense.

P.P.S. Commercials are the best things going on Monday Night Football these days. Except the commercials for other ABC shows. I don't know why I turn on television.

Print | posted @ Tuesday, October 05, 2004 6:30 AM

Comments on this entry:

Gravatar # re: WebRequest and Binary Data
by Andy at 10/6/2004 4:41 AM

You know I've been working with Java for the last few days on a wireless http connection that downloads images and it looks almost identical to what you have above. The class names are a bit different but d@mn, I had no idea they were so syntactically close to each other.
<br>
<br>In Java I have to close the connections in a try finally block. Do you need to also close the C# ones or do they close automatically when they pass from scope?
  
Gravatar # re: WebRequest and Binary Data
by Scott Allen at 10/7/2004 12:04 AM

Yes - Java and C# are really close! The 'using' clause sets up a try catch finally block in C#. It's just a shortcut to use with a resource that has to be closed or disposed of even when an exception happens. If I had written it that way I bet it would have looked exactly like Java, eh?
  
Gravatar # re: WebRequest and Binary Data
by Ren at 2/9/2005 11:33 PM

Downloading - that's easy. Now, try to *upload* files with just webrequest :P I've been 100% unable to find ANYthing remotely relating to that.
  
Gravatar # re: WebRequest and Binary Data
by Scott at 2/10/2005 5:31 AM

Interesting - I'll have to give it a try.
  
Gravatar # re: WebRequest and Binary Data
by Joe Ehrenfeld at 4/6/2005 1:16 AM

I am using POST to get some binary data and when I try to write the data to a file I get this error:
<br>
<br>Additional information: Non-negative number required.
<br>
<br>What am I doing worng
  
Gravatar # re: WebRequest and Binary Data
by Scott at 4/6/2005 3:54 PM

Hi Joe:
<br>
<br>It's hard to tell without seeing some code. Feel free to send me an email and I can take a look.
  
Gravatar # re: WebRequest and Binary Data
by Joe Ehrenfeld at 4/6/2005 8:04 PM

I sent you an email with some code, but I don't know if I used the right address? Did you get an email from joseph.ehrenfeld?
  
Gravatar # re: WebRequest and Binary Data
by Scott at 4/6/2005 8:15 PM

No - you can use scott @ OdeToCode .com
  
Gravatar # re: WebRequest and Binary Data
by Joe Ehrenfeld at 4/6/2005 9:19 PM

Scott, I did send it to that address but our firewall may have blocked it. I'll paste the code here:
<br> //Joe Code
<br> ASCIIEncoding encodedData=new ASCIIEncoding();
<br> byte[] byteArray = encodedData.GetBytes(postData);
<br> Stream stream;
<br>
<br> byte[] result;
<br> byte[] buffer = new byte[4096];
<br>
<br> HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(url);
<br> wr.Method = &quot;POST&quot;;
<br> wr.ContentType = &quot;application/x-www-form-urlencoded&quot;;
<br> wr.ContentLength = byteArray.Length;
<br>
<br> stream = wr.GetRequestStream();
<br> stream.Write(byteArray, 0, byteArray.Length);
<br>
<br> stream.Flush();
<br> stream.Close();
<br>
<br> using(WebResponse response = wr.GetResponse())
<br> {
<br> Console.WriteLine(&quot;Headers: &quot; + response.Headers.ToString());
<br>
<br> using(Stream responseStream = response.GetResponseStream())
<br> {
<br> using(MemoryStream memoryStream = new MemoryStream())
<br> {
<br> int count = 0;
<br> do
<br> {
<br> count = responseStream.Read(buffer, 0, buffer.Length);
<br> memoryStream.Write(buffer, 0, count);
<br>
<br> } while(count != 0);
<br>
<br> result = memoryStream.ToArray();
<br> }
<br> }
<br> }
<br>
<br> //Console.ReadLine();
<br> }
  
Gravatar # re: WebRequest and Binary Data
by Scott at 4/6/2005 10:06 PM

Seems odd, the code looks ok. Do you know what line the exception is thrown?
  
Gravatar # re: WebRequest and Binary Data
by Joe Ehrenfeld at 4/6/2005 10:10 PM

yes, it throws an exception at the line:
<br>
<br>while(count != 0);
<br>
<br>What's odd is I know I have a response, but for some reason I just can't open that attachment and read it to save it onto my server.
  
Gravatar # re: WebRequest and Binary Data
by Swee at 5/20/2005 10:21 PM

I believe the problem lies in these two lines:
<br>
<br>count = responseStream.Read(buffer, 0, buffer.Length);
<br>memoryStream.Write(buffer, 0, count);
<br>
<br>Read() returns -1 when it has no more data. Trying to subsequently do a Write() with count of -1 most likely caused the problem.
<br>
<br>/Swee
  
Gravatar # re: WebRequest and Binary Data
by geoff at 3/8/2006 9:16 PM

Thanks for the code. Is there anyway to tell through C# if the file the user wants to download is text or binary? I've got a shopping cart that allows users to purchase downloadable items but I'm stuck on the page that actually delivers them. I don't want to give a direct url to the file for security, so I figured I'd stream the data, but I'm stuck on how to figure out the encoding of the original file. Any ideas? Thanks.
  
Gravatar # Need Help Please!
by Marc at 5/17/2006 8:24 PM

I have an ASPX page with the following code. It's a variation of the above. The goal is to get a PDF from a URL and display in in Acrobat. This works in Firefox, but in IE the actual contects (binary) of the PDF display in the browser, they don't launch Acrobat.

Any Ideas would be appreciated!
Marc
Lampcov@aim.com

<%@ Import Namespace="System.Net" %>
<%@ Import Namespace="System.IO" %>

<script language="c#" runat="server" debug="true">

private void Page_Load(object sender, System.EventArgs e)
{
byte[] result;
byte[] buffer = new byte[4096];

WebRequest wr = WebRequest.Create("localhost:16200/?MRN=123&doc_type=pdf");
WebResponse response = wr.GetResponse();
Stream responseStream = response.GetResponseStream();
MemoryStream memoryStream = new MemoryStream();

Response.AddHeader("content-disposition", "filename=report.PDF");
Response.ContentType = "application/pdf";

int count = 0;
do
{
count = responseStream.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, count);
Response.BinaryWrite(buffer);
//Response.OutputStream.Write(buffer,0,count);
}
while (count != 0);
result = memoryStream.ToArray();
}

</script>
  

Your comment:

Title:
Name:
Email:
Website:
 
Italic Underline Blockquote Hyperlink
 
 
Please add 1 and 2 and type the answer here: