Optimizing data transfer performance

If you recall from Chapter 5, Building Integrated Services, you've built a facility to transfer an account from one device to another via Infrared and Bluetooth. In these cases, you are transferring the dataset as is without any sort of compression. This can lead to slow transfer, especially for larger datasets or larger numbers of records.

Microsoft offers data compression functionality via the System.io.compression namespace. The GZipstream class allows you to compress a byte array based on the GZIP standard.

The GZipStream class cannot be used to compress I files larger than 4 GB in size. I

After compressing data, the GZipstream class does not store the original size of the uncompressed data. You must, therefore, implement your own method to "remember" and store this original size somewhere so that it can be used later during decompression. A common approach is to store this size as the first four bytes of the compressed stream.

Let's take a look at the following code to do this. You can also print the size of the data before and after compression to see the performance difference with and without using GZIP compression. Make the following changes to your SerializeDataset function (highlighted in bold):

public static byte[] SerializeDataset(DataSet Data) {

StringWriter strWriter = new StringWriter(); string _result;

ASCIIEncoding _encoding = new ASCIIEncoding(); Data.WriteXml(strWriter);

_result = strWriter.GetStringBuilder().ToString(); byte[] _buffer = _encoding.GetBytes(_result);

MessageBox.Show("Buffer length before compression :" + _buffer.Length.ToString());

//We create a GZipStream object or compression MemoryStream _stream = new MemoryStream(); GZipStream compressedzipStream = new GZipStream(_stream, CompressionMode.Compress, true);

//We calculate the uncompressed data length and write it as //the first four bytes of the stream. We will need this //length during decompression int _totalLength = _buffer.Length; compressedzipStream.Write(BitConverter.GetBytes (_totalLength), 0, 4);

//We now write the remaining data to the stream compressedzipStream.Write(_buffer, 0, _totalLength); compressedzipStream.Close(); _buffer = _stream.ToArray();

MessageBox.Show("Buffer length after compression :" + _buffer.Length.ToString());

return _buffer;

Now, let's take a look at the opposite equivalent. The changes you need to make to the DeserializeDataset method are highlighted in bold in the following code snippet:

public static DataSet DeserializeDataset(byte[] Data) {

//Use the GZipStream class to decompress the incoming data MemoryStream _stream = new MemoryStream(Data); GZipStream compressedzipStream = new GZipStream(_stream,


//Read the first four bytes from the stream - this is the //size (in bytes) of the rest of the stream byte[] _totalLength = new byte[4]; compressedzipStream.Read(_totalLength, 0, 4); int _dataSize = BitConverter.ToInt3 2(_totalLength, 0);

//Create a byte array that has the exact size of the //decompressed data, and read from the GZipStream object byte[] _data = new byte[_dataSize]; compressedzipStream.Read(_data, 0, _dataSize); compressedzipStream.CloseO;

//Convert the decompressed stream into a Dataset ASCIIEncoding _encoding = new ASCIIEncoding(); string _stringData;

_stringData = _encoding.GetString(_data, 0, _data.Length); StringReader _reader = new StringReader(_stringData); DataSet _ds = new DataSet(); _ds.ReadXml(_reader); return _ds;

As it is probably tedious to test using two mobile devices running Bluetooth at the same time, you can create a simple form to see the effect of GZIP compression. Create a new form with any name, place a button on this form, and write the following code in the click event of the button:

private void button1_Click(object sender, EventArgs e) {

DataSet _testSet= new DataSet();

_testSet.Tables.Add (new DataTable("TestTable")); _testSet.Tables[0].Columns.Add (new DataColumn ("TestCol", System.Type.GetType ("System.String")));

//Generate 1000 dummy records for (int i = 0; i <= 1000; i++) {

DataRow myrow = _testSet.Tables[0].NewRow(); myrow["TestCol"] = "Test"; _testSet.Tables[0].Rows.Add(myrow);

//Serialize and compress the dataset byte[] _testArray= SerializeDataset (_testSet);

//Decompress and deserialize the dataset _testSet= DeserializeDataset (_testArray);

//Ensure the dataset has been decompressed correctly - show //the number of records in the decompressed dataset MessageBox.Show("Total Records: " +

_testSet.Tables[0].Rows.Count.ToString ());

If you run this form and click on the button, you will be greeted with this pop-up message box. You can see that the serialized dataset before compression is about 54,081 bytes in size.

Buffer length before compression :54081

After compression you will notice that the length of the data is now 736 bytes. This is about a mere 1.4 percent of the original size of the data!

Buffer length after compression :736

To compress or not to compress

GZIP compression works exceptionally well with text-based data. You will get better compression rates with larger text data. In fact, for smaller bits of text data (for example, 500 bytes or less), the compression may even yield sizes larger than the original data itself. You should implement routines in your code to check the size of your input data before deciding whether to run your data through compression or not.

Was this article helpful?

0 0

Post a comment