Flash/Amfphp data transfer: using strings is fastest

February 22nd, 2007 in Developer Diary · By Dolph aka Zaphod B. Edited by Markus Weichselbaum.

This post compares transfer time and execution speed of several methods of data exchange between the Flash player and amfphp-enabled gateway on the server, written in PHP. Our test results indicate that for larger arrays (with approx. 1000 elements), using strings performs better than transferring array objects or mysql result sets.

Amfphp

For those not familiar with Amfphp, here is a description from the Amfphp website:

Amfphp is an RPC toolkit for PHP. It allows seamless communication between PHP and :

* Flash and Flex with Remoting
* JavaScript and Ajax with JSON
* XML clients with XML-RPC

Essentially, Amfphp converts certain PHP data types to their Actionscript equivalent data types, and where possible, converts Actionscript data types to their PHP equivalents. For example, it can convert a PHP array into an Actionscript Array, or a PHP resource (e.g., a mysql_result result) into an Actionscript Recordset.

Gzip and PHP/Actionscript data type decisions

Since Amfphp 1.9 beta 2, Amfphp has included an option to use gzip compression on the data sent back to the client. This new feature of Amfphp, coupled with suspicions that splitting strings in Actionscript was time consuming, made us rethink our current data exchange method between Flash, Amfphp and PHP, so we thought it was time to experiment some more with the different data types.

Our goal was simple: to find which data types resulted in Actionscript having the data available and ready to process in the least amount of time.

In our specific case, we use Ampfphp to get information about each TheBroth mosaic from the server to the Flash player on the client side. We needed to determine which of the available methods takes the least amount of time to have the data ready to pass to our tile generating function createTile on the client side.

For our experiments, the $gateway->enableGzipCompression(25*1024); line in Amfphp’s gateway.php setting was changed to $gateway->enableGzipCompression(5*1024);. This was because the uncompressed content length of our testing string result was less than the default Amfphp gzip threshold of 25KB (it was about 21KB, to be exact).

Mysql table structure

Here’s the Mysql table create code for the test data we used in this experiment. Each loading will select 1000 rows from the database, eg. SLOTSCOUNT=1000.

CREATE TABLE `thebroth_experiment` (
  `set_id` int(11) NOT NULL default '0',
  `slot_number` smallint(6) NOT NULL default '0',
  `x` smallint(6) default '0',
  `y` smallint(6) default '0',
  `move_number` int(11) unsigned NOT NULL default '0',
  `color` char(6) NOT NULL default '',
  `rotation` smallint(6) NOT NULL default '0',
  PRIMARY KEY  (`set_id`,`slot_number`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1

In the context of this experiment, the sql queries only selected slot_number, x, y, color and rotation. The server in this experiment was a Dual AMD Opteron 246 2.0 GHz with 2 GB RAM and little load. It runs Mysql 5.0.22 and Php version 4.3.9. This PHP version was *not* recompiled with amfphp.

PHP/Actionscript data type options

PHP string to Actionscript String

Approach: Create a comma separated PHP string on the server side, and let the client side Actionscript split the Actionscript String into an Actionscript Array in order to loop through the data.

Server side code:

function getTilesAsString() {
	$ret = mysql_query($sql);

	while ($t = mysql_fetch_assoc($ret)) {
		$s .= "$t[n],$t[x],$t[y],$t[c],$t[r]\n"; // where n,x,y,c,r were the attributes whose values we wanted
	}

	return trim($s);
}

Client side code:

function handleDataString(re:ResultEvent) {
	lines = re.result.split("\n");
	for(var i=0; i < SLOTSCOUNT; i++) {
		var tile = lines[i].split(",");
		//createTile(Number(tile[0]), Number(tile[1]), Number(tile[2]), tile[3], Number(tile[4]));
	}
}

PHP array to Actionscript Array

Approach: Create a PHP array on the server side, and let the Actionscript loop through the data as an Actionscript Array.

Server side code:


function getTilesAsArray() {
	$ret = mysql_query($sql);

	while ($t = mysql_fetch_assoc($ret)) {
		$r[] = $t;
	}

	return $r;
}

Client side code:

function handleDataArray(re:ResultEvent) {
	for(var i=0; i < SLOTSCOUNT; i++) {
		var tile = re.result[i];
		//createTile(Number(tile['n']), Number(tile['x']), Number(tile['y']), tile['c'], Number(tile['r']));
	}
}

PHP resource to Actionscript Recordset

Approach: Create a PHP Mysql Resource on the server side, and let the Actionscript loop through the data as an Actionscript RecordSet.

Server side code:

function getTilesAsMysqlResult() {
	return mysql_query($sql)
}

Client side code:

function handleDataMysql(re:ResultEvent) {
	var rs:RecordSet = RecordSet(re.result);
	for(var i=0; i < SLOTSCOUNT; i++) {
		var tile = rs.getItemAt(i);
		//createTile(Number(tile['n']), Number(tile['x']), Number(tile['y']), tile['c'], Number(tile['r']));
	}
}

Results

The following tables show the values that varied depending on the data types used:

Server side

Data type Uncompressed size (bytes) Amfphp encode time (ms) 1 PHP result creation (ms) total (ms) Content-length (bytes)
array 49584 368 (sd 10.2) 8.75 (sd 2.8) 376.75 12940
mysql 50312 125.25 (sd 1.9) n/a 125.25 14099
string 20581 2.5 (sd 0.6) 11 (sd 0) 13.5 10420

1 Measured using Amfphp’s service browser.

Notes:

‘Amfphp encode time’ : the time taken by Amfphp to convert the PHP variable into the equivalent Actionscript variable.
‘PHP result creation’ : the time taken by PHP to convert the Mysql result resource into the PHP variable to send back to Amfphp. In the server side code examples above, it’s the while loop.
‘total’: the sum of the above 2 times.
‘Content-length’: the number of bytes used for the response that Amfphp created for the client.

Client side

‘client function’ is the time it took to run the Actionscript function for the particular data type. For testing, the calls to createTile were commented out (they remain in the code to show that the data are in a state ready to be used in such a way).

Machines used

  1. Laptop: P3 895 MHz 256MB RAM WIN XP FF 1.509 Shockwave Flash 9.0 r28
  2. Desktop: AMD Athlon XP 2500+ 512MB RAM Firefox/2.0 (Ubuntu-edgy) Shockwave Flash 9.0 r31
  3. Mac: Imac 1.8 GHz PPC G5 512MB RAM Firefox/2.0 OSX 10.4.8 Shockwave Flash 9.0 r28
execution time of client function (ms)
Data type Laptop Desktop Mac
array 20 (sd 0) 8.5 (sd 0.58) 15 (sd 0)
mysql 50.25 (sd 0.5) 21.25 (sd 0.5) 39.75 (sd 11.84)
string 67.75 (sd 5.19) 26.75 (sd 0.5) 58.25 (sd 3.3)

As you can see from the results, the extra time taken to split the string into an array on the client side (47.75 ms longer for string splitting than using the array method, on the slow laptop) is still a lot less than the extra server side processing time required for the non-string types (363.25 ms for the PHP array, 111.75 ms for the PHP resource). The string response is also the smallest download of the three methods.

It is also important to note that each client will cause the extra time processing time on the server, so moving the processing effort and time to the client is a good idea when you wish to reduce server load.

Summary

Based on our observations, we recommend the following:

Unless your clients’ processing power is extremely limited (in which case the client side string to array processing can take significantly longer), transfer the data as a comma separated string.

Disclaimer: For smaller or much larger arrays or other datatypes the above results may not apply.

Share this:
  • del.icio.us
  • StumbleUpon
  • Furl
  • Netscape
  • Reddit
  • Technorati
  • YahooMyWeb

4 Responses to “Flash/Amfphp data transfer: using strings is fastest”

Patrick Mineault22 Feb 07

Could you post the SQL required to recreate the database table? The amount of time taken to encode will be different depending on the column datatypes you are using. Also, as for the amount of data being sent, could you report the uncompressed data size versus the compressed data size (you can find this out in Charles)? Are you doing your tests over AMF0 or over AMF3? Are you using the C extension?

Markus23 Feb 07

Wow, Patrick, you were quick… I hadn’t even finished editing this post. :-)

I have updated the post and included the information you requested. Oh, and we’re using AMF0.

pmineault23 Feb 07

All right, so let me interpret those results now that I have all the info.

The amount of time spent in the serializer is proportional to the amount of work that amfphp has to do to figure out how to serialize things. When you send it a string, it only has to figure out its length and send it has is, so it’s very fast. When you send it an array, then it has to figure out all the types of all the elements in the array. When you send it a recordset, that is, the result of mysql_query, it can figure out the types of each of the columns beforehand, so it doesn’t have to do as much work when it goes through each element. So serializing an array is slower than serializing a recordset, and that is slower than serializing a string. If you used the C extension, then the work is done at a low level, so there really wouldn’t be much of a difference between these three cases.

As for the difference in file size, that is also expected. For every element in the arrays, amf includes a type byte, and then the data itself. If the data is a string, then there are two extra bytes for the string length. So encoding “asdasd” versus encoding “asd” and “asd” will result in 9 bytes (string byte + length bytes + data) versus 12 bytes (string byte + length bytes + data + string byte + length bytes + data). Now you may be wondering why the mysql recordset ends up being bigger when encoded than the array. In the array case, you are encoding strings, because that is what is given by mysql_fetch_*. In the recordset case, amfphp looks at the column types, and if it sees it’s an int or float type will encode the data as a double.

Now a double in AMF0 takes up 9 bytes, while a string takes up 3 bytes plus the string length. Thus if you were to encode the number “128″ as a string, it would take 6 bytes, and 9 bytes as a double. AMF3 solves this issue by using Huffman coding on integer types, so if you were to use AMF3 instead the recordset case would result in lower sent size than the string case.

As for the use of gzip, I’d say that in your case it probably won’t make much of a difference. gzip helps a lot when using string types, but not much when using number or int types. Those sizes you quoted above are probably the sizes after decompression; you should take a look at the compressed size through Charles to figure out how much gzip really helps. Remember that gzip’d data has to be decompressed by the browser and compressed by php, so it really helps when you have more data, but with less data the extra overhead might slow it down. This is why I chose 25 KBs as the threshold for enabling gzip compression.

Bottom line: your data is all int and floats, so if you’re not going to use the C extension, you’re probably going to find appreciably better speed by sending the data as a custom-serialized string. AMF was designed to handle all types of data and always have “pretty decent size”, but of course, if you can exploit properties of the sent data to figure out an optimal way of encoding it, you can exploit it. As a rule of thumb, I don’t recommend that people start doing their own serializing for most practical purposes, but given the very special data set you’re using, it might be worth it.

MorningStar23 Feb 07

Patrick, gzip does make a difference in our case. As you can see in the table above, the compressed content length for the string is 10420 bytes, the smallest of all methods.

There is no table summarising the total times taken (I shall add one, actually) but even on a fast connection (8000 kbs) where the datalength saving of 50% shouldn’t matter much, sending strings simply is the quickest, going by the time from sending the request to having the data ready in the client. Obviously, slower connections will benefit even more from the significant compression that we see in our case.

From past (amfphp-unrelated) experiments with cutoff values for gzip compression I would suggest to change the threshold to a much lower value, such as 5Kb, but then again, this really does depend a lot on the datatype. In our case, our string is comma separated values and presumably this alone provides for a lot of redundancy that is very compressible.

I would certainly agree with your rule of thumb. In our case, we needed to find ways to get the data to the client as quickly as possible to reduce the overall delay for the flash application to start, so the experiment was certainly worth it. I suppose anyone needing the extra speed while keeping an eye on CPU load will probably conduct similar experiments to determine what works best for their case.

Our data are mostly integers indeed, and we’ll definitely experiment some more with AMF3 and the C extension, should we be able to recompile the PHP. As you have pointed out in your blog, this is somewhat of a hurdle in many situations.

I’d be definitely interested to see whether Huffman encoding is more effective on our data than gzip or if it can even be combined effectively.

Once again, big thumbs up for Amfphp - it’s great!

Leave a comment