libReplay | Documentation

libReplay Testing Utilities

libReplay is a collection of static PHP functions that convert raw packets to HTTP requests and perform various web server testing tasks.

Testing Workflow

Raw network packets are captured and stored in PCAP format as the users' communication with the web server is intercepted. The PCAP file is then converted into a PHP object that contains HTTP requests as well as other meta information such as the wait time between each request. This PHP object can be then passed on to other functions for benchmarking and subsequently reporting. Alternatively, it can be used to generate testing scripts for more sophisticated testing tools, such as Silk Performer.

Sample Use

The following codes are copied from sift.php in the libReplay package to examplify the use of libReplay. A raw packet file, pmademo.pcap is captured by WireShark with the following filter:

ip.dst == 77.78.110.71

The test script signs in a live version of phpMyAdmin on demo.phpmyadmin.net and performs some simple tasks. The top 5 bandwidth consuming and longest waited pages are listed in the final output, together with a test script that can be used in SilkPerformer.

include 'libreplay.php';
$req=parsepcap('pmademo.pcap');

$res=benchmark($req,3);

echo "\r\nTop Bandwidth Consumers\r\n=======\r\n";
report_rank($res,'totalbytes',5);

echo "\r\nSlowest Pages\r\n=======\r\n";
report_rank($res,'averagewaittime',5);

echo "\r\n//SilkPerformer Script\r\n";
render_silk($req);

Functions

$http_requests parsepcap($filename)

This function reads in the PCAP file, specified by the filename, and extracts raw packets. Each packet is then converted to an HTTP request object by the parsetcp function.
If one HTTP request is sent in multiple TCP segments, this function also reassembles them.
An array of HTTP request objects is returned.

It is important to filter by the destination host IP when the PCAP file is exported. Otherwise, requests to other servers are also tested.

(private) $http_packet checktcp($raw_packet)

This function is called internally by parsepcap. It converts raw packets to HTTP packets by stripping off the ethernet-, IP- and TCP headers.
An empty string is returned if the packet is not a TCP packet.
The returned object contains a method field that is used to assemble segmented TCP packets:

HTTP packet(
'method' => GET, POST or ASM
'data' => Body of HTTP request in the segment
)

(private) $http_request parsetcp($http_packet,$time_diff)

This function is called internally by parsepcap. It takes the HTTP packet data (including Ethernet frame header) and the timestamp difference from the previous packet as input parameters, and returns an HTTP request object of the following format:

HTTP request(
'method' => GET or POST
'uri' => Request URI
'postdata' => POST fields if applicable
'contenttype' => from the Content-type header
'diff' => wait time since the last packet/request
'referer' => HTTP Referer
'agent' => browser user agent
)

$performance sendrequest($http_request)

As its name suggests, this function sends an HTTP request as described in the HTTP request object.
This is where the name libReplay is from.

If more fields are needed for more realistic emulation, the parsetcp function needs to be modified.

Response time and content length are returned in the following structure:

performance (
'waittime' => response time, including latency
'bytes' => content length, including server headers
'reqsize' => request size, roughly calculated by uri+cookie+post_data
)

(private) string parsecookies($cookie_filename)

A temporary file replaycookie is generated when client-side cookies are issued in the server response header. To achieve the maximum compatibility, this function forcifully resends all the cookies that are described in the cookie file. This means that cookies that fail tail matching tests are also sent to the server.

$benchmark_results benchmark($http_requests,$iterations)

This function calls sendrequest on each request in the $http_requests list.

The $iterations parameter specifies how many times all the requests are repeated. It is important to understand that the iteration count is not meant to emulate concurrent users. Repeated reading of bandwidth consumption and server response time increases the accuracy of tests.

Each item in the returned array has the following structure:

HTTP request URI => (
'count' => # of times being requested
'totalbytes' => total size of returned content, including all server headers
'totalwaittime' => total response time, including connection time and latency
'totalreqsize' => total request size
'averagebytes' =>
'averagewaittime' =>
'averagereqsize' =>
)

direct_output report_rank($benchmark_results,$sort_field,[$limit])

This function sorts the benchmark results by the specified field, e.g. average wait time, and outputs the top results in desending order. The purpose of this function is to identify the most resource consuming scripts. If the limit parameter is not spefied, all the results are included in the output.

The following is a sample output of the top 5 bandwidth consumers:

10398	http://192.168.1.100/ika/imgs/spartan.jpg
4052	http://192.168.1.100/ika/viewport.js
3857	http://192.168.1.100/ika/tabs.js
3329	http://192.168.1.100/ika/index.php
3210	http://192.168.1.100/ika/imgs/powered.png

direct_output render_silk($http_requests)

This function outputs a SilkPerformer testing script.
SilkPerformer has a recording utility that captures most of the requests from a handful of browsers. libReplay extends SilkPerformer's recorder, or "agent" by capturing ALL the requests from ANY browser or client.

Add the following line to the beginning of the script:

use "WebAPI.bdh"