CS290I - Scalable Internet Services and Systems
Thorsten von Eicken - UCSB - Winter 2002
 

Project 3 HowTo - Testing the Auctions Site

We have written a perl script to test the auctions site. It runs a request trace through the server, collects the results, and checks the correctness of the responses. The checks are not 100% bullet proof and we're improving them little by little. You can run the test script on your own to debug your server.

Warning: the script is still a bit incomplete in its verification of responses, in particular, it doesn't handle bids at bidPrice well and it doesn't recognize when auctions are closed. So some of the errors it prints are in fact not errors.

Download

issue.pl usage

issue.pl [options...]
    -f <file prefix>       file prefix for .init and .trace file, e.g. "-f 500" uses 500.init and 500.trace
    -h <host> host name of server, e.g. "-h daimler.cs.ucsb.edu"
    -p <port> port number of server, e.g. "-p 8056"
    -v verbose output: prints complete requests issued and responses received, otherwise only prints first 35 chars of each

issue.pl non-verbose output

Issue.pl first prints the init request and the corresponding response. Then follows on line per request/response with the first 35 characters of the request and the response, for example: 
/robots/auction?10000                    -> 10000, auction_0, acct07, 1720.0, 8
/robots/user?acct11                      -> acct11, Account11, 1114, 0\n 
/robots/newBid?qty=6&price=930&bidderId= -> 2317\n 
/robots/user?acct09                      -> acct09, Account09, 1094, 0\n 
/robots/user?acct01                      -> acct01, Account01, 1014, 0\n 
/robots/newBid?qty=2&price=940&bidderId= -> 2318\n 
/robots/newAuction?auctionid=10001&name= -> 10001, 2002-03-12 11:05:30, 2002-03

If a response is incorrect, an error message is printed with the complete request and response, for example:

---------------------------------------------------------------------------
**** ERROR: Request accepted incorrectly
**** REQUEST: http://daimler.cs.ucsb.edu:8015/robots/newBid?qty=8&price=870&bidderId=acct20&auctionId=10000
**** RESPONSE: OK\n2312\n
---------------------------------------------------------------------------

In this case the problem is that the bidderId (acct20) is not a valid user id.

At the end of the run, issue.pl prints the number of requests issues, the elapsed time, and the requests/second processed. This performance metric is just a rough indication of speed given that the perl script is not optimized for performance itself.

issue.pl verbose output

The verbose form of the output (using the -v flag) prints the entire request and reponse. It's a lot harder to read. For example: 

/robots/user?acct15 -> OK: acct15, Account15, 1154, 0\n
/robots/auction?10001 -> ERR: auction not found
/robots/newBid?qty=5&price=900&bidderId=acct19&auctionId=10000 -> OK: 2400\n
/robots/newBid?qty=4&price=910&bidderId=acct19&auctionId=10000 -> OK: 2401\n
/robots/newBid?qty=4&price=920&bidderId=acct19&auctionId=10000 -> OK: 2402\n
/robots/auction?10000 -> OK: 10000, auction_0, acct07, 1720.0, 8, 860.0, 10.0,
2002-03-12 11:08:09, 2002-03-12 11:10:09, LA suburb #0, http://daimler.cs.ucsb
.edu:8015/graphics/spacer.gifactive, 0, 0\n2402, acct19, 920.0, 4, 2002-03-12 
11:08:11\n2399, acct14, 890.0, 8, 2002-03-12 11:08:10\n2398, acct17, 880.0, 2,
 2002-03-12 11:08:10\n2397, acct20, 870.0, 8, 2002-03-12 11:08:10\n\n\n

Incorrect responses are flagged in the same way as in thee non-verbose output.