| CS290I - Scalable Internet Services and Systems | |
|
Thorsten von Eicken - UCSB - Spring 2001 |
|
The results below were measured while the machines and database were used by others, so the numbers are preliminary... Also note that a virtual day of trading lasts for 86 seconds, so the longer runs include the day-end overhead while the shorter ones don't...
| Group | Consistency | Single server | Three servers | ||||||
| Who | Port | doreq.pl | Host | q0-100-1 | q4-200-1 | q4-200-5 | q4-200-15 | q4-2000-15 | fail test |
| vishal, sxs | 8018 | OK | lotus | 11 Req/sec | 4 Req/sec | 44 Req/sec (internal error in order id generation) | times
out |
not tested | |
| sverma, madhavan | 8016 | - trade #21 incorrectly accepted, should return ERROR | lotus | 16 Req/sec | 43 Req/sec | 75-80 Req/sec | 102 Req/sec | 115 Req/sec (deadlock at end-of-day processing) | passed |
| ynshenoy, vivek | 8025 | - trade #21 incorrectly accepted, should return ERROR | lotus | 13 Req/sec | 39 Req/sec | 97 Req/sec | 120 Req/sec (DB errors) | 115 Req/sec (DB errors) | implemented, but problems (DB & fake time) |
| vveytser, esben, ljmiller | 8008 | OK | bentley | 18 Req/sec | 21 Req/sec | 30 Req/sec | |||
| yjpark, suny | 8024 | OK | bentley | 19 Req/sec | 46 Req/sec | 188 Req/sec | 140 Req/sec | 410 Req/sec (seems to violate stable storage guarantees) | passed, but not handling server recovery |
| ljosa, ranjan | 8009 | OK | lotus | 22 Req/sec | 19 Req/sec | 44 Req/sec | |||
| jyzhou, cdxiong | 8002 | OK | lotus | 21 Req/sec | 66 Req/sec | 116 Req/sec | 144 Req/sec | 116 Req/sec | passed! |
| longjiao, yingf | 8023 |
OK | daimler | 14 Req/sec | 31 Req/sec | 60 Req/sec | (can't insert into database ERROR) | ||
| srsiva, phanse | 8014 |
OK | daimler | 14 Req/sec | 3 Req/sec | Various errors (HTML stuff, java exceptions, ...) | |||
| ashish, hugh1su | 8001 | OK | daimler | 16 Req/sec | 32 Req/sec | 50 Req/sec | 57 Req/sec | some problems | passed! |
| zhp, kuschner | 8026 |
OK | lotus | 32 Req/sec | 6.6 Req/sec | 16 Req/sec | 78
Req/sec (w/errors) |
||
Please send us both email if you have questions about the tests or results.
For doreq.pl see the description
qX-YYYY-C refers to
the traces in ~blanquer/RESULTS/trace--r-qX-YYYY run with C
concurrent requests, e.g.:
client -h lotus -p 8000 -i trace-r-q0-100.init -f
trace-r-q0-100.ordr -n 1 -o output
where X is the number of quote requests
per trade requests, and YYYY is the total number of trade requests
The output from the load generator for each server can be found at /fs/psl2/blanquer/RESULTS/"your port number" for each of the above tests. Inside each directory is a file for each of the tests tried, the file name ends with the test type (10.1 or 100.1 or 100.10). These files have a line per response received with the format "requestNumber Result message" where requestNumber is the number of the request to which a response was received (the numbering corresponds to the request ordering of the *.ordr files) and 0 corresponds to the result of the robots/init request (*.init file, see below), and the result is:
The initial post data for the /robots/init request and the list of requests used for the tests can be also found at /fs/psl2/blanquer/RESULTS/ where trace-r-q0-10.init has the accounts created for the /robots/init and trace-r-q0-10.ordr the trades performed in the case of the 10 requests test. trace-r-q0-100.init and trace-r-q0-100.ordr are for the tests with 100 requests.
You can run the test yourself
using the client at /fs/psl2/blanquer/RESULTS/client, for example:
client -h lotus -p 8000 -i trace-r-q0-10.init -f
trace-r-q0-10.ordr -n 1 -o output
Note that n is limited to a maximum of 10, and if an invalid file is
given for the init, no post request will be sent (just the orders, which can be
handy).