Swazoo vs. Apache Benchmark

Introduction
Results
Apache
Swazoo on VisualWorks 7.6
Swazoo on Squeak 3.9
Swazoo on Gemstone/S

Introduction

Tool: httperf

Serving 1.) 3KB image 2.) 100KB image

Benchmarking was done on P4 3.20GHz with 2GB memory running SuSE Linux 10.0 64bit, on VisualWorks 7.6, Squeak 3.9 and Gemstone64 2.2.5 Web edition, using Swazoo 2.1, Aida/Web 5.6 and Seaside 2.8 (on Gemstone).

Results


3K picture

3K picture

3k picture

100K picture

100K picture

100K picture

Web server

source

req/s

KBytes/s

ratio

req/s

KBytes/s

ratio

Apache

file, cached

2.000

6.700

1

1.500

156.000

1

Swazoo VW

memory

600

2.000

3

 

 

 

 

file, cached

550

1.900

4

350

36.400

4

 

file, not cached

430

1.500

5

300

31.000

5

Swazoo Squeak

memory

110

370

20

 

 

 

 

file, cached

42

144

46

40

4.100

40

 

file, not cached

4

14

480

1,3

87

1.800

Swazoo Gemstone

memory, no commits

110

360

20

 

 

 

 

memory, commits

52

180

40

 

 

 

 

file, cached, no commits

180

600

10

7

720

240

 

file, not cached

3

10

1600

0,7

70

2.200

Hyper Gemstone 

Seaside-Pier logo

60

198

40

 

 

 

Update 1 13feb08:

  1. VW 100K file, not cached from 20/2M to 300 req/s, 31MBytes/s, 15x improvement! After optimization of serving non cached files by streaming them in blocks about 8K and not byte by byte anymore.
  2. VW 100K file, cached, from 120/12M to 350 req/s, 36MBytes/s, 3x improvement! After optimization of serving cached content as directly to a TCP socket as possible, without intermediary copying.
  3. VW 3K from memory 20% improvement, cached file 17% improvement, non cached 2x improvement.

Update 2 18fe08: Swazoo 2.1 port to Gemstone :

  1. 3K from memory, no commits 2x better, with commits 15% better

Apache

Apache 3KB image: 2000 req/s, 6700 KB/s

rate 2000 req/s, 20000 requests per same session

  • httperf --server bench --port 88 --uri "/eranovalogogif.gif" --rate 2000 --num-conn 20000

  • Connection time ms: min 0.2 avg 0.8 max 46.0 median 0.5 stddev 1.5
  • Request rate: 2000.1 req/s (0.5 ms/req)
  • Reply rate replies/s: min 1999.9 avg 1999.9 max 1999.9 stddev 0.0
  • Net I/O: 6.700 KB/s

rate 2100 req/s, 2000 requests per same session

  • httperf --server bench --port 88 --uri "/eranovalogogif.gif" --rate 2100 --num-conn 21000

  • httperf: connection failed with unexpected error 99
  • Connection time ms: min 0.2 avg 9.3 max 29465.0 median 0.5 stddev 482.9
  • Request rate: 225.6 req/s (4.4 ms/req)
  • Reply rate replies/s: min 0.0 avg 355.2 max 710.3 stddev 502.3
  • Net I/O: 750 KB/s

Apache 100KB image: 1500 req/s, 156.000 KB/s

  • httperf --server bench --port 88 --uri "/french-village.jpg" --rate 1500 --num-conn 15000

  • Connection time ms: min 0.5 avg 12.4 max 137.1 median 3.5 stddev 22.8
  • Request rate: 1492.1 req/s (0.7 ms/req)
  • Reply rate replies/s: min 1480.8 avg 1486.7 max 1492.7 stddev 8.4
  • Net I/O: 155.633 KB/s

Swazoo on VisualWorks 7.6

Swazoo with Aida/Web, 3KB image from memory: 500 req/s, 1700KB/s

rate 500 req/s, 5000 requests per same session

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 500 --num-conn 5000

  • Connection time ms: min 0.3 avg 7.7 max 101.1 median 2.5 stddev 15.5
  • Request rate: 499.5 req/s (2.0 ms/req)
  • Reply rate replies/s: min 492.0 avg 499.2 max 506.4 stddev 10.2
  • Net I/O: 1700 KB/s

rate 550 req/s, 5500 requests per same session

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 550 --num-conn 5500

  • Connection time ms: min 0.4 avg 90.7 max 3108.5 median 72.5 stddev 327.0
  • Request rate: 447.8 req/s (2.2 ms/req)
  • Reply rate replies/s: min 537.6 avg 544.2 max 550.9 stddev 9.4
  • Net I/O: 1520 KB/s

Swazoo with Aida/Web, 3KB image from file, cached: 470 req/s, 1600 KB/s

  • httperf --server localhost --port 8888 --uri "/eranovalogogif.gif?id=451646722" --rate 470 --num-conn 4700

  • Connection time ms: min 0.5 avg 22.0 max 131.2 median 5.5 stddev 32.0
  • Request rate: 466.3 req/s (2.1 ms/req
  • Reply rate replies/s: min 460.8 avg 465.4 max 470.0 stddev 6.5
  • Net I/O: 1600 KB/s

Swazoo with Aida/Web, 3KB image from file, not cached: 200 req/s, 690 KB/s

  • httperf --server localhost --port 8888 --uri "/eranovalogogif.gif?id=451646722" --rate 200 --num-conn 2000

  • Connection time ms: min 0.9 avg 14.9 max 108.1 median 5.5
  • Request rate: 200.0 req/s (5.0 ms/req)
  • Reply rate replies/s: min 199.2 avg 199.9 max 200.6 stddev 1.0
  • Net I/O: 690 KB/s

Swazoo with Aida/Web, 100KB image from file, cached: 120 req/s, 12.500 KB/s

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 120 --num-conn 1200

  • Connection time ms: min 5.1 avg 32.0 max 127.5 median 19.5 stddev 27.2
  • Request rate: 119.7 req/s (8.4 ms/req)
  • Reply rate replies/s: min 117.8 avg 119.5 max 121.2 stddev 2.4
  • Net I/O: 12.500 KB/s

Swazoo with Aida/Web, 100KB image from file, not cached: 20 req/s, 2080 KB/s

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 20 --num-conn 200

  • Connection time ms: min 38.4 avg 46.8 max 145.6 median 43.5 stddev 11.9
  • Request rate: 19.9 req/s (50.2 ms/req)
  • Reply rate replies/s: min 19.8 avg 19.9 max 20.0 stddev 0.1
  • Net I/O: 2081 KB/s
Update 1:

Swazoo with Aida/Web, 100KB image from file, cached: 350 req/s, 36.400 KB/s, 3x improvement

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 350 --num-conn 3500

  • Connection time ms: min 1.8 avg 27.8 max 142.5 median 21.5 stddev 23.4
  • Request rate: 348.8 req/s (2.9 ms/req)
  • Reply rate replies/s: min 348.4 avg 349.4 max 350.4 stddev 1.4
  • Net I/O: 36.407 KB/s

Swazoo with Aida/Web, 100KB image from file, not cached: 300 req/s, 31.000 KB/s, 15x improvement!

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 20 --num-conn 200

  • Connection time ms: min 2.3 avg 30.2 max 192.7 median 6.5 stddev 37.7
  • Request rate: 295.9 req/s (3.4 ms/req)
  • Reply rate replies/s: min 296.0 avg 297.3 max 298.6 stddev 1.8
  • Net I/O: 30891KB/s

Swazoo with Aida/Web, 3KB image from memory: 600 req/s, 2.000KB/s, 20% improvement

rate 500 req/s, 5000 requests per same session

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 600 --num-conn 6000

  • Connection time ms: 0.3 avg 25.2 max 3049.2 median 9.5 stddev 48.8
  • Request rate: 599.8 req/s (1.7 ms/req)
  • Reply rate replies/s: min 597.4 avg 599.7 max 602.0 stddev 3.3
  • Net I/O: 2030 KB/s

Swazoo with Aida/Web, 3KB image from file, cached: 550 req/s, 1900 KB/s, 17% improvement

  • httperf --server localhost --port 8888 --uri "/eranovalogogif.gif?id=451646722" --rate 550 --num-conn 5500

  • Connection time ms: min 0.2 avg 19.3 max 99.7 median 9.5 stddev 24.1
  • Request rate: 550.0 req/s (1.8 ms/req)
  • Reply rate replies/s: min 549.0 avg 549.9 max 550.8 stddev 1.3
  • Net I/O: 1883 KB/s

Swazoo with Aida/Web, 3KB image from file, not cached: 430 req/s, 1.500 KB/s, 2x improvement

  • httperf --server localhost --port 8888 --uri "/eranovalogogif.gif?id=451646722" --rate 200 --num-conn 2000

  • Connection time ms: min 0.4 avg 27.4 max 103.9 median 31.5 stddev 24.2
  • Request rate: 427.9 req/s (2.3 ms/req)
  • Reply rate replies/s:
  • Net I/O: 1465KB/s

Swazoo on Squeak 3.9

Swazoo with Aida/Web, 3KB image from memory: 110 req/s, 370KB/s

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 110 --num-conn 1100

  • Connection time ms: min 5.2 avg 34.2 max 148.2 median 10.5 stddev 36.7
  • Request rate: 110.0 req/s (9.1 ms/req)
  • Reply rate replies/s: min 108.4 avg 109.9 max 111.4 stddev 2.1
  • Net I/O: 370 KB/s

Swazoo with Aida/Web, 3KB image from file, cached: 42 req/s, 144KB/s

  • httperf --server localhost --port 8888 --uri "/eranovalogogif.gif?id=451646722" --rate 42 --num-conn 420

  • Connection time ms: min 17.4 avg 33.6 max 192.3 median 21.5
  • Request rate: 42.0 req/s (23.8 ms/req)
  • Reply rate replies/s: min 42.0 avg 42.0 max 42.0 stddev 0.0
  • Net I/O: 144 KB/s

Swazoo with Aida/Web, 3KB image from file, not cached: 4 req/s, 14KB/s

  • httperf --server localhost --port 8888 --uri "/eranovalogogif.gif?id=451646722" --rate 4 --num-conn 40

  • Connection time ms: min 133.5 avg 150.8 max 181.4 median 150.5 stddev 10.4
  • Request rate: 4.0 req/s (247.3 ms/req)
  • Reply rate replies/s: min 4.0 avg 4.0 max 4.0 stddev 0.0
  • Net I/O: 14 KB/s

Swazoo with Aida/Web, 100KB image from file, cached: 40 req/s, 4.100KB/s

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 40 --num-conn 400

  • Connection time ms: min 15.0 avg 63.7 max 321.7 median 25.5 stddev 67.1
  • Request rate: 39.2 req/s (25.5 ms/req)
  • Reply rate replies/s: min 38.0 avg 39.0 max 40.0 stddev 1.4
  • Net I/O: 4096 KB/s

Swazoo with Aida/Web, 100KB image from file, not cached: 1,3 req/s, 87KB/s

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 4 --num-conn 40

  • Connection time ms: min 1333.1 avg 3822.8 max 5847.7 median 3764.5 stddev 1702.9
  • Request rate: 1.3 req/s (755.4 ms/req)
  • Reply rate replies/s: min 0.4 avg 0.4 max 0.4 stddev 0.0
  • Net I/O: 87 KB/s

Swazoo on Gemstone/S

Hyper on Aida/Web, 3KB image from memory, commited: 45 req/s, 140KB/s

rate 45 req/s, 450 requests per same session

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 45 --num-conn 450

  • Connection time ms: min 14.6 avg 309.9 max 940.9 median 208.5 stddev 304.5
  • Request rate: 41.6 req/s (24.0 ms/req)
  • Reply rate replies/s: min 39.6 avg 41.5 max 43.4 stddev 2.7
  • Net I/O: 140 KB/s

Hyper on Aida/Web, 3KB image from memory, without commits: 58 req/s, 195KB/s

rate 60 req/s, 600 requests per same session

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 60 --num-conn 600

  • Connection time ms: min 11.1 avg 22.5 max 69.4 median 15.5 stddev 13.7
  • Request rate: 57.9 req/s (17.3 ms/req)
  • Reply rate replies/s: min 57.8 avg 57.8 max 57.8 stddev 0.0
  • Net I/O: 195 KB/s

Hyper on Aida/Web, 3KB image from file, not cached: 1 req/s, 3KB/s

rate 1 req/s, 10 requests per same session

  • httperf --server localhost --port 8888 --uri "/eranovalogogif.gif?id=451646722" --rate 1 --num-conn 10

  • Connection time ms: min 1062.6 avg 1427.4 max 1833.1 median 1422.5 stddev 271.6
  • Request rate: 0.9 req/s (1083.4 ms/req)
  • Reply rate replies/s: min 0.8 avg 0.9 max 1.0 stddev 0.1
  • Net I/O: 3 KB/s

Hyper on Seaside, 3KB image from memory (without commits?): 60 req/s, 198KB/s

rate 60 req/s, 600 requests per same session

  • httperf --server localhost --port 8899 --uri "/seaside/files/PRBlueprintLibrary/pier.png" --rate 60 --num-conn 600

  • Connection time ms: min 9.0 avg 17.9 max 69.3 median 13.5 stddev 10.3
  • Request rate: 59.9 req/s (16.7 ms/req)
  • Reply rate replies/s: min 59.6 avg 59.8 max 60.0 stddev 0.3
  • Net I/O: 198 KB/s

Hyper on Aida/Web, 100KB image from file, not cached: 0,4 req/s, 52KB/s

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 1 --num-conn 10

  • Connection time ms: min 5074.1 avg 11892.1 max 16045.0 median 13045.5
  • Request rate: 0.4 req/s (2368.6 ms/req)
  • Reply rate replies/s: min 0.0 avg 0.4 max 1.0 stddev 0.5
  • Net I/O: 52 KB/s

Hyper on Aida/Web, 100KB image from file, cached: 2,3 req/s, 243KB/s

rate 1 req/s, 10 requests per same session

  • httperf --server localhost --port 8888 --uri "/french-village.jpg?id=451646722" --rate 3 --num-conn 30

  • Connection time ms: min 426.6 avg 2196.4 max 4870.8 median 1927.5
  • Request rate: 2.3 req/s (429.1 ms/req)
  • Reply rate replies/s: min 1.4 avg 1.9 max 2.4 stddev 0.7
  • Net I/O: 243 KB/s

Update 2:

Swazoo on Aida/Web, 3KB image from memory, commited: 52 req/s, 180KB/s, 15% improvement

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 52 --num-conn 520

  • Net I/O: 176 KB/s

Swazoo on Aida/Web, 3KB image from memory, without commits: 110 req/s, 360KB/s, 2x improvement

  • httperf --server localhost --port 8888 --uri "/img/eranovalogogif.gif?id=451646722" --rate 110 --num-conn 1100

  • Net I/O: 357 KB/s



Updated: 18.2.2008