CDN performance in China

08 Jun 2011

We recently had a project that required CDN-hosted files and, in order to prove a point, I ran a series of traceroute/ping tests on popular CDNs. Seeing as I’ve already formatted the data and made the graphs I thought I’d post it here in case it’s of use to anyone.

The tests were run with mtr for 10 minutes each over the course of an evening on a typical China Unicom home ADSL connection in Xi’an.

The common CDNs don’t have edge nodes in mainland China, and suffer as a result. You can see an obvious “step” in the graphs when the route goes international.

Microsoft Ajax CDN

I get routed to a node in Hong Kong so the response is good but intermittent delays and a little packet loss and the average suffers.

SVG chart of Microsoft Ajax CDN response times

HOST: deck                        Loss%   Snt   Last   Avg  Best  Wrst StDev
   1.|-- 111.194.0.1                0.0%    60    6.5  18.4   5.7 155.7  27.6
   2.|-- 61.148.149.241             0.0%    60   10.6   9.8   3.2  70.4  11.0
   3.|-- 61.148.155.5               1.7%    60   11.3  15.0   8.1 110.8  18.3
   4.|-- 123.126.0.85               0.0%    60   11.4  20.1   8.5 177.8  26.9
   5.|-- 219.158.101.230            0.0%    60   43.5  49.1  43.1 123.7  14.0
   6.|-- 219.158.3.62               0.0%    60  112.2 115.4  48.6 273.4  32.5
   7.|-- 219.158.96.198             1.7%    60  128.1 117.1  49.2 241.0  30.3
   8.|-- 219.158.102.122            0.0%    60  125.6 122.1  53.0 303.2  35.3
   9.|-- 219.158.39.26              3.3%    60  121.1 125.0  52.2 291.2  42.8
  10.|-- 191.234.80.198             6.7%    60  117.5 118.5  52.4 214.1  23.1
  11.|-- xe-4-0-0-0.hkn-96cbe-1a.n  0.0%    60  140.2 123.0  49.4 297.4  40.8
  12.|-- ???                       100.0    60    0.0   0.0   0.0   0.0   0.0
  13.|-- cds49.hkn9.msecn.net       3.3%    60  112.4 115.9  48.4 307.4  33.5

Google Hosted Libraries

Good latency for a US route (Mountain View), and not a single dropped packet which is rare for Chinese ADSL.

SVG chart of Google Hosted Libraries response times

HOST: deck                        Loss%   Snt   Last   Avg  Best  Wrst StDev
   1.|-- 111.194.0.1                0.0%    10    6.6   7.6   5.6  12.4   2.5
   2.|-- 61.148.149.237             0.0%    10   30.8  22.3   3.1  61.0  19.4
   3.|-- 61.148.6.117               0.0%    10   11.3  12.9   7.0  25.5   6.1
   4.|-- 123.126.0.253              0.0%    10   11.0  10.3   6.9  13.1   1.8
   5.|-- 219.158.101.122            0.0%    10   49.2  50.0  46.9  58.6   3.1
   6.|-- 219.158.11.170             0.0%    10   48.3  47.2  46.1  49.4   0.9
   7.|-- 219.158.97.6               0.0%    10   48.6  48.9  46.9  53.7   1.9
   8.|-- 219.158.3.238              0.0%    10   48.2  50.6  47.9  57.7   2.7
   9.|-- 72.14.215.130              0.0%    10   68.3  61.7  50.9 113.6  19.0
  10.|-- 209.85.248.62              0.0%    10   70.9  67.5  46.8  80.5  14.1
  11.|-- 209.85.253.71              0.0%    10   49.9  55.8  45.8 122.6  23.7
  12.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0
  13.|-- hg-in-f95.1e100.net        0.0%    10   48.0  49.4  47.4  54.5   2.4

Amazon CloudFront

Amazon route me via Japan but the latency is higher that I was expecting.

SVG chart of Amazon CloudFront response times

HOST: deck                        Loss%   Snt   Last   Avg  Best  Wrst StDev
   1.|-- 111.194.0.1                0.0%    10   44.5  35.3   7.2  62.1  20.9
   2.|-- 61.148.149.241             0.0%    10    5.9  12.2   5.6  57.3  15.9
   3.|-- 124.65.59.1                0.0%    10    7.6   8.6   6.9  11.8   1.3
   4.|-- 123.126.0.97               0.0%    10    8.8  11.6   7.7  20.6   3.6
   5.|-- 219.158.100.166            0.0%    10   59.8  58.1  55.8  62.1   1.9
   6.|-- 219.158.11.50              0.0%    10  153.3 128.8 118.4 155.6  13.6
   7.|-- 219.158.97.6               0.0%    10  125.1 125.2 121.7 135.9   4.1
   8.|-- 219.158.33.22              0.0%    10  185.5 184.0 181.3 186.8   2.0
   9.|-- otejbb205.int-gw.kddi.ne.  0.0%    10  179.6 181.4 179.2 189.7   3.0
  10.|-- obpjbb205.int-gw.kddi.ne.  0.0%    10  238.8 213.7 206.8 238.8  10.2
  11.|-- jc-osa302.kddnet.ad.jp     0.0%    10  218.8 211.3 203.5 237.4  10.1
  12.|-- 118.155.203.62            10.0%    10  199.0 200.2 191.0 237.7  14.9
  13.|-- 27.0.0.191                10.0%    10  313.4 310.2 300.2 319.8   6.8
  14.|-- 27.0.0.150                10.0%    10  316.3 312.7 299.8 328.3   9.4
  15.|-- ???                       100.0    10    0.0   0.0   0.0   0.0   0.0
  16.|-- server-54-230-114-37.nrt5 10.0%    10  317.2 312.7 305.7 320.1   5.8

MaxCDN

I included this as it’s the new jQuery official CDN, so should see widespread use. Not bad overall, but still serving out of the US.

SVG chart of MaxCDN response times

HOST: deck                        Loss%   Snt   Last   Avg  Best  Wrst StDev
   1.|-- 111.194.0.1                0.0%    60    5.7  12.9   5.7  67.9  12.3
   2.|-- 61.148.149.237             0.0%    60    6.3  11.4   5.4  72.2  11.9
   3.|-- 123.126.8.161              0.0%    60    7.3  12.1   6.2  70.6  11.3
   4.|-- 123.126.0.185              0.0%    60    9.5  15.0   5.9 143.1  22.3
   5.|-- 219.158.97.190             0.0%    60    9.7  16.0   7.5 104.8  15.5
   6.|-- 219.158.97.178             0.0%    60   13.0  17.8   7.4 102.5  14.4
   7.|-- 219.158.102.154            1.7%    60  190.1 209.4 179.9 336.6  36.5
   8.|-- xe-6-0-3.ar1.lax2.us.nlay  3.3%    60  253.1 271.0 203.5 441.0  45.2
   9.|-- 108.161.181.2             23.3%    60  190.5 207.6 180.4 343.1  30.4
  10.|-- 108.161.188.209            1.7%    60  256.5 271.5 197.1 446.2  45.2

CloudFlare

Another popular choice, but their performance in China is just dire. Mediocre at best and awful at worst, plus more packet loss than anyone else I tested.

SVG chart of CloudFlare response times

HOST: deck                        Loss%   Snt   Last   Avg  Best  Wrst StDev
   1.|-- 111.194.0.1                0.0%    60    7.2  39.5   5.5 938.3 120.8
   2.|-- 61.148.149.241             0.0%    60    6.1  26.4   3.1 871.4 111.8
   3.|-- bt-228-181.bta.net.cn      0.0%    60   11.7  28.4   5.9 808.0 104.0
   4.|-- 124.65.194.21              0.0%    60    9.8  31.6   6.5 741.1  96.7
   5.|-- 219.158.100.170            2.1%    60   55.0  74.6  53.2 720.6  87.9
   6.|-- 219.158.11.22              0.0%    60  118.6 154.0  61.2 1719. 221.8
   7.|-- 219.158.96.214             0.0%    60  124.5 159.7  58.0 1652. 211.8
   8.|-- 219.158.102.158            1.8%    60  346.0 379.8 285.7 1807. 199.9
   9.|-- 144.223.54.221             6.7%    60  333.4 364.9 277.5 1753. 199.4
  10.|-- 144.232.0.194              6.7%    60  337.8 367.6 279.3 1667. 186.7
  11.|-- ae-0.gtt.lsanca03.us.bb.g 16.7%    60  414.3 418.4 288.4 1661. 186.0
  12.|-- ae1-80g.cr1.lax1.us.nlaye  5.0%    60  392.9 411.9 306.4 1592. 165.0
  13.|-- ae2-50g.ar1.lax1.us.nlaye  5.0%    60  401.0 418.3 294.4 1530. 155.2
  14.|-- as13335.xe-9-0-7.ar1.lax1  6.7%    60  410.6 413.5 300.6 1463. 146.4
  15.|-- 198.41.212.162            11.7%    60  398.2 412.9 307.2 1394. 141.7

Akamai

Not a bad showing for Akamai out of Hong Kong, but MS do better.

SVG chart of Akamai response times

HOST: deck                        Loss%   Snt   Last   Avg  Best  Wrst StDev
   1.|-- 111.194.0.1                0.0%    60   19.0  14.6   5.1 104.0  18.2
   2.|-- 61.148.149.237             0.0%    60   34.7  13.6   5.2  97.7  19.9
   3.|-- 123.126.8.161              0.0%    60    7.3  14.4   6.3 101.3  16.1
   4.|-- 123.126.0.169              0.0%    60   13.0  16.4   5.9 104.5  20.0
   5.|-- 219.158.4.142              0.0%    60    8.1  14.2   7.1 109.2  17.4
   6.|-- 219.158.98.2               0.0%    60  110.5  64.6  42.7 159.7  26.1
   7.|-- 219.158.32.238            10.0%    60   80.7  95.6  80.4 174.5  23.8
   8.|-- as-1.r20.tkokhk01.hk.bb.g 91.7%    60  317.6 235.0 139.5 317.6  86.8
   9.|-- ae-1.r03.chwahk02.hk.bb.g  0.0%    60  120.0 140.1 118.7 299.7  45.6
  10.|-- 203.131.246.250            0.0%    60  119.4 133.2 118.3 337.3  34.6
  11.|-- a23-2-16-40.deploy.static  1.7%    60  129.5 143.6 119.7 296.8  44.9

Conclusion

These are obviously flawed and essentially anecdotal results, but they are in line with my experiences over the last two years on domestic ADSL. If you are serious about serving your content in China don’t assume your CDN will “just work” - it almost certainly won’t.

Update: January 2014

The selection of decent China-targetting CDNs is much improved, as are the reporting tools. If you’re shopping around make sure you check out the Cedexis report for China. To demonstrate I ran mtr with the same settings from my original notes on an image on the ChinaCache CDN:

HOST: ophelia                     Loss%   Snt   Last   Avg  Best  Wrst StDev
   1.|-- 111.194.0.1                0.0%    60    6.6  29.4   5.3 119.5  28.8
   2.|-- 61.148.149.237             0.0%    60    6.2  13.0   5.4  98.1  16.3
   3.|-- 123.126.8.201              0.0%    60   10.4  17.0   7.1 127.1  18.4
   4.|-- ???                       100.0    60    0.0   0.0   0.0   0.0   0.0
   5.|-- 219.158.96.106             0.0%    60   15.5  20.1  10.9  86.3  14.1
   6.|-- www.he.cninfo.net          0.0%    60   14.3  17.8  10.7 103.4  17.4
   7.|-- 61.182.174.178             0.0%    60   18.6  27.9  18.3 185.2  25.2
   8.|-- 221.192.129.185            0.0%    60   20.5  27.8  19.7 103.3  17.8
   9.|-- 211.90.1.74                0.0%    60   20.1  27.2  18.7  86.7  13.7
  10.|-- 221.192.149.107            0.0%    60   18.9  25.3  17.7 106.6  16.7

I also took the opportunity to redo my graphs as I recently did some work generating SVG graphs from mtr reports. Read those notes if you would like to script gnuplot to process mtr reports as I’ve done here.

Other notes

2016

2014

2013

2012

2011

2010